Key takeaways
- Context windows combine prompt and completion, so long histories drain limits faster than expected.
- Streaming requires handling SSE events carefully to avoid corrupted JSON or lost tokens.
- JSON mode demands strict schema adherence; malformed responses cause silent parsing failures.
- Prepaid crypto billing means errors are free, but token counting must be precise to avoid surprises.
Why your API key might be rejected
API keys get rejected when the client sends malformed headers or exceeds request body limits. The server validates the authentication header before processing any logic. If the header is missing or contains extra whitespace, the request fails immediately.
Another common cause is sending a payload that exceeds the 8 MB body size limit. Large context windows with many messages can easily surpass this threshold. Check your message array length and token count before sending.
- Ensure the Authorization header uses the 'Bearer' prefix correctly.
- Verify the request body size stays under 8 MB.
- Check that the API key is active and not expired.
Understanding context window limits
The context window includes both the input prompt and the generated completion. Our API supports a 100,000 token window, but you must account for both sides. If you send 80,000 tokens of context, you only have 20,000 left for the response.
Most models default to a small output limit if you do not specify max_tokens. This can truncate long answers unexpectedly. Set max_tokens to your desired length only if you want a cap. Remember that the limit applies to the total request, not just the prompt.
Handling streaming interruptions correctly
Streaming via Server-Sent Events (SSE) requires robust error handling. If the connection drops, you might lose the final token or the usage metadata. Always buffer incoming chunks and reconstruct the full response on the client side.
Token usage data is typically sent in the last chunk of the stream. If you close the connection early, you might miss the exact token count. This affects billing accuracy in prepaid systems. Implement a retry mechanism with exponential backoff to handle transient network issues.
Use a dedicated streaming client that handles SSE parsing automatically. This reduces the risk of parsing errors and ensures you capture all data.
JSON mode parsing errors
When using response_format: {"type": "json_object"}, the model must return valid JSON. Any deviation causes parsing errors in your application. Ensure your prompt explicitly instructs the model to output only JSON.
Sometimes the model includes markdown code blocks around the JSON, which breaks strict parsers. Remove wrappers before parsing. Also, validate the JSON structure on the client side before using the data.
Use a schema validation library to catch errors early. This prevents crashes when the model returns unexpected keys or types.
Token counting and billing surprises
Prepaid billing charges only for real token usage. Errors and refusals are free, which is a significant advantage over subscription models. However, token counting can be tricky if you do not account for the entire context.
Each message in the conversation history counts towards the input token limit. Long conversations drain your prepaid credit faster than you might expect. Monitor your usage regularly to avoid running out of credits mid-session.
Set up alerts for low credit balances. This ensures you can top up before your requests fail. Remember that credits never expire, so you can top up whenever convenient.
Avoiding soft refusals in NSFW prompts
Soft refusals occur when the model hesitates or provides a partial answer instead of a direct response. This often happens with controversial or adult topics. Even in an uncensored model, certain triggers can cause hesitation.
Adjust the temperature parameter to reduce randomness. A lower temperature makes the model more deterministic and less likely to hedge. Also, ensure your prompt is clear and direct. Avoid ambiguous language that might confuse the model.
If you encounter frequent soft refusals, try rephrasing the prompt slightly. Small changes in wording can often bypass the hesitation and get a direct answer.
Rate limit spikes and backoff strategies
The API allows 300 requests per minute and 8 concurrent requests per key. Spikes in traffic can quickly hit these limits. Implement exponential backoff to handle rate limit errors gracefully.
When you receive a 429 status code, wait before retrying. Start with a short delay and increase it with each retry. This prevents overwhelming the server during peak times.
Monitor your request rate and adjust your concurrency levels accordingly. If you are making many simultaneous requests, consider spreading them out over time. This ensures steady throughput without hitting limits.
Function calling payload structures
Function calling requires a specific payload structure. Define your functions in the tools parameter and specify the tool_choice. The model will return a function call if it determines that is the appropriate response.
Ensure your function definitions are accurate and complete. Missing parameters or incorrect types can cause the model to generate invalid calls. Always validate the function call arguments before executing them.
Use the tool_choice parameter to control when the model calls functions. You can force a function call, let the model decide, or disable function calling entirely. This gives you fine-grained control over the model's behavior.
Debugging 400 vs 500 errors
400 errors indicate client-side issues, such as invalid JSON or missing parameters. 500 errors suggest server-side problems, like internal errors or timeouts. Distinguishing between the two helps you debug faster.
For 400 errors, check the error message provided in the response. It usually points directly to the issue. For 500 errors, retry the request after a short delay. If the error persists, contact support with the request ID.
Log both request and response details for each error. This helps identify patterns and recurring issues. Use a logging service to track errors over time and improve your API integration.
