Rate limits
What is rate limited, what is not, and how to retry safely.
zrouter keeps the inference path deliberately unthrottled while applying clear, documented limits to the higher-risk administrative surfaces. This page explains each boundary and how to handle limit responses correctly.
Inference rate limits#
If you see a 429 on an inference endpoint, it is not a zrouter per-client rate limit. It is a passthrough of the upstream's technical_limit response, forwarded to you as-is. Full details are covered in the Error Handling section.
Reseller rate limits#
The reseller program enforces rate limits on its administrative endpoints. Exceeding a limit returns 429 rate_limited:
| Operation | Limit |
|---|---|
| Login (sign-in / sign-up) | 5 requests per minute per email + IP |
| Top-up creation | 10 requests per minute |
| Mint | 30 requests per minute |
These limits protect the reseller program's prepaid-quota accounting. For the reseller API contract, see the Reseller Program Overview.
Timeouts#
Non-streaming inference requests have a 120-second server-side timeout, applied to the round trip against the upstream. The exact value is configurable per deployment, but it is fixed for a given environment — the gateway does not let a single hung upstream hold a connection open indefinitely.
Streaming requests are not subject to this limit: once a stream starts, the gateway relays events for the duration of generation (with a keepalive to prevent idle disconnects).
Retry best practices#
Transient failures do happen — an upstream hiccup, a network blip, a timeout. Retry them, but do it carefully:
| Practice | Why |
|---|---|
Use exponential backoff with jitter on 429, 5xx, and timeouts | Spacing and randomizing retries avoids thundering-herd contention on transient upstream errors. |
| Retry only idempotent requests | A non-streaming chat completion is safe to retry. Once a stream has begun, do not retry it — the generation is already in flight and settling the retry could double-charge usage. |
| Set your client timeout slightly below 120s | Letting your client give up just before the server timeout means you see a clean error instead of an unexpected disconnect. |
| Reuse connections and enable keep-alive | Connection reuse avoids the overhead of re-establishing TLS on every request and reduces tail latency. |
| Cap retries at 2–3 attempts | More attempts rarely help against a persistent outage and can amplify load during one. |