Rate Limits
The admission caps on your team, how they are counted, and how to raise them.
Every team has admission caps: throughput (tokens per minute), requests per minute, and concurrent requests. They exist so one caller cannot flood the pool. Your team's limits are shown on the usage page.
How they are counted#
- The window is a rolling 60 seconds.
- Cached input does not count. Prompt tokens served from the gateway's prefix cache are subtracted first, so reusing a long conversation counts only the new tokens. An agent loop that keeps its prefix stable fits far more turns into the same cap than the raw counts suggest. Cached tokens are still billed at the cached rate, so they cost money even though the cap ignores them.
- The caps are team-wide. Every key draws on the same budget, so five keys do not get five times the throughput.
Because a request counts only once it completes, no request is refused for its own size. The default cap covers the largest context any model advertises plus room for the reply, so a full-context turn fits inside one window.
When you go over#
A response on a capped key carries the active policy and remaining capacity:
q = quota, w = window in seconds, r = remaining after this request, t = seconds to full reset. Concurrency has no time window. The values are advisory: pace against them, but don't treat them as a contract.
Over-limit requests return HTTP 429 and are not charged. RateLimit names the dimension with r=0; Retry-After says when to try again.
| What happened | What to do |
|---|---|
RateLimit says requests |
Wait for Retry-After, or send fewer requests per minute |
RateLimit says throughput |
Wait for Retry-After, set a realistic max_tokens, or send fewer tokens per minute |
RateLimit says concurrency |
Wait for an in-flight request to finish, or lower concurrency |
Credits exhausted (budget_exceeded) |
Top up. There is no Retry-After, because waiting will not help |
No RateLimit field |
Gateway briefly at capacity; follow Retry-After |
Handling them#
- Back off and retry. The window is 60 seconds, so a short exponential backoff clears an ordinary overshoot. OpenAI SDKs retry 429s by default.
- Limit concurrency instead of retrying harder. Requests count as they complete, so a burst can pass the check together and push the window over the cap all at once. The same volume sent one request at a time stays under it.
- Keep your prompt prefix stable. A conversation whose opening is reused verbatim is served from cache, and cached input costs nothing against the cap.
Anonymous agent key limits#
An anonymous agent key is not on a team. It has its own per-key caps of 6 requests and 30,000 tokens per minute, and a $0.01 budget.
Minting is limited to 3 keys an hour per caller.
Raising your limit#
Limits are raised per team. Email support with your account email and a rough throughput target.