---
title: "Rate Limits"
description: "The throughput cap on your team, and how to raise it"
canonical: "https://inference.boundless.network/docs/rate-limits"
last-updated: "2018-10-20T01:46:40.000Z"
---

# Rate Limits

The throughput cap on your team, how it is counted, and how to raise it.


Every team has a throughput cap, measured in **tokens per minute**. It exists so one team cannot flood the pool and slow everyone else down.

Your team's limit is shown on the [usage page](/usage) in the console. Limits are set per team and can be raised, so the console is where your current number lives.

## How it is counted

- **The window is 60 seconds and rolls continuously.** It does not reset on the minute.
- **A request is counted before it runs.** The gateway admits it against `prompt_tokens + max_tokens`, what you sent plus the most you could get back, and checks that before dispatching anything.
- **The cap is team-wide.** Every key in your team draws on the same budget, so splitting work across five keys does not buy five times the throughput.

Because admission counts what a request *could* return, your limit is also the largest single request you can make. A request whose prompt plus `max_tokens` exceeds your limit is never admissible, however idle the team is.

**Set `max_tokens`.** If you omit it, we count the model's maximum for that request, which is substantially more than a typical completion returns. A realistic `max_tokens` fits more requests inside your limit.

## When you go over

Requests over the limit return **HTTP 429** and are **not charged**. The request is refused rather than queued, so you can retry it. The refusal carries a `Retry-After` header saying how many seconds to wait.

In the console, the [request log](/usage) marks these rows so you can tell them apart from requests that failed for other reasons.

A 429 does not always mean your rate limit. Two other things return the same status:

| What happened | What to do |
| --- | --- |
| You are over your rate limit | Retry after a moment, or lower concurrency |
| Your credits are exhausted (`budget_exceeded`) | [Top up](/docs/pricing#topping-up); retrying will not help |
| The gateway is briefly at capacity | Retry after a moment |

## Handling them

- **Back off and retry.** The window is 60 seconds, so a short exponential backoff clears an ordinary overshoot. The official OpenAI SDKs retry 429s for you by default.
- **Limit concurrency** rather than retrying harder. The cap counts tokens in flight per minute, so ten parallel requests carry the same volume as ten sequential ones but arrive at once.
- **Send a realistic `max_tokens`**, as above.

## The agent key's own limits

An anonymous [agent key](/docs/quickstart) is not on a team, so the numbers above are not its numbers. It carries its own per-key caps — 6 requests and 30,000 tokens per minute — and a $0.01 budget that stops it for good rather than for a minute. A 429 on an agent key means one of the two: back off for the per-minute caps, or stop, because the cent is spent and no retry recovers it.

Minting is limited too: 3 keys an hour per caller. A 429 from the mint carries `Retry-After`, and retrying harder cannot beat it.

## Raising your limit

Limits are raised per team. [Email support](/support) with your account email and a rough idea of the throughput you need. A raise is durable: it is set on your team, and later changes to our defaults do not reset it.