Rate Limits & Quotas

Both are per-application policy, set via the Management API (rateLimitPerMinute, monthlyTokenQuota). null means unlimited for either — enforcement is skipped entirely rather than checked against an infinite limit.

Rate limiting#

A fixed 60-second window counter, checked on every /v1/** request after authentication:

Authorization: Bearer lattice_sk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

The window is a clock-aligned bucket (e.g. 12:00:00–12:00:59), not a rolling window from the first request. Each request atomically increments the counter for the application's current window and compares it to rateLimitPerMinute; exceeding it returns 429 rate_limit_exceeded before the request reaches routing or dispatch. The counter resets automatically at the next window boundary — a new (application_id, window_start) row starts at 0.

Monthly token quota#

Enforced before dispatch, as a conservative reservation rather than a post-hoc check:

1. The request's likely token cost is estimated as min(max_tokens ?? 4096, 4096) — an upper bound, since the real cost is only known once the provider responds.
2. That estimate is atomically reserved against the application's current-month counter, in the same conditional write used to check the limit — so two concurrent requests can't both read a stale "under quota" count and both proceed past a limit neither of them alone would have exceeded.
3. If the reservation would exceed monthlyTokenQuota, the request fails immediately with 429 quota_exceeded — no provider is ever called.
4. If the request completes, the reservation is reconciled down to the real token usage the provider reported (refunding the difference between the estimate and the actual cost).
5. If the request fails for any other reason after the reservation was taken, the full reservation is refunded.

The quota period is calendar-month, UTC. There's no rollover and no partial-month proration — a quota resets at the start of each month regardless of when in the previous month it was set.

Checking current usage#

Both counters can be inspected without waiting for a 429 — see the Management API's usage endpoints for per-application and gateway-wide summaries, or the Dashboard for a visual view.

Designing a client around these limits#

• Treat 429 rate_limit_exceeded as retryable after a short backoff (the current window is at most 60 seconds).
• Treat 429 quota_exceeded as not retryable by the client itself — it needs either a quota increase (an admin action) or to wait for the next calendar month.
• If your application makes bursty, high-max_tokens requests, expect the quota reservation's conservative estimate (capped at 4096) to sometimes reserve more than the request ends up actually costing; this is refunded automatically once the response completes, but a burst of concurrent requests can transiently reserve more than their eventual real total.