Rate limits & quotas

Cloud model providers enforce rate limits — understand when they trigger, how to cope, and why local models are entirely unaffected.

What rate limiting is

Cloud model providers cap the number of requests per time window to manage load and prevent abuse. Agent work is request-dense, so with a cloud provider you may occasionally hit the limit — requests get rejected or throttled.

How to cope

• Upgrade your plan: frequent throttling means your usage outgrew the tier; higher tiers carry higher rate caps;
• Lower request density: split large jobs into several sessions; pause briefly between tasks;
• Consider providers with built-in rate management: some aggregators (e.g. OpenRouter) offer a multi-model entry point that can switch models automatically near the cap, reducing interruptions;
• Just retry: most limits are transient — waiting a moment and retrying the same request works.

Local models: zero rate limits

Local models are not rate-limited at all — inference runs on your own GPU: no API, no quota, no usage billing. If your work is sensitive to request density (bulk processing, many small calls), moving it to a local model is the fundamental fix.

The two model kinds switch instantly (see Local models & providers): everyday chats and sensitive tasks on local, heavy reasoning on cloud — mix freely per task.

This page is adapted from upstream open-source documentation (Apache-2.0) with modifications; see /legal/provenance for sources and changes.