Praison AI

Rate Limiter

Cap API request rate and token usage across agents and threads

Cap LLM call rate so agents stay within provider quotas and budget — safely, even when many agents share one limiter.

python
from praisonaiagents import Agent, ExecutionConfig

agent = Agent(
    name="Researcher",
    instructions="Research topics concisely.",
    execution=ExecutionConfig(max_rpm=60),
)

agent.start("Summarise the latest Mars rover news")

The user runs agents at scale; the limiter queues or throttles LLM calls so quotas are not exceeded.

Diagram

Allow

Wait

Agent 1

RateLimiter

Agent 2

LLM API

Queue

How It Works

Diagram
RateLimiterAgentUserRateLimiterAgentUserRequestProcessResultResponse

Quick Start

  1. Simple Usage

    Set max_rpm on execution config for a single agent:

    python
    from praisonaiagents import Agent, ExecutionConfig

agent = Agent(
    name="Researcher",
    instructions="You research topics on the web.",
    execution=ExecutionConfig(max_rpm=60),
)

agent.start("Summarise the latest Mars rover news")

    max_rpm=60 alone is enough — the Agent auto-creates a live RateLimiter(requests_per_minute=60) behind the scenes. No RateLimiter import required for the simple case.

  2. With Configuration

    Share one RateLimiter across multiple agents:

    python
    from praisonaiagents import Agent, AgentTeam, ExecutionConfig
from praisonaiagents.llm import RateLimiter

shared = RateLimiter(requests_per_minute=60, burst=5)

researcher = Agent(
    name="Researcher",
    instructions="Research topics",
    execution=ExecutionConfig(rate_limiter=shared),
)
writer = Agent(
    name="Writer",
    instructions="Write articles",
    execution=ExecutionConfig(rate_limiter=shared),
)

AgentTeam(agents=[researcher, writer]).start()

How It Works

Diagram
Rate LimiterAgentUserRate LimiterAgentUserRequestProcess requestResultResponse

Token bucket algorithm: tokens refill at requests_per_minute / 60 per second; each LLM call consumes one token. Under contention, callers wait until a token is available.

The limiter applies to both the initial LLM call and the follow-up after tool execution in streaming mode.

The rate limiter applies to every LLM call the agent issues — single-shot, streaming, tool-iteration and reflection turns, sync and async — so a per-model token budget is never spent on a path that bypasses throttling.

StepWhat happens
RefillTokens regenerate on elapsed time
AcquireCaller reserves a token (thread-safe)
WaitSleeps or awaits when bucket is empty
ReleaseAutomatic — rolling window, no explicit release

Precedence

The Agent picks a limiter from three levels: an explicit rate_limiter wins, else max_rpm auto-builds one, else no throttling.

rate_limiter (explicit) > max_rpm (auto-built) > no limiter

max_rpm <= 0 short-circuits and raises ValueError at construction — regardless of whether an explicit rate_limiter was also supplied, because validation runs before the precedence check.

max_rpmrate_limiterResult
NoneNoneNo limiter — zero overhead
N > 0NoneAgent auto-builds RateLimiter(requests_per_minute=N)
NoneobjectExplicit limiter used as-is
N > 0objectExplicit limiter wins; max_rpm still recorded on the Agent
N <= 0any (including object)ValueError at construction
Diagram

Yes

No

Yes

No

Need throttling?

Have a
RateLimiter
object?

Pass rate_limiter=
takes precedence

Simple
RPM cap?

Pass max_rpm=N
auto-builds limiter

Leave both unset
zero overhead

Configuration Options

OptionTypeDefaultDescription
max_rpmintNonePositive int auto-builds RateLimiter(requests_per_minute=max_rpm) when no rate_limiter is passed; explicit rate_limiter takes precedence; max_rpm <= 0 raises ValueError (validated before the precedence check, so max_rpm=0 still raises even with a rate_limiter)
requests_per_minuteintNoneMax requests per rolling 60s window
tokens_per_minuteintNoneToken budget for TPM-quoted providers
burstint1Back-to-back requests before rate kicks in
max_retry_delayint120Max wait seconds when rate limited

Errors

Zero or negative max_rpm raises ValueError at construction, before the explicit-limiter precedence check.

python
from praisonaiagents import Agent, ExecutionConfig
from praisonaiagents.llm import RateLimiter

# All three raise ValueError at construction:
Agent(name="a", instructions="x", execution=ExecutionConfig(max_rpm=0))
Agent(name="a", instructions="x", execution=ExecutionConfig(max_rpm=-5))
Agent(name="a", instructions="x",
      execution=ExecutionConfig(max_rpm=0, rate_limiter=RateLimiter(requests_per_minute=60)))

YAML

max_rpm in agents.yaml now throttles requests — it auto-creates a live RateLimiter when no limiter is set on the agent, and raises ValueError on max_rpm <= 0.

yaml
agents:
  assistant:
    name: Assistant
    instructions: Be brief.
    llm: gpt-4o-mini
    max_rpm: 20      # now throttles at 20 rpm

Best Practices

Share one limiter across related agents

When multiple agents use the same API key, pass the same RateLimiter so combined throughput stays in quota.

Match burst to your workload

Low burst (1–5) smooths traffic; higher burst tolerates spiky demand.

Set tokens_per_minute for TPM limits

Providers quote RPM and TPM — limiting only RPM can still trigger 429 errors.

Use async paths in async flows

agent.achat() / astart() / arun() call acquire_async() automatically, and sync chat() calls acquire() — sync and async entry points are throttled equally on both direct-LLM and managed-backend calls, so pick whichever fits your event loop.

CLI

bash
praisonai "task" --rpm 60

Related

Thread Safety

Thread-safe chat history and caches

 Concurrency

Limit parallel agent runs