> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Quota Coordinator

> Share credential cooldowns across replicas so a benched API key is skipped fleet-wide

The Quota Coordinator lets multiple replicas share credential cooldowns — when one replica benches an API key after a 429, every replica skips it until the cooldown expires.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Fleet"
        R1[🖥️ Replica A] --> C[🧠 Coordinator]
        R2[🖥️ Replica B] --> C
        R3[🖥️ Replica C] --> C
    end
    C --> K[🔑 Benched Keys<br/>with TTL]

    classDef replica fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef coord fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef store fill:#189AB4,stroke:#7C90A0,color:#fff

    class R1,R2,R3 replica
    class C coord
    class K store
```

<Note>
  The Quota Coordinator symbols live in `praisonaiagents.llm`: `from praisonaiagents.llm import LocalQuotaCoordinator`. `Agent`, `AuthProfile`, and `FailoverManager` import from the top-level `praisonaiagents` package.
</Note>

## Quick Start

<Steps>
  <Step title="Default (single replica)">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent, AuthProfile, FailoverManager

    # No coordinator argument = the default in-memory coordinator.
    # Single-replica behaviour is exactly as before.
    manager = FailoverManager()
    manager.add_profile(AuthProfile(name="openai", provider="openai", api_key="sk-..."))

    agent = Agent(name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager})
    agent.start("Hello!")
    ```
  </Step>

  <Step title="Attach a coordinator">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent, AuthProfile, FailoverManager
    from praisonaiagents.llm import LocalQuotaCoordinator

    coordinator = LocalQuotaCoordinator()
    manager = FailoverManager(coordinator=coordinator)
    manager.add_profile(AuthProfile(name="openai", provider="openai", api_key="sk-..."))

    agent = Agent(name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager})
    ```
  </Step>

  <Step title="Share across managers in one process">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import AuthProfile, FailoverManager
    from praisonaiagents.llm import LocalQuotaCoordinator

    shared = LocalQuotaCoordinator()

    # Two independent failover managers now share cooldowns.
    mgr_a = FailoverManager(coordinator=shared)
    mgr_b = FailoverManager(coordinator=shared)

    mgr_a.add_profile(AuthProfile(name="key1", provider="openai", api_key="sk-shared"))
    mgr_b.add_profile(AuthProfile(name="key1", provider="openai", api_key="sk-shared"))
    ```
  </Step>

  <Step title="Bring your own backend (replicas > 1)">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import FailoverManager
    from praisonaiagents.llm import QuotaCoordinatorProtocol

    # A concrete Redis-backed coordinator lives in the wrapper/bot package and
    # implements the same protocol. Core stays dependency-free.
    manager = FailoverManager(coordinator=my_redis_coordinator)
    ```
  </Step>
</Steps>

***

## How It Works

Replica A hits a 429, benches the credential, then replica B skips the benched key based on the shared coordinator state.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant ReplicaA
    participant Coordinator
    participant ReplicaB

    ReplicaA->>Coordinator: bench(cred_id, until=t+60, reason="429")
    Note over Coordinator: TTL-based entry stored
    ReplicaB->>Coordinator: benched_until(cred_id)
    Coordinator-->>ReplicaB: t+60
    ReplicaB->>ReplicaB: Skip this profile (RATE_LIMITED)
    Note over Coordinator: At t+60 the bench self-expires
```

| Step | What happens |
| - | - |
| Bench | Replica A calls `bench(cred_id, until=t+60)` on a rate-limit failure. |
| Sync | Replica B calls `get_next_profile()`, which reads `benched_until(cred_id)`. |
| Skip | Replica B marks its local profile `RATE_LIMITED` and skips it. |
| Expire | At `t+60` the bench self-clears; the key returns to rotation fleet-wide. |

***

## Choosing a Backend

Pick a coordinator based on how many replicas run and whether you have a shared store.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{How many<br/>replicas?} -->|1| L[LocalQuotaCoordinator<br/>default, zero setup]
    Q -->|N > 1| SharedQ{Have Redis /<br/>shared store?}
    SharedQ -->|Yes| R[Custom coordinator<br/>implementing the protocol]
    SharedQ -->|No| L2[LocalQuotaCoordinator<br/>each replica meters privately]

    classDef choice fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef local fill:#10B981,stroke:#7C90A0,color:#fff
    classDef redis fill:#189AB4,stroke:#7C90A0,color:#fff

    class Q,SharedQ choice
    class L,L2 local
    class R redis
```

***

## Configuration Options

`QuotaCoordinatorConfig` selects the backend.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import QuotaCoordinatorConfig, build_quota_coordinator

config = QuotaCoordinatorConfig(backend="local")
coordinator = build_quota_coordinator(config)
```

| Option | Type | Default | Description |
| - | - | - | - |
| `backend` | `str` | `"local"` | `"local"` (in-memory, default) or a shared backend name such as `"redis"` provided by the wrapper/bot package. |
| `url` | `Optional[str]` | `None` | Optional backend URL. When omitted a shared backend reuses the already-established connection (e.g. gateway `RedisConfig`). |

`QuotaCoordinatorProtocol` defines the contract every coordinator implements.

| Method | Signature | Purpose |
| - | - | - |
| `bench` | `bench(cred_id, *, until, reason="")` | Bench a credential until an epoch timestamp (TTL-based). |
| `is_benched` | `is_benched(cred_id, *, now=None) -> bool` | Return True if the credential is currently benched. |
| `benched_until` | `benched_until(cred_id, *, now=None) -> Optional[float]` | Return the epoch timestamp the credential is benched until, or None. |
| `clear` | `clear(cred_id) -> None` | Clear any bench for the credential (e.g. on recovery). |

***

## Key Behaviours

Each behaviour is enforced by the SDK and covered by tests.

| Behaviour | What it means |
| - | - |
| **Fail-open by contract** | Any coordinator error is logged as a warning and treated as "not benched" so the failover path is never wedged. |
| **TTL semantics** | Bench entries carry an expiry — a dead replica's benches self-expire. |
| **Longest-cooldown wins** | `bench()` on an already-benched credential keeps the later expiry, never shortens it. |
| **Hashed credential id** | `AuthProfile.credential_id` derives a stable, non-secret 16-char SHA-256 digest from `provider + base_url + api_key`. Two profiles named `"default"` with different keys will not collide. |
| **Stale success is safe** | `mark_success` clears the shared bench only when it is not newer than the cooldown being recovered from. |
| **Backward-compatible default** | `FailoverManager()` with no `coordinator` creates a `LocalQuotaCoordinator` internally — single-replica behaviour is unchanged. |

<Warning>
  Never wrap coordinator calls in your own catch/retry blocks. A coordinator outage degrades to local behaviour automatically — extra retries only add latency.
</Warning>

***

## Credential Identity

Benches key on the credential, not the profile label, so two profiles named `"default"` with different keys stay independent.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import AuthProfile

p1 = AuthProfile(name="default", provider="openai", api_key="key-1")
p2 = AuthProfile(name="default", provider="openai", api_key="key-2")

# Distinct, stable, non-secret ids — the raw key is never exposed.
assert p1.credential_id != p2.credential_id
```

***

## Operator resets vs. scoped recoveries

Two paths mutate the shared bench — a scoped per-call recovery and an operator-initiated hard override.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start[Need to clear a bench?] --> Q1{Recovering from<br/>a specific call?}
    Q1 -->|Yes| MS[mark_success profile]
    Q1 -->|No| Q2{Hard override<br/>operator action?}
    Q2 -->|Yes| RA[reset_all]
    Q2 -->|No| Wait[Let cooldown expire]

    classDef start fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef action fill:#10B981,stroke:#7C90A0,color:#fff
    classDef wait fill:#6366F1,stroke:#7C90A0,color:#fff

    class Start start
    class Q1,Q2 decision
    class MS,RA action
    class Wait wait
```

| Action | Scope | When to use |
| - | - | - |
| `mark_success(profile)` | **Scoped recovery.** Only clears the shared bench when it is not newer than the local cooldown being recovered from. A concurrent fleet-wide bench is preserved. | After a successful call that recovered the credential. |
| `reset_all()` | **Hard override.** Clears every profile's shared bench unconditionally. Also clears benches other replicas recorded. | After rotating a key, after a known fleet-wide incident, during maintenance. |

<Note>
  **Fail-open with retry.** If the coordinator is briefly unavailable when `reset_all()` runs, it retries once. If it still fails, the local reset is applied and a warning is logged — the shared bench may persist until it expires naturally or until another call clears it.
</Note>

***

## Best Practices

<AccordionGroup>
  <Accordion title="Default is safe">
    Single-replica or single-process deployments do not need to set anything — leave `coordinator` unset and `FailoverManager` uses a private in-memory coordinator.
  </Accordion>

  <Accordion title="Share across agents in the same process">
    Pass one `LocalQuotaCoordinator` to every `FailoverManager` in the process so all agents observe each other's benches.
  </Accordion>

  <Accordion title="Bring your own backend for replicas > 1">
    The concrete Redis coordinator ships in the wrapper/bot package to keep core dependency-free. Wire it up via config and pass the instance to `FailoverManager(coordinator=...)`.
  </Accordion>

  <Accordion title="Trust the fail-open">
    A coordinator outage logs a warning and degrades to local behaviour — never write catch/retry blocks around coordinator calls at the call site.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card title="Failover" icon="rotate" href="/docs/features/failover">
    Automatic fallback between LLM providers
  </Card>

  <Card title="Rate Limiter" icon="gauge" href="/docs/features/rate-limiter">
    Throttle requests to stay within provider limits
  </Card>

  <Card title="LLM Error Classification" icon="triangle-exclamation" href="/docs/features/llm-error-classification">
    Typed errors that drive failover decisions
  </Card>

  <Card title="Thread Safety" icon="lock" href="/docs/features/thread-safety">
    Share a credential pool across concurrent agents
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.