> ## Documentation Index > Fetch the complete documentation index at: https://praison.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Failover > Automatic fallback between LLM providers for reliability and cost optimization Failover automatically switches between LLM providers when one fails, keeping agents running during API outages or rate limits. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager manager = FailoverManager(FailoverConfig(max_retries=3)) manager.add_profile(AuthProfile( name="openai", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=1, )) manager.add_profile(AuthProfile( name="anthropic", provider="anthropic", api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2, )) agent = Agent(name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager}) agent.start("Hello!") ``` The user sends a message; if the primary LLM profile fails, the agent tries the next profile in the failover chain. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph LR subgraph "Failover Chain" A[🤖 Agent] --> P1[🟢 Primary] P1 -->|Fail| P2[🟡 Secondary] P2 -->|Fail| P3[🔵 Tertiary] P3 --> R[✅ Response] end classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff classDef primary fill:#10B981,stroke:#7C90A0,color:#fff classDef secondary fill:#F59E0B,stroke:#7C90A0,color:#fff classDef tertiary fill:#6366F1,stroke:#7C90A0,color:#fff classDef result fill:#10B981,stroke:#7C90A0,color:#fff class A agent class P1 primary class P2 secondary class P3 tertiary class R result classDef tool fill:#189AB4,color:#fff classDef agent fill:#8B0000,color:#fff ``` Failover integrates with [LLM Error Classification](/docs/features/llm-error-classification) via `FailoverDecision`, which coordinates profile rotation with typed error handling. ## Quick Start ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager manager = FailoverManager(FailoverConfig(max_retries=3, exponential_backoff=True)) manager.add_profile(AuthProfile( name="openai", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=1, )) manager.add_profile(AuthProfile( name="anthropic", provider="anthropic", api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2, )) agent = Agent( name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager}, ) agent.start("Hello!") ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import AuthProfile, FailoverManager manager = FailoverManager() manager.add_profile(AuthProfile( name="groq", provider="groq", api_key=os.getenv("GROQ_API_KEY"), priority=3, )) ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} status = manager.status() for name, info in status.items(): print(f"{name}: {info['status']}") ``` *** ## How failover activates during retries Failover now drives LLM retries through direct integration with the retry mechanism: ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} sequenceDiagram participant User participant Agent participant LLM participant FailoverManager participant Provider A participant Provider B User->>Agent: Request Agent->>LLM: Call LLM->>FailoverManager: get_next_profile() FailoverManager-->>LLM: Provider A profile LLM->>LLM: Apply profile (api_key, base_url, model) LLM->>Provider A: API call Provider A--xLLM: Rate limit error LLM->>FailoverManager: mark_failure(profile, error, is_rate_limit=True) LLM->>FailoverManager: get_next_profile() FailoverManager-->>LLM: Provider B profile LLM->>LLM: Switch api_key, base_url, model LLM->>Provider B: Retry call Provider B-->>LLM: Success LLM->>FailoverManager: mark_success(profile) LLM-->>Agent: Response Agent-->>User: Final response ``` * On every LLM call, the system first gets the current profile via `get_next_profile()` and applies its `api_key`, `base_url`, and `model` settings * On success, `mark_success(profile)` is called to track the working provider * On failure, `mark_failure(profile, error, is_rate_limit=...)` marks the provider as failed, then `get_next_profile()` fetches the next available provider * Profile switching **overrides** non-retryable classification—one extra attempt is always granted after switching providers * The LLM automatically updates request parameters (api\_key, base\_url, model) when switching between profiles *** ## How It Works ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} sequenceDiagram participant Agent participant Manager participant Primary participant Secondary Agent->>Manager: Request Manager->>Primary: Try Primary Primary--xManager: Rate Limited Manager->>Manager: Wait + Backoff Manager->>Secondary: Try Secondary Secondary-->>Manager: Success Manager-->>Agent: Response ``` | Component | Role | | ------------------- | --------------------------------- | | **AuthProfile** | Credentials for a single provider | | **FailoverManager** | Orchestrates failover logic | | **FailoverConfig** | Retry and backoff settings | | **ProviderStatus** | Tracks provider health | *** ## Configuration Options Manager class reference Provider credential profile ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import FailoverConfig config = FailoverConfig( max_retries=3, retry_delay=1.0, exponential_backoff=True, max_retry_delay=60.0, ) ``` | Option | Type | Default | Description | | ------------------------ | ------- | ------- | ----------------------------- | | `max_retries` | `int` | `3` | Maximum retry attempts | | `retry_delay` | `float` | `1.0` | Initial retry delay | | `exponential_backoff` | `bool` | `True` | Use exponential backoff | | `max_retry_delay` | `float` | `60.0` | Maximum retry delay | | `cooldown_on_rate_limit` | `float` | `60.0` | Rate limit cooldown (seconds) | | `cooldown_on_error` | `float` | `30.0` | Error cooldown (seconds) | | `rotate_on_success` | `bool` | `False` | Rotate profiles on success | *** ## Auth Profiles Configure credentials for each provider: ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import AuthProfile profile = AuthProfile( name="openai-primary", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=1, rate_limit_rpm=100, ) ``` | Field | Type | Description | | ---------------- | ------ | ------------------------------------------- | | `name` | `str` | Unique profile identifier | | `provider` | `str` | Provider: openai, anthropic, etc. | | `api_key` | `str` | API key (masked in logs) | | `base_url` | `str` | Custom API endpoint | | `model` | `str` | Default model for this profile | | `priority` | `int` | Failover priority (lower = higher priority) | | `rate_limit_rpm` | `int` | Requests per minute limit | | `rate_limit_tpm` | `int` | Tokens per minute limit | | `metadata` | `dict` | Additional provider-specific config | *** ## Common Patterns ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import AuthProfile, FailoverManager manager = FailoverManager() # Add multiple providers manager.add_profile(AuthProfile( name="openai", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=1, )) manager.add_profile(AuthProfile( name="anthropic", provider="anthropic", api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2, )) manager.add_profile(AuthProfile( name="groq", provider="groq", api_key=os.getenv("GROQ_API_KEY"), priority=3, )) ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import AuthProfile, FailoverManager manager = FailoverManager() manager.add_profile(AuthProfile( name="gpt-4o-mini", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=1, model="gpt-4o-mini", )) manager.add_profile(AuthProfile( name="gpt-4o", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), priority=2, model="gpt-4o", )) ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import AuthProfile, FailoverManager manager = FailoverManager() manager.add_profile(AuthProfile( name="openai-us", provider="openai", api_key=os.getenv("OPENAI_API_KEY"), base_url="https://api.openai.com/v1", priority=1, )) manager.add_profile(AuthProfile( name="azure-eu", provider="azure", api_key=os.getenv("AZURE_OPENAI_API_KEY"), base_url=os.getenv("AZURE_OPENAI_ENDPOINT"), priority=2, )) ``` *** ## Failover Callbacks React to failover events: ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import FailoverManager, FailoverConfig def on_failover(from_profile, to_profile, error): print(f"Failing over from {from_profile} to {to_profile}") print(f"Reason: {error}") # Log to monitoring system config = FailoverConfig( on_failover=on_failover ) manager = FailoverManager(config) ``` **Sync + async parity (from PraisonAI #2386).** The `retry` callback now fires on **both** the sync (`agent.chat`) and async (`await agent.achat`) LLM paths. Older versions silently skipped the callback on async calls — upgrade to get retry observability for gateway bots, async tools, and any `await`-based agent code. *** ## Provider Status Monitor provider health: ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import FailoverManager manager = FailoverManager() # Get status of all providers status = manager.status() for name, info in status.items(): print(f"{name}: {info['status']}") print(f" Failures: {info['failure_count']}") print(f" Last success: {info['last_success']}") # Reset a provider after recovery manager.mark_success("openai") # Reset all profiles manager.reset_all() ``` *** ## Best Practices Always have at least 2-3 providers configured. This ensures availability even during major outages. Enable `exponential_backoff=True` to avoid hammering providers during issues. This helps you stay within rate limits. Order providers by cost and reliability. Put cheaper/faster providers first, with premium providers as fallback. Use the `on_failover` callback to track when failovers occur. This helps identify provider issues early. The callback fires on both sync and async LLM paths — works the same whether your agent runs via `agent.chat()` or `await agent.achat()`. Pair failover with [LLM Error Classification](/docs/features/llm-error-classification) so `FailoverDecision` coordinates profile rotation with typed errors. Load keys from environment variables or a secrets manager — never commit credentials to version control. *** ## Related Typed errors that drive failover decisions Supported LLM providers