Failover

Failover automatically switches between LLM providers when one fails, keeping agents running during API outages or rate limits.

import os
from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager

manager = FailoverManager(FailoverConfig(max_retries=3))
manager.add_profile(AuthProfile(
    name="openai", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
))
manager.add_profile(AuthProfile(
    name="anthropic", provider="anthropic",
    api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
))

agent = Agent(name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager})
agent.start("Hello!")

The user sends a message; if the primary LLM profile fails, the agent tries the next profile in the failover chain.

Failover integrates with LLM Error Classification via FailoverDecision, which coordinates profile rotation with typed error handling.

Quick Start

Run an agent with failover

import os
from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager

manager = FailoverManager(FailoverConfig(max_retries=3, exponential_backoff=True))
manager.add_profile(AuthProfile(
    name="openai", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
))
manager.add_profile(AuthProfile(
    name="anthropic", provider="anthropic",
    api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
))

agent = Agent(
    name="assistant",
    llm={"model": "gpt-4o-mini", "failover_manager": manager},
)
agent.start("Hello!")

Add more profiles

import os
from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()
manager.add_profile(AuthProfile(
    name="groq", provider="groq",
    api_key=os.getenv("GROQ_API_KEY"), priority=3,
))

Monitor provider health

status = manager.status()
for name, info in status.items():
    print(f"{name}: {info['status']}")

How failover activates during retries

Failover now drives LLM retries through direct integration with the retry mechanism:

On every LLM call, the system first gets the current profile via get_next_profile() and applies its api_key, base_url, and model settings
On success, mark_success(profile) is called to track the working provider
On failure, mark_failure(profile, error, is_rate_limit=...) marks the provider as failed, then get_next_profile() fetches the next available provider
Profile switching overrides non-retryable classification—one extra attempt is always granted after switching providers
The LLM automatically updates request parameters (api_key, base_url, model) when switching between profiles

How It Works

Component	Role
AuthProfile	Credentials for a single provider
FailoverManager	Orchestrates failover logic
FailoverConfig	Retry and backoff settings
ProviderStatus	Tracks provider health

Configuration Options

FailoverManager

Manager class reference

AuthProfile

Provider credential profile

from praisonaiagents import FailoverConfig

config = FailoverConfig(
    max_retries=3,
    retry_delay=1.0,
    exponential_backoff=True,
    max_retry_delay=60.0,
)

Option	Type	Default	Description
`max_retries`	`int`	`3`	Maximum retry attempts
`retry_delay`	`float`	`1.0`	Initial retry delay
`exponential_backoff`	`bool`	`True`	Use exponential backoff
`max_retry_delay`	`float`	`60.0`	Maximum retry delay
`cooldown_on_rate_limit`	`float`	`60.0`	Rate limit cooldown (seconds)
`cooldown_on_error`	`float`	`30.0`	Error cooldown (seconds)
`rotate_on_success`	`bool`	`False`	Rotate profiles on success

Auth Profiles

Configure credentials for each provider:

import os
from praisonaiagents import AuthProfile

profile = AuthProfile(
    name="openai-primary",
    provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"),
    priority=1,
    rate_limit_rpm=100,
)

Field	Type	Description
`name`	`str`	Unique profile identifier
`provider`	`str`	Provider: openai, anthropic, etc.
`api_key`	`str`	API key (masked in logs)
`base_url`	`str`	Custom API endpoint
`model`	`str`	Default model for this profile
`priority`	`int`	Failover priority (lower = higher priority)
`rate_limit_rpm`	`int`	Requests per minute limit
`rate_limit_tpm`	`int`	Tokens per minute limit
`metadata`	`dict`	Additional provider-specific config

Common Patterns

Multi-Provider
Cost Optimization
Regional Failover

from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()

# Add multiple providers
manager.add_profile(AuthProfile(
    name="openai", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
))

manager.add_profile(AuthProfile(
    name="anthropic", provider="anthropic",
    api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
))

manager.add_profile(AuthProfile(
    name="groq", provider="groq",
    api_key=os.getenv("GROQ_API_KEY"), priority=3,
))

import os
from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()
manager.add_profile(AuthProfile(
    name="gpt-4o-mini", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
    model="gpt-4o-mini",
))
manager.add_profile(AuthProfile(
    name="gpt-4o", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=2,
    model="gpt-4o",
))

import os
from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()
manager.add_profile(AuthProfile(
    name="openai-us", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"),
    base_url="https://api.openai.com/v1", priority=1,
))
manager.add_profile(AuthProfile(
    name="azure-eu", provider="azure",
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    base_url=os.getenv("AZURE_OPENAI_ENDPOINT"), priority=2,
))

Failover Callbacks

React to failover events:

from praisonaiagents import FailoverManager, FailoverConfig

def on_failover(from_profile, to_profile, error):
    print(f"Failing over from {from_profile} to {to_profile}")
    print(f"Reason: {error}")
    # Log to monitoring system
    
config = FailoverConfig(
    on_failover=on_failover
)

manager = FailoverManager(config)

Sync + async parity (from PraisonAI #2386). The retry callback now fires on both the sync (agent.chat) and async (await agent.achat) LLM paths. Older versions silently skipped the callback on async calls — upgrade to get retry observability for gateway bots, async tools, and any await-based agent code.

Failover callbacks run outside the pool lock

Your on_failover callbacks are invoked after FailoverManager releases its internal lock (PraisonAI PR #3574). This means:

Blocking work in a callback (network I/O, logging, metrics) does not stall other agents queuing on the pool.
A callback that acquires an external lock will not deadlock against the pool’s lock.
Callbacks still run in the thread that triggered the failover — if you care about ordering, that is preserved.

Provider Status

Monitor provider health:

from praisonaiagents import FailoverManager

manager = FailoverManager()

# Get status of all providers
status = manager.status()
for name, info in status.items():
    print(f"{name}: {info['status']}")
    print(f"  Failures: {info['failure_count']}")
    print(f"  Last success: {info['last_success']}")

# Reset a provider after recovery
manager.mark_success("openai")

# Reset all profiles
manager.reset_all()

Thread safety

FailoverManager is safe to share across concurrent agents and threads — it is the process-wide credential pool (PraisonAI PR #3574). Concurrent agents hitting a rate limit at the same time cannot observe torn reads or corrupt the pool: every read and mutation of the profile list and each profile’s status is serialised internally. Wire one FailoverManager into many agents and let them share the same credential pool — no external locking required. See Thread Safety for how this fits with the rest of the SDK’s concurrency model.

Best Practices

Configure multiple providers

Always have at least 2-3 providers configured. This ensures availability even during major outages.

Use exponential backoff

Enable exponential_backoff=True to avoid hammering providers during issues. This helps you stay within rate limits.

Set appropriate priorities

Order providers by cost and reliability. Put cheaper/faster providers first, with premium providers as fallback.

Monitor failover events

Use the on_failover callback to track when failovers occur. This helps identify provider issues early. The callback fires on both sync and async LLM paths — works the same whether your agent runs via agent.chat() or await agent.achat().

Integrate with error classification

Pair failover with LLM Error Classification so FailoverDecision coordinates profile rotation with typed errors.

Keep API keys out of source

Load keys from environment variables or a secrets manager — never commit credentials to version control.

LLM Error Classification

Typed errors that drive failover decisions

Providers

Supported LLM providers

Getting Started

Install

Core Concepts

Guides

Features

Models

Databases

Observability

Memory

Telemetry & Analytics

Knowledge

RAG

Persistence

Tools

Other Features

Developers

Configuration

Best Practices

Getting Started (No Code)

Quick Start

How failover activates during retries

How It Works

Configuration Options

FailoverManager

AuthProfile

Auth Profiles

Common Patterns

Failover Callbacks

Failover callbacks run outside the pool lock

Provider Status

Thread safety

Best Practices

LLM Error Classification

Providers

​Quick Start

​How failover activates during retries

​How It Works

​Configuration Options

FailoverManager

AuthProfile

​Auth Profiles

​Common Patterns

​Failover Callbacks

​Failover callbacks run outside the pool lock

​Provider Status

​Thread safety

​Best Practices

​Related

LLM Error Classification

Providers

Quick Start

How failover activates during retries

How It Works

Configuration Options

Auth Profiles

Common Patterns

Failover Callbacks

Failover callbacks run outside the pool lock

Provider Status

Thread safety

Best Practices

Related