Skip to main content
Model Fallback keeps your agent answering by automatically retrying on alternate models when the primary model is overloaded or unavailable. 
from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="assistant",
    instructions="You are a helpful assistant",
    llm=LLMConfig(
        model="gpt-4o",
        fallbacks=["anthropic/claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Answer even when the primary model is overloaded")
The user sends a prompt; the agent retries on fallback models when the primary returns errors.

​
Quick Start

1

One-line resilience

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant",
    llm=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Summarise today's news")
2

Cross-provider chain

Use LiteLLM-style prefixes when mixing providers:
from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    llm=LLMConfig(
        model="openai/gpt-4o",
        fallback_models=["anthropic/claude-3-5-sonnet", "openai/gpt-4o-mini"],
    ),
)

​
How It Works

On transient errors (503, timeout, model overloaded), the agent retries the same turn against the next model in fallback_models. Successful calls stay on the primary model. Failover fires on retryable errors classified by the LLM error classifier and covers every turn shape — non-streaming, streaming, tool-iteration turns, reflection turns, and their async equivalents. A 503 on a streaming chunk pushes the same turn to the next model in fallbacks; the user sees continuous output, not a failure.

​
Configuration Options

| Option | Type | Default | Description | |---Model Fallback keeps your agent answering by automatically retrying on alternate models when the primary model is overloaded or unavailable. 
from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant",
    llm=LLMConfig(model="gpt-4o", fallbacks=["claude-3-5-sonnet", "gpt-4o-mini"]),
)
agent.start("Hello!")
The user sends a message; if the primary model fails, the agent retries on configured fallbacks.
Streaming, tool-iteration and reflection turns route through the same failover engine as plain single-shot calls (unified in PraisonAI PR #2665). You don’t need a separate fallbacks= setting for streaming paths.

​
Best Practices

Useful for rate limits, not full provider outages — a cheap model on the same API may still fail if the provider is down.
Fallback runs the same prompt; a much weaker model may return a worse answer, not a missing one.
Longer chains delay user-visible errors without improving success rates much.
LiteLLM-style names (anthropic/..., openai/...) route credentials correctly across providers.

LLM Configuration

Endpoints, API keys, and auth headers.

Models

Choosing models for agents.

Model Router

Dynamic model selection policies.

Rate Limiter

Throttle requests before they fail.