> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Fallback

> Keep your agent answering when a model is overloaded by automatically falling back to alternates

Model Fallback keeps your agent answering by automatically retrying on alternate models when the primary model is overloaded or unavailable.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="assistant",
    instructions="You are a helpful assistant",
    llm=LLMConfig(
        model="gpt-4o",
        fallbacks=["anthropic/claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Answer even when the primary model is overloaded")
```

The user sends a prompt; the agent retries on fallback models when the primary returns errors.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Model Fallback"
        User[User] --> Agent[Agent]
        Agent --> Primary[gpt-4o]
        Primary -->|503 / overloaded| FB1[claude-3-5-sonnet]
        FB1 -->|still failing| FB2[gpt-4o-mini]
        FB2 --> Reply[Reply]
        Primary -->|ok| Reply
    end

    classDef user fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef primary fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef fallback fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef reply fill:#10B981,stroke:#7C90A0,color:#fff

    class User user
    class Agent agent
    class Primary primary
    class FB1,FB2 fallback
    class Reply reply
```

## Quick Start

<Steps>
  <Step title="One-line resilience">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent
    from praisonaiagents.config import LLMConfig

    agent = Agent(
        instructions="You are a helpful assistant",
        llm=LLMConfig(
            model="gpt-4o",
            fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
        ),
    )
    agent.start("Summarise today's news")
    ```
  </Step>

  <Step title="Cross-provider chain">
    Use LiteLLM-style prefixes when mixing providers:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent
    from praisonaiagents.config import LLMConfig

    agent = Agent(
        llm=LLMConfig(
            model="openai/gpt-4o",
            fallback_models=["anthropic/claude-3-5-sonnet", "openai/gpt-4o-mini"],
        ),
    )
    ```
  </Step>
</Steps>

## How It Works

On transient errors (503, timeout, model overloaded), the agent retries the **same turn** against the next model in `fallback_models`. Successful calls stay on the primary model.

Failover fires on retryable errors classified by the LLM error classifier and covers **every** turn shape — non-streaming, streaming, tool-iteration turns, reflection turns, and their async equivalents. A 503 on a streaming chunk pushes the same turn to the next model in `fallbacks`; the user sees continuous output, not a failure.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant Agent
    participant Primary as Primary LLM
    participant Fallback as Fallback LLM

    User->>Agent: prompt
    Agent->>Primary: chat completion
    Primary-->>Agent: 503 overloaded
    Note over Agent: error classifier: should_fallback_model
    Agent->>Fallback: retry same messages
    Fallback-->>Agent: response
    Agent-->>User: reply
```

## Configuration Options

\| Option | Type | Default | Description |
\|---Model Fallback keeps your agent answering by automatically retrying on alternate models when the primary model is overloaded or unavailable.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant",
    llm=LLMConfig(model="gpt-4o", fallbacks=["claude-3-5-sonnet", "gpt-4o-mini"]),
)
agent.start("Hello!")
```

The user sends a message; if the primary model fails, the agent retries on configured fallbacks.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{Need resilience on provider outages?}
    Q -->|No| Plain[Agent model=str]
    Q -->|Yes| Q2{Different providers?}
    Q2 -->|Yes| Cross[Cross-provider chain]
    Q2 -->|No| Cost[Cost-degradation chain]

    classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef answer fill:#10B981,stroke:#7C90A0,color:#fff

    class Q,Q2 question
    class Plain,Cross,Cost answer
```

<Note>
  Streaming, tool-iteration and reflection turns route through the same failover engine as plain single-shot calls (unified in [PraisonAI PR #2665](https://github.com/MervinPraison/PraisonAI/pull/2665)). You don't need a separate `fallbacks=` setting for streaming paths.
</Note>

## Best Practices

<AccordionGroup>
  <Accordion title="Put a cheap same-provider fallback last">
    Useful for rate limits, not full provider outages — a cheap model on the same API may still fail if the provider is down.
  </Accordion>

  <Accordion title="Order by latency and cost">
    Fallback runs the same prompt; a much weaker model may return a worse answer, not a missing one.
  </Accordion>

  <Accordion title="Limit chain length to 2–3">
    Longer chains delay user-visible errors without improving success rates much.
  </Accordion>

  <Accordion title="Use provider prefixes when mixing">
    LiteLLM-style names (`anthropic/...`, `openai/...`) route credentials correctly across providers.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="LLM Configuration" icon="sliders" href="/docs/configuration/llm-config">
    Endpoints, API keys, and auth headers.
  </Card>

  <Card title="Models" icon="microchip" href="/docs/models">
    Choosing models for agents.
  </Card>

  <Card title="Model Router" icon="route" href="/docs/features/model-router">
    Dynamic model selection policies.
  </Card>

  <Card title="Rate Limiter" icon="gauge" href="/docs/features/rate-limiter">
    Throttle requests before they fail.
  </Card>
</CardGroup>