Model Fallback

Model Fallback keeps your agent answering by automatically retrying on alternate models when the primary model is overloaded or unavailable.

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="assistant",
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["anthropic/claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Answer even when the primary model is overloaded")

The user sends a prompt; the agent retries on fallback models when the primary returns errors.

Quick Start

One-line resilience

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Summarise today's news")

Cross-provider chain

Use LiteLLM-style prefixes when mixing providers:

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    model=LLMConfig(
        model="openai/gpt-4o",
        fallback_models=["anthropic/claude-3-5-sonnet", "openai/gpt-4o-mini"],
    ),
)

Notice the switch

Subscribe to HookEvent.MODEL_FALLBACK to react the moment the primary model is swapped out:

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig
from praisonaiagents.hooks import HookRegistry, HookEvent, HookResult

registry = HookRegistry()

@registry.on(HookEvent.MODEL_FALLBACK)
def notice(evt):
    print(f"[fallback] {evt.from_model} → {evt.to_model} ({evt.reason_category})")
    return HookResult.allow()

agent = Agent(
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
    hooks=registry,
)
agent.start("Answer even during a provider outage")

How It Works

On transient errors (503, timeout, model overloaded), the agent retries the same turn against the next model in fallback_models. Successful calls stay on the primary model. Failover fires on retryable errors classified by the LLM error classifier and covers every turn shape — non-streaming, streaming, tool-iteration turns, reflection turns, and their async equivalents. A 503 on a streaming chunk pushes the same turn to the next model in fallback_models; the user sees continuous output, not a failure.

Observing the Switch

Subscribe to HookEvent.MODEL_FALLBACK (or the MODEL_FALLBACK stream event) to react the moment the primary model is swapped out — for alerts, metrics, a UI notice, or a per-user notification.

MODEL_FALLBACK is dispatched through the agent-scoped hook registry passed via Agent(hooks=...). Register on the agent’s registry (as shown below) so the hook fires on both sync and async runs. Hooks registered only on the global default registry may be skipped on the async path.

`ModelFallbackInput` fields

Field	Type	Default	Description
`from_model`	`str`	`""`	Model id the turn was using when the failure hit (e.g. `"gpt-4o"`).
`to_model`	`str`	`""`	Next entry in the `fallback_models` chain that the turn continues on.
`reason_category`	`str`	`""`	Failure classification from the LLM error classifier (e.g. `"rate_limit"`, `"provider_down"`, `"timeout"`). Provider internals stay redacted.
`fallback_index`	`int`	`0`	0-based index into `fallback_models` of the entry now in use.
`session_id`, `cwd`, `event_name`, `timestamp`, `agent_name`	—	—	Standard `HookInput` fields.

Notification only — the turn already continued on to_model. Provider internals are redacted; only the failure class reaches your hook. Zero overhead when unsubscribed. Errors inside the hook never break the fallback path.

Hook subscription

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig
from praisonaiagents.hooks import HookRegistry, HookEvent, HookResult

registry = HookRegistry()

@registry.on(HookEvent.MODEL_FALLBACK)
def on_fallback(evt):
    print(f"[fallback] {evt.from_model} → {evt.to_model}")
    print(f"           reason={evt.reason_category} index={evt.fallback_index}")
    return HookResult.allow()

agent = Agent(
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
    hooks=registry,
)
agent.start("Answer even during a provider outage")

Stream event

When a stream callback is active, the same swap emits a StreamEventType.MODEL_FALLBACK event carrying the four fields in metadata, plus agent_id, session_id, and run_id.

The stream event honours the run’s emit_events flag: when events are suppressed, StreamEventType.MODEL_FALLBACK is skipped, but HookEvent.MODEL_FALLBACK still fires. Use the hook when you need a guaranteed signal regardless of stream configuration.

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig
from praisonaiagents.streaming import StreamEvent, StreamEventType

agent = Agent(
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
    stream=True,
)

def on_event(event: StreamEvent):
    if event.type == StreamEventType.MODEL_FALLBACK:
        m = event.metadata
        print(f"[stream] switched {m['from_model']} → {m['to_model']} ({m['reason_category']})")

agent.stream_emitter.add_callback(on_event)
agent.start("Answer even during a provider outage")

Sync and async paths

Both call sites emit — the sync _chat_completion path and the async _handle_async_llm_error path — so the hook fires whether you run agent.start(...) or await agent.astart(...).

Sync path
Async path

@registry.on(HookEvent.MODEL_FALLBACK)
def on_fallback(evt):
    print(f"[fallback] {evt.from_model} → {evt.to_model}")
    return HookResult.allow()

agent.start("Answer even during a provider outage")

@registry.on(HookEvent.MODEL_FALLBACK)
async def on_fallback(evt):
    print(f"[fallback] {evt.from_model} → {evt.to_model}")
    return HookResult.allow()

await agent.astart("Answer even during a provider outage")

Async hooks are awaited (not fire-and-forget). The fallback retry does not proceed until the hook returns, so long-running notifiers should keep their work non-blocking or offload it.

Configuration Options

Model Fallback is configured through LLMConfig — set model for the primary and fallback_models for the ordered backup chain.

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant",
    model=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet", "gpt-4o-mini"],
    ),
)
agent.start("Hello!")

Option	Type	Default	Description
`model`	`str`	—	Primary model the turn starts on.
`fallback_models`	`Optional[List[str]]`	`None`	Ordered backup chain; the runtime advances to the next entry on a retryable failure.

The decision diagram below shows which chain shape fits which scenario.

Streaming, tool-iteration and reflection turns route through the same failover engine as plain single-shot calls (unified in PraisonAI PR #2665). You don’t need a separate fallback_models= setting for streaming paths.

LLMConfig API Reference

Full LLMConfig surface is covered by the auto-generated SDK reference — see Hook Events for the paired MODEL_FALLBACK hook.

Best Practices

Put a cheap same-provider fallback last

Useful for rate limits, not full provider outages — a cheap model on the same API may still fail if the provider is down.

Order by latency and cost

Fallback runs the same prompt; a much weaker model may return a worse answer, not a missing one.

Limit chain length to 2–3

Longer chains delay user-visible errors without improving success rates much.

Use provider prefixes when mixing

LiteLLM-style names (anthropic/..., openai/...) route credentials correctly across providers.

LLM Configuration

Endpoints, API keys, and auth headers.

Models

Choosing models for agents.

Model Router

Dynamic model selection policies.

Rate Limiter

Throttle requests before they fail.

Getting Started

Install

Core Concepts

Guides

Features

Models

Databases

Observability

Memory

Telemetry & Analytics

Knowledge

RAG

Persistence

Tools

Other Features

Developers

Configuration

Best Practices

Getting Started (No Code)

Quick Start

How It Works

Observing the Switch

`ModelFallbackInput` fields

Hook subscription

Stream event

Sync and async paths

Configuration Options

LLMConfig API Reference

Best Practices

LLM Configuration

Models

Model Router

Rate Limiter

​Quick Start

​How It Works

​Observing the Switch

​ModelFallbackInput fields

​Hook subscription

​Stream event

​Sync and async paths

​Configuration Options

LLMConfig API Reference

​Best Practices

​Related

LLM Configuration

Models

Model Router

Rate Limiter

Quick Start

How It Works

Observing the Switch

`ModelFallbackInput` fields

Hook subscription

Stream event

Sync and async paths

Configuration Options

Best Practices

Related