Failover Manager • AI Agent SDK

FailoverManager: Manages failover between multiple LLM auth profiles.

FailoverManager

Defined in the failover module.

AI Agent

Manages failover between multiple LLM auth profiles.

Provides automatic failover when rate limits or errors occur, with configurable retry behavior and cooldown periods.

Constructor

ParamField Optional[FailoverConfig]

No description available.

ParamField Optional[QuotaCoordinatorProtocol]

No description available.

Methods

add_profile()

Add an auth profile.

remove_profile()

Remove a profile by name.

get_profile()

Get a profile by name.

list_profiles()

List all profiles.

get_next_profile()

Get the next available profile.

mark_failure()

Mark a profile as failed.

mark_success()

Mark a profile as successful.

on_failover()

get_retry_delay()

Calculate retry delay for an attempt.

status()

Get failover manager status.

reset_all()

Reset all profiles to available status.

Usage

python

manager = FailoverManager()
    manager.add_profile(AuthProfile(
        name="openai-primary",
        provider="openai",
        api_key="sk-...",
        priority=0,
    ))
    manager.add_profile(AuthProfile(
        name="openai-backup",
        provider="openai",
        api_key="sk-...",
        priority=1,
    ))

    # Get next available profile
    profile = manager.get_next_profile()

    # On failure
    manager.mark_failure(profile, "Rate limit exceeded")

    # Get next profile (will return backup)
    profile = manager.get_next_profile()

Source

View on GitHub

praisonaiagents/llm/failover.py at line 206