LLM Config

Configure primary and fallback LLM models with provider prefixes and API settings

LLM Config lets you set the model, API endpoint, and fallback chain so your agent always has a working model.

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="Assistant",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet-20241022", "gpt-4o-mini"],
    ),
)

result = agent.start("Explain quantum computing simply")
print(result)

The user sends a prompt; the agent tries the primary model and transparently falls back if the provider fails.

Diagram

Quick Start

Primary Model Only

Set just the model to get started:

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="Assistant",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(model="gpt-4o")
)

agent.start("What is the speed of light?")

With Fallback Chain

Add fallback models for reliability — if the primary fails, the agent automatically tries the next:

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="ResilientAssistant",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet-20241022", "gpt-4o-mini"],
    ),
)
agent.start("Summarize today's top AI research papers.")

With LiteLLM Provider Prefix

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="anthropic/claude-3-5-sonnet-20241022",
        fallback_models=["openai/gpt-4o", "openai/gpt-4o-mini"],
    ),
)
agent.start("Write a technical blog post about microservices.")

How It Works

Diagram

Phase	What happens
1. Primary	Agent calls the configured `model` first
2. Fallback	On failure, tries each model in `fallback_models` in order
3. Response	First successful response is returned

OPENAI_MODEL_NAME with a provider prefix

A provider/model value in OPENAI_MODEL_NAME now routes through litellm, matching Agent(llm="provider/model").

python

import os
from praisonaiagents import Agent

os.environ["OPENAI_MODEL_NAME"] = "ollama/llama3.2"

# Routes through litellm — same as Agent(llm="ollama/llama3.2").
agent = Agent(instructions="Answer briefly.")
agent.start("2+2")

As of PR #4795, OPENAI_MODEL_NAME="ollama/x" (or any provider/model value) routes to litellm, matching Agent(llm="ollama/x"). Before the fix, those two paths disagreed — an OPENAI_MODEL_NAME with a provider prefix ended up on the native OpenAI client with a model name OpenAI never heard of. A bare value like OPENAI_MODEL_NAME="gpt-4o" still uses the native OpenAI client.

Configuration Options

LLMConfig SDK Reference

Full parameter reference for LLMConfig

Option	Type	Default	Description
`model`	`str`	(required)	Primary model to use
`fallback_models`	`list[str] \	None`	`None`	Ordered fallback models
`base_url`	`str \	None`	`None`	API endpoint override
`api_key`	`str \	None`	`None`	API key (defaults to env vars)
`auth`	`dict \	None`	`None`	Additional auth headers

Common Patterns

Pattern 1 — Self-hosted model via custom endpoint

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="llama3.2",
        base_url="http://localhost:11434/v1",
        api_key="ollama",
    ),
)
response = agent.start("What is the capital of France?")
print(response)

Pattern 2 — Multi-provider fallback for resilience

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="PrivateModelAgent",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="my-custom-model",
        base_url="https://my-llm-gateway.example.com/v1",
        api_key="sk-my-private-key",
        auth={"X-Organization-ID": "org-123"},
    )
)

Self-hosted model via Ollama:

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="LocalAgent",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="ollama/llama3.2",
        base_url="http://localhost:11434",
    )
)

Pattern 3 — Multi-provider fallback for business continuity

python

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="ResilientAgent",
    instructions="You are a reliable assistant for critical tasks.",
    llm_config=LLMConfig(
        model="openai/gpt-4o",
        fallback_models=[
            "anthropic/claude-3-5-sonnet-20241022",
            "openai/gpt-4o-mini",
        ],
    ),
)
agent.start("Generate a business continuity plan for our e-commerce platform.")

Pattern 4 — Passing LLMConfig to a team or flow

python

from praisonaiagents import Agent, AgentTeam
from praisonaiagents.config import LLMConfig

researcher = Agent(name="Researcher", instructions="Research.")
writer     = Agent(name="Writer",     instructions="Write.")

# The container unwraps LLMConfig to its .model string,
# then fills in members that didn't name a model themselves.
team = AgentTeam(
    agents=[researcher, writer],
    llm=LLMConfig(model="anthropic/claude-3-5-sonnet-20241022"),
)
team.start("Summarise the latest AI safety research")

Containers keep only .model from the LLMConfig.

Best Practices

Always add at least one fallback model

Even one fallback prevents your agent from failing completely during rate limits or outages. Use a smaller, cheaper model as the fallback — it costs less and is often available when the primary isn’t.

Use environment variables for API keys

Never hardcode api_key in your code. Leave it as None and set OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. in your environment instead. LLMConfig picks them up automatically via LiteLLM.

Use LiteLLM prefixes for multi-provider setups

Prefix models with the provider name (anthropic/, openai/, ollama/) to avoid ambiguity when using multiple providers in a fallback chain.

Test your fallback chain

Deliberately trigger a failure (e.g., invalid primary model name) and verify your fallback fires correctly before going to production.

For team-wide endpoint or fallback config, pin per-agent

AgentTeam(llm=LLMConfig(...)) and AgentFlow(llm=LLMConfig(...)) only propagate the .model string today. If every member should share a custom base_url / api_key / fallback_models, set an LLMConfig on each member Agent rather than on the container.

LLM Gateways

Route requests through a unified LLM gateway

LLM Endpoint Config

Fine-tune endpoint parameters per model

LLM Error Classification

How errors trigger fallback in the chain

Failover

Automatic failover strategies for reliability