Models in PraisonAI

Overview of supported language models in PraisonAI, including OpenAI, Groq, Google Gemini, Anthropic Claude, and configuration examples

Point an Agent at any provider by setting its llm parameter — PraisonAI routes to OpenAI, Anthropic, Gemini, Groq, Cohere, or a local Ollama model.

from praisonaiagents import Agent

agent = Agent(instructions="You are a helpful assistant", llm="gpt-4o-mini")
agent.start("Why is the sky blue?")

Code

Set model by 3 ways

1. OpenAI Compatible Endpoints

Example Groq Implementation:

export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
export OPENAI_BASE_URL=https://api.groq.com/openai/v1

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="llama-3.1-8b-instant",
)

agent.start("Why sky is Blue?")

2. Litellm Compatible model names (eg: gemini/gemini-1.5-flash-8b)

pip install "praisonaiagents[llm]"

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-1.5-flash-8b",
    reflection=True,

)

agent.start("Why sky is Blue?")

3. Litellm Compatible Configuration

pip install "praisonaiagents[llm]"

from praisonaiagents import Agent

llm_config = {
    "model": "gemini/gemini-1.5-flash-latest",  # Model name without provider prefix

    # Core settings
    "temperature": 0.7,                # Controls randomness (like temperature)
    "timeout": 30,                 # Timeout in seconds
    "top_p": 0.9,                    # Nucleus sampling parameter
    "max_tokens": 1000,               # Max tokens in response

    # Advanced parameters
    "presence_penalty": 0.1,         # Penalize repetition of topics (-2.0 to 2.0)
    "frequency_penalty": 0.1,        # Penalize token repetition (-2.0 to 2.0)

    # API settings (optional)
    "api_key": None,                 # Your API key (or use environment variable)
    "base_url": None,                # Custom API endpoint if needed

    # Response formatting
    "response_format": {             # Force specific response format
        "type": "text"               # Options: "text", "json_object"
    },

    # Additional controls
    "seed": 42,                      # For reproducible responses
    "stop_phrases": ["##", "END"],   # Custom stop sequences
}

agent = Agent(
    instructions="You are a helpful Assistant."
    llm=llm_config
)
agent.start()

Advanced Configuration (Litellm Support)

1. Install Package

   Install required packages:

   pip install "praisonaiagents[llm]"

2. Setup Environment

   Configure environment:

   export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"

3. Create Agent

   Create app.py:

   # if json_object is supported by the model
   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="gemini/gemini-1.5-flash-8b",
       reflection=True,
   
   )
   
   agent.start("Why sky is Blue?")

   # if json_object is not supported by the model
   from praisonaiagents import Agent
   
   # Detailed LLM configuration
   llm_config = {
       "model": "gemini/gemini-1.5-flash-latest",  # Model name without provider prefix
   
       # Core settings
       "temperature": 0.7,                # Controls randomness (like temperature)
       "timeout": 30,                 # Timeout in seconds
       "top_p": 0.9,                    # Nucleus sampling parameter
       "max_tokens": 1000,               # Max tokens in response
   
       # Advanced parameters
       "presence_penalty": 0.1,         # Penalize repetition of topics (-2.0 to 2.0)
       "frequency_penalty": 0.1,        # Penalize token repetition (-2.0 to 2.0)
   
       # API settings (optional)
       "api_key": None,                 # Your API key (or use environment variable)
       "base_url": None,                # Custom API endpoint if needed
   
       # Response formatting
       "response_format": {             # Force specific response format
           "type": "text"               # Options: "text", "json_object"
       },
   
       # Additional controls
       "seed": 42,                      # For reproducible responses
       "stop_phrases": ["##", "END"],   # Custom stop sequences
   }
   
   agent = Agent(
       instructions="You are a helpful Assistant specialized in scientific explanations. "
                   "Provide clear, accurate, and engaging responses.",
       llm=llm_config,                  # Pass the detailed configuration
                           # Enable detailed output
                          # Format responses in markdown
       reflection=True,              # Enable self-reflection
       max_iterations=3,                  # Maximum reflection iterations
       min_iterations=1                   # Minimum reflection iterations
   )
   
   # Test the agent
   response = agent.start("Why is the sky blue? Please explain in simple terms.")

export OPENAI_BASE_URL=http://localhost:11434/v1

export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
export OPENAI_BASE_URL=https://api.groq.com/openai/v1

export OPENAI_API_KEY="${GEMINI_API_KEY:?Set GEMINI_API_KEY in your shell}"
export OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/

export OPENAI_BASE_URL=http://localhost:1337/v1

export OPENAI_BASE_URL=http://localhost:1234/v1

export OPENAI_API_KEY="${OPENROUTER_API_KEY:?Set OPENROUTER_API_KEY in your shell}"
export OPENAI_BASE_URL=https://openrouter.ai/api/v1

Provider Auto-Detection (no-config first run)

When you run praisonai run without setting --model or a model: key in config.yaml, PraisonAI inspects which supported provider credential is present in your environment and picks a provider-appropriate default — so a user whose only key is ANTHROPIC_API_KEY no longer gets an OpenAI auth error on first run.

Detection is catalogue-driven: any provider in PROVIDER_ENV_CATALOGUE (praisonaiagents/llm/catalogue.py) with its API-key env var set is recognised — not just the historical handful. The ordered preference is preserved (OpenAI still wins when multiple keys are set).

Two distinct concepts are easy to confuse, so keep them apart:

• The credential-inferred default — the model picked when a provider key is set. Since PR #5412 this is the capability-ranked best-available chat model for that provider (e.g. gpt-4o for OpenAI), not a fixed cheap representative. It is not gpt-4o-mini for OpenAI any more.
• The terminal fallback — the model used only when no credential is present and no local endpoint is reachable. This is still gpt-4o-mini.

Models shown are representatives — the actual id is the capability-ranked best-available for that provider at run time. Check praisonai models list --provider <p> for the current pick, or praisonai models refresh after a new flagship ships.

Precedence: the first catalogued credential that is set wins, in the order above (the head of the catalogue preserves the historical preference). If multiple provider keys are set, the one listed first takes effect.

Same resolver from the Python SDK

Constructing Agent(instructions="…") without llm= uses the same _PROVIDER_DEFAULT_MODELS table — with OLLAMA_HOST set alone, the Agent picks ollama/llama3.2 and routes it through litellm.

import os
from praisonaiagents import Agent

os.environ["OLLAMA_HOST"] = "http://localhost:11434"

# No llm= argument, no OPENAI_API_KEY — the Agent picks ollama/llama3.2
# and routes through litellm.
agent = Agent(instructions="Answer briefly.")
agent.start("2+2")

Before PR #4795, six of the seven defaults resolved the right model name but were then routed to the native OpenAI client, causing ValueError: OPENAI_API_KEY environment variable is required. This is fixed as of #4795 — the SDK Agent() constructor now matches praisonai run. See Agent Model Resolution for the full flow.

Keyless local-first fallback (no env vars set)

If no cloud provider key is set and a local OpenAI-compatible endpoint answers on http://127.0.0.1:11434 (or wherever OPENAI_BASE_URL / OLLAMA_HOST points), PraisonAI uses that local server as the zero-config default — the first praisonai run "..." just works before you configure anything.

# Terminal 1
ollama serve
ollama pull llama3.2

# Terminal 2 — no OPENAI_API_KEY, no OLLAMA_HOST, nothing set
praisonai run "hello"
# > No cloud key found; using local model ollama/llama3.2.
# > Run `praisonai setup` to add a hosted provider.

Precedence (first match wins):

1. --model <name> on the command line.
2. Any cloud provider key from the table above.
3. A reachable local endpoint at OPENAI_BASE_URL → OLLAMA_HOST → http://127.0.0.1:11434.
4. gpt-4o-mini fallback.

Detection recognises two shapes at the local server:

See Local Models for the full guide.

Routed model provider detection

When a model id routes through Bedrock, Vertex AI, or OpenRouter to reach Claude or Gemini, PraisonAI detects the true provider from the model id — not the routing prefix — so the correct streaming adapter is used.

Async streaming (agent.astart(...)) and agent.astream(...) now work correctly for these routed model ids. Older versions fell through to the OpenAI streaming adapter and raised a runtime error on the async path.

Primary vs Auxiliary Model

PraisonAI splits your model choice into two knobs:

The auxiliary knob falls back to model when unset, so single-provider (Anthropic, Ollama, on-prem) setups make zero unexpected third-party calls — the auxiliary calls all stay on your configured provider.

Two config surfaces resolve the auxiliary model:

• Config-file resolver ([defaults].small_model) — used by session-title generation via get_small_model(primary_model, fallback).
• Env-var resolver (PRAISONAI_AUXILIARY_MODEL → OPENAI_MODEL_NAME → gpt-4o-mini) — used by memory quality scoring, context compaction, workflow routing, lite helpers, task callbacks, and learn-manager extraction. Session-title generation also falls through to this ladder when defaults.small_model is unset.

The env-var resolver covers 8 modules: memory/memory.py, memory/learn/manager.py, context/compressor.py, context/optimizer.py, workflows/workflows.py, lite/__init__.py, task/task.py, and session/title.py. Cost tables (utils/cost_utils.py, llm/_cost.py), docstring examples, and public dataclass field defaults are deliberately not routed — PRAISONAI_AUXILIARY_MODEL does not change pricing behaviour. Empty or whitespace-only env values (OPENAI_MODEL_NAME=" ") are ignored.

# .praisonai/config.toml
[defaults]
model = "gpt-4o"
small_model = "gpt-4o-mini"

# Or purely via environment — no config file required
export OPENAI_MODEL_NAME=ollama/llama3.3:70b        # primary agent model
export PRAISONAI_AUXILIARY_MODEL=ollama/qwen3:0.6b  # cheap helper model

See Configuration File → Cheap auxiliary model for internal calls for the full precedence ladder.

Supported Models for No Code

• OpenAI
• Groq
• Google Gemini
• Anthropic Claude
• Cohere
• Mistral
• Ollama
• Other Models

Example agents.yaml

This uses Multi-Agents with Multi-LLMs.

framework: crewai
topic: research about the causes of lung disease
agents:  # Canonical: use 'agents' instead of 'roles'
  research_analyst:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Experienced in analyzing scientific data related to respiratory health.
    goal: Analyze data on lung diseases
    role: Research Analyst
    llm:  
      model: "groq/llama3-70b-8192"
    function_calling_llm: 
      model: "google/gemini-1.5-flash-001"
    tasks:
      data_analysis:
        description: Gather and analyze data on the causes and risk factors of lung
          diseases.
        expected_output: Report detailing key findings on lung disease causes.
    tools:
    - 'InternetSearchTool'
  medical_writer:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Skilled in translating complex medical information into accessible
      content.
    goal: Compile comprehensive content on lung disease causes
    role: Medical Writer
    llm:  
      model: "anthropic/claude-3-haiku-20240307"
    function_calling_llm: 
      model: "openai/gpt-4o"
    tasks:
      content_creation:
        description: Create detailed content summarizing the research findings on
          lung disease causes.
        expected_output: Document outlining various causes and risk factors of lung
          diseases.
    tools:
    - ''
  editor:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Proficient in editing medical content for accuracy and clarity.
    goal: Review and refine content on lung disease causes
    role: Editor
    llm:  
      model: "cohere/command-r"
    tasks:
      content_review:
        description: Edit and refine the compiled content on lung disease causes for
          accuracy and coherence.
        expected_output: Finalized document on lung disease causes ready for dissemination.
    tools:
    - ''
dependencies: []

How It Works

The Agent passes your llm value to the provider resolver, which routes the request to the matching model and returns the response.

Best Practices

Related