LLM Endpoint Configuration

Configure where your agents send LLM requests using environment variables, with automatic provider-specific routing for major LLM providers.

from praisonaiagents import Agent

agent = Agent(name="assistant", llm="anthropic/claude-sonnet-4-6")
agent.start("Draft a release note for today's deploy.")

The user sets MODEL_NAME and provider keys; PraisonAI resolves the OpenAI-compatible endpoint before the agent runs.

As of PraisonAI 4.6.106, the endpoint resolver is available directly from praisonai-code:

from praisonai_code.llm.env import LLMEndpoint, resolve_llm_endpoint

The legacy path from praisonai.llm.env import ... still works via a sys.modules shim — both resolve to the same object.

Quick Start

Use OpenAI (default)

Set your OpenAI API key and create an agent:

import os
from praisonaiagents import Agent

if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")

agent = Agent(
    name="Research Assistant",
    instructions="You are a helpful research assistant"
)

result = agent.start("Explain quantum computing in simple terms")

Use Anthropic directly

No base URL needed - automatic provider routing:

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
if not os.getenv("ANTHROPIC_API_KEY"):
    raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

agent = Agent(
    name="Claude Assistant",
    instructions="You are Claude, an AI assistant"
)

result = agent.start("What makes you different from other AI models?")

Use Groq for fast inference

High-speed inference with automatic routing:

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "groq/llama3-70b"
if not os.getenv("GROQ_API_KEY"):
    raise EnvironmentError("Set GROQ_API_KEY in your environment")

agent = Agent(
    name="Fast Assistant",
    instructions="You are a speed-optimized assistant"
)

result = agent.start("Generate a quick summary of machine learning")

Use Google Gemini

Access Google’s latest models directly:

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
if not os.getenv("GOOGLE_API_KEY"):
    raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

agent = Agent(
    name="Gemini Assistant",
    instructions="You are powered by Google Gemini"
)

result = agent.start("Analyze this complex dataset")

How It Works

Provider-specific defaults

If you set MODEL_NAME=anthropic/claude-3-5-sonnet, you do not need to set OPENAI_BASE_URL — the right base URL is picked automatically.

Model prefix	API key env var	Default base URL
`anthropic/`	`ANTHROPIC_API_KEY`	`https://api.anthropic.com/v1`
`google/`	`GOOGLE_API_KEY`	`https://generativelanguage.googleapis.com/v1beta`
`gemini/`	`GEMINI_API_KEY`	`https://generativelanguage.googleapis.com/v1beta`
`groq/`	`GROQ_API_KEY`	`https://api.groq.com/openai/v1`
`cohere/`	`COHERE_API_KEY`	`https://api.cohere.ai/v1`
`openrouter/`	`OPENROUTER_API_KEY`	`https://openrouter.ai/api/v1`
`ollama/`	`OLLAMA_API_KEY`	`http://localhost:11434/v1`
no prefix match	`OPENAI_API_KEY`	`https://api.openai.com/v1`

Provider keys do not cross-fallback. If you use anthropic/claude-3-5-sonnet and only OPENAI_API_KEY is set, the call has no credentials. This is a security fix, not a bug — it prevents accidental credential exposure.

Environment Variables

Variable	Purpose	Precedence
`MODEL_NAME`	Model name (highest priority)	1
`OPENAI_MODEL_NAME`	Model name (legacy compat)	2
`OPENAI_BASE_URL`	LLM endpoint URL (highest priority)	1
`OPENAI_API_BASE`	LLM endpoint URL (legacy compat)	2
`OLLAMA_API_BASE`	Ollama endpoint URL	3
`ANTHROPIC_API_KEY`	Anthropic API key (for `anthropic/*` models)	—
`GOOGLE_API_KEY`	Google API key (for `google/*` models)	—
`GEMINI_API_KEY`	Gemini API key (for `gemini/*` models)	—
`GROQ_API_KEY`	Groq API key (for `groq/*` models)	—
`COHERE_API_KEY`	Cohere API key (for `cohere/*` models)	—
`OPENROUTER_API_KEY`	OpenRouter API key (for `openrouter/*` models)	—
`OLLAMA_API_KEY`	Ollama API key (for `ollama/*` models)	—
`OPENAI_API_KEY`	OpenAI API key (for OpenAI models and fallback)	—

Defaults

Setting	Default Value
Model	`gpt-4o-mini`
Base URL	Provider-specific or `https://api.openai.com/v1`
API Key	`None`

Common Patterns

Run against Ollama

bash
python

export OPENAI_BASE_URL="http://localhost:11434/v1"
export MODEL_NAME="llama3"
python your_agent.py

import os
from praisonaiagents import Agent

# Configure Ollama
os.environ["OPENAI_BASE_URL"] = "http://localhost:11434/v1"
os.environ["MODEL_NAME"] = "llama3"

agent = Agent(
    name="Ollama Assistant",
    instructions="You are running on Ollama"
)

result = agent.start("Explain the benefits of local AI")

Run against a corporate OpenAI proxy

bash
python

export OPENAI_BASE_URL="https://corporate-proxy.company.com/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="gpt-4"
python your_agent.py

import os
from praisonaiagents import Agent

# Configure corporate proxy
os.environ["OPENAI_BASE_URL"] = "https://corporate-proxy.company.com/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "gpt-4"

agent = Agent(
    name="Corporate Assistant",
    instructions="You are using a corporate OpenAI proxy"
)

result = agent.start("Generate a business report")

Use Anthropic directly

bash
python

export MODEL_NAME="anthropic/claude-3-5-sonnet"
export ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:?Set ANTHROPIC_API_KEY in your shell}"
python your_agent.py

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
if not os.getenv("ANTHROPIC_API_KEY"):
    raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

agent = Agent(
    name="Anthropic Assistant",
    instructions="You are powered by Claude"
)

result = agent.start("Write a thoughtful analysis")

Use Groq for speed

bash
python

export MODEL_NAME="groq/llama3-70b"
export GROQ_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
python your_agent.py

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "groq/llama3-70b"
if not os.getenv("GROQ_API_KEY"):
    raise EnvironmentError("Set GROQ_API_KEY in your environment")

agent = Agent(
    name="Speed Assistant",
    instructions="You provide fast responses"
)

result = agent.start("Quick summary of quantum physics")

Use Google Gemini

bash
python

export MODEL_NAME="google/gemini-1.5-pro"
export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"
python your_agent.py

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
if not os.getenv("GOOGLE_API_KEY"):
    raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

agent = Agent(
    name="Gemini Assistant",
    instructions="You are powered by Google Gemini"
)

result = agent.start("Analyze this complex problem")

Use Azure OpenAI / Bedrock via LiteLLM

bash
python

export OPENAI_BASE_URL="https://your-litellm-proxy/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="azure/gpt-4"
python your_agent.py

import os
from praisonaiagents import Agent

# Configure LiteLLM for Azure
os.environ["OPENAI_BASE_URL"] = "https://your-litellm-proxy/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "azure/gpt-4"

agent = Agent(
    name="Azure Assistant",
    instructions="You are using Azure OpenAI via LiteLLM"
)

result = agent.start("Analyze this data")

User Interaction Flow

Best Practices

Set OPENAI_BASE_URL, not OPENAI_API_BASE

OPENAI_BASE_URL is the standard OpenAI SDK environment variable and has the highest precedence. Use this for all new configurations rather than the legacy OPENAI_API_BASE.

Empty string ≠ unset

An empty string value is skipped during resolution, and the next variable in precedence order is tried. To disable a variable, unset it completely rather than setting it to an empty string.

# This skips OPENAI_BASE_URL and tries OPENAI_API_BASE
export OPENAI_BASE_URL=""
export OPENAI_API_BASE="https://proxy.com/v1"

# This uses OPENAI_BASE_URL
unset OPENAI_BASE_URL
export OPENAI_API_BASE="https://proxy.com/v1"

Use .env files for local dev

Create a .env file in your project root for local development:

# .env
OPENAI_BASE_URL=http://localhost:11434/v1
MODEL_NAME=llama3
# OPENAI_API_KEY not needed for Ollama

Load it in your Python code:

from dotenv import load_dotenv
load_dotenv()

from praisonaiagents import Agent
# Environment variables are now loaded

Realtime/WebSocket endpoints

For realtime features, WebSocket URLs are auto-derived from HTTP URLs. The system automatically:

Converts https:// to wss://
Strips /v1 suffix to avoid /v1/v1/realtime
Appends the appropriate realtime path

You only need to set OPENAI_BASE_URL - the realtime endpoint is handled automatically.

AutoGen Config List

Build an AutoGen-style config_list from the resolved endpoint

Standalone LLM Modules

All LLM modules available without the wrapper package

LLM Configuration

Complete LLM configuration options

Models

Supported models and providers

Provider Pages

Anthropic

Claude models and configuration

Groq

High-speed inference setup

Google

Gemini models and API access

Cohere

Language model configuration

OpenRouter

Multi-provider router setup

Ollama

Local model deployment

C7 note: As of PR #2550, the tool-discovery pipeline (tool_resolver), safe loader (_safe_loader), framework probes (_framework_availability), and plugin registry (tool_registry) also moved to praisonai_code — completing the same C7 arc as LLM config. See Tool Discovery Order and Local Tools Loading.

​Quick Start

​How It Works

​Provider-specific defaults

​Environment Variables

​Defaults

​Common Patterns

​Run against Ollama

​Run against a corporate OpenAI proxy

​Use Anthropic directly

​Use Groq for speed

​Use Google Gemini

​Use Azure OpenAI / Bedrock via LiteLLM

​User Interaction Flow

​Best Practices

​Related

AutoGen Config List

Standalone LLM Modules

LLM Configuration

Models

​Provider Pages

Anthropic

Groq

Google

Cohere

OpenRouter

Ollama

Quick Start

How It Works

Provider-specific defaults

Environment Variables

Defaults

Common Patterns

Run against Ollama

Run against a corporate OpenAI proxy

Use Anthropic directly

Use Groq for speed

Use Google Gemini

Use Azure OpenAI / Bedrock via LiteLLM

User Interaction Flow

Best Practices

Related

Provider Pages