LLM Endpoint Configuration

Point your agents at any OpenAI-compatible endpoint with environment variables

Configure where your agents send LLM requests using environment variables, with automatic provider-specific routing for major LLM providers.

python

from praisonaiagents import Agent

agent = Agent(name="assistant", llm="anthropic/claude-sonnet-4-6")
agent.start("Draft a release note for today's deploy.")

The user sets MODEL_NAME and provider keys; PraisonAI resolves the OpenAI-compatible endpoint before the agent runs.

Note

As of PraisonAI 4.6.106, the endpoint resolver is available directly from praisonai-code:

python

from praisonai_code.llm.env import LLMEndpoint, resolve_llm_endpoint

The legacy path from praisonai.llm.env import ... still works via a sys.modules shim — both resolve to the same object.

Diagram

Quick Start

Use OpenAI (default)

Set your OpenAI API key and create an agent:

python

import os
from praisonaiagents import Agent

if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")

agent = Agent(
    name="Research Assistant",
    instructions="You are a helpful research assistant"
)

result = agent.start("Explain quantum computing in simple terms")

Use Anthropic directly

No base URL needed – automatic provider routing:

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
if not os.getenv("ANTHROPIC_API_KEY"):
    raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

agent = Agent(
    name="Claude Assistant",
    instructions="You are Claude, an AI assistant"
)

result = agent.start("What makes you different from other AI models?")

Use Groq for fast inference

High-speed inference with automatic routing:

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "groq/llama3-70b"
if not os.getenv("GROQ_API_KEY"):
    raise EnvironmentError("Set GROQ_API_KEY in your environment")

agent = Agent(
    name="Fast Assistant",
    instructions="You are a speed-optimized assistant"
)

result = agent.start("Generate a quick summary of machine learning")

Use Google Gemini

Access Google’s latest models directly:

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
if not os.getenv("GOOGLE_API_KEY"):
    raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

agent = Agent(
    name="Gemini Assistant",
    instructions="You are powered by Google Gemini"
)

result = agent.start("Analyze this complex dataset")

How It Works

Diagram

Provider-specific defaults

If you set MODEL_NAME=anthropic/claude-3-5-sonnet, you do not need to set OPENAI_BASE_URL — the right base URL is picked automatically.

Model prefix	API key env var	Default base URL
`anthropic/`	`ANTHROPIC_API_KEY`	`https://api.anthropic.com/v1`
`google/`	`GOOGLE_API_KEY`	`https://generativelanguage.googleapis.com/v1beta`
`gemini/`	`GEMINI_API_KEY`	`https://generativelanguage.googleapis.com/v1beta`
`groq/`	`GROQ_API_KEY`	`https://api.groq.com/openai/v1`
`cohere/`	`COHERE_API_KEY`	`https://api.cohere.ai/v1`
`openrouter/`	`OPENROUTER_API_KEY`	`https://openrouter.ai/api/v1`
`ollama/`	`OLLAMA_API_KEY`	`http://localhost:11434/v1`
_no prefix match_	`OPENAI_API_KEY`	`https://api.openai.com/v1`

Environment Variables

Variable	Purpose	Precedence
`MODEL_NAME`	Model name (highest priority)	1
`OPENAI_MODEL_NAME`	Model name (legacy compat)	2
`PRAISONAI_AUXILIARY_MODEL`	Model used for internal helper calls (session titles, context compaction, memory quality scoring, workflow routing). Overrides `OPENAI_MODEL_NAME` for these calls only; the agent’s own primary model is unaffected.	—
`OPENAI_BASE_URL`	LLM endpoint URL (highest priority)	1
`OPENAI_API_BASE`	LLM endpoint URL (legacy compat)	2
`OLLAMA_API_BASE`	Ollama endpoint URL	3
`ANTHROPIC_API_KEY`	Anthropic API key (for `anthropic/*` models)	—
`GOOGLE_API_KEY`	Google API key (for `google/*` models)	—
`GEMINI_API_KEY`	Gemini API key (for `gemini/*` models)	—
`GROQ_API_KEY`	Groq API key (for `groq/*` models)	—
`COHERE_API_KEY`	Cohere API key (for `cohere/*` models)	—
`OPENROUTER_API_KEY`	OpenRouter API key (for `openrouter/*` models)	—
`OLLAMA_API_KEY`	Ollama API key (for `ollama/*` models)	—
`OPENAI_API_KEY`	OpenAI API key (for OpenAI models and fallback)	—

Defaults

Setting	Default Value
Model	`gpt-4o-mini`
Base URL	Provider-specific or `https://api.openai.com/v1`
API Key	`None`

Common Patterns

Run against Ollama

Name the model with the ollama/ prefix. It resolves the base URL automatically and routes through OllamaAdapter (tool-call repair) instead of the plain OpenAIClient.

bash

export MODEL_NAME="ollama/llama3.2"
# OPENAI_BASE_URL is not needed — ollama/ resolves it automatically.
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "ollama/llama3.2"

agent = Agent(
    name="Ollama Assistant",
    instructions="You are running on Ollama"
)

result = agent.start("Explain the benefits of local AI")

Run against a corporate OpenAI proxy

bash

export OPENAI_BASE_URL="https://corporate-proxy.company.com/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="gpt-4"
python your_agent.py

python

import os
from praisonaiagents import Agent

# Configure corporate proxy
os.environ["OPENAI_BASE_URL"] = "https://corporate-proxy.company.com/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "gpt-4"

agent = Agent(
    name="Corporate Assistant",
    instructions="You are using a corporate OpenAI proxy"
)

result = agent.start("Generate a business report")

Use Anthropic directly

bash

export MODEL_NAME="anthropic/claude-3-5-sonnet"
export ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:?Set ANTHROPIC_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
if not os.getenv("ANTHROPIC_API_KEY"):
    raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

agent = Agent(
    name="Anthropic Assistant",
    instructions="You are powered by Claude"
)

result = agent.start("Write a thoughtful analysis")

Use Groq for speed

bash

export MODEL_NAME="groq/llama3-70b"
export GROQ_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "groq/llama3-70b"
if not os.getenv("GROQ_API_KEY"):
    raise EnvironmentError("Set GROQ_API_KEY in your environment")

agent = Agent(
    name="Speed Assistant",
    instructions="You provide fast responses"
)

result = agent.start("Quick summary of quantum physics")

Use Google Gemini

bash

export MODEL_NAME="google/gemini-1.5-pro"
export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
if not os.getenv("GOOGLE_API_KEY"):
    raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

agent = Agent(
    name="Gemini Assistant",
    instructions="You are powered by Google Gemini"
)

result = agent.start("Analyze this complex problem")

Use Azure OpenAI / Bedrock via LiteLLM

bash

export OPENAI_BASE_URL="https://your-litellm-proxy/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="azure/gpt-4"
python your_agent.py

python

import os
from praisonaiagents import Agent

# Configure LiteLLM for Azure
os.environ["OPENAI_BASE_URL"] = "https://your-litellm-proxy/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "azure/gpt-4"

agent = Agent(
    name="Azure Assistant",
    instructions="You are using Azure OpenAI via LiteLLM"
)

result = agent.start("Analyze this data")

User Interaction Flow

Diagram

Best Practices

Set OPENAI_BASE_URL, not OPENAI_API_BASE

OPENAI_BASE_URL is the standard OpenAI SDK environment variable and has the highest precedence. Use this for all new configurations rather than the legacy OPENAI_API_BASE.

Empty string ≠ unset

An empty string value is skipped during resolution, and the next variable in precedence order is tried. To disable a variable, unset it completely rather than setting it to an empty string.

bash

# This skips OPENAI_BASE_URL and tries OPENAI_API_BASE
export OPENAI_BASE_URL=""
export OPENAI_API_BASE="https://proxy.com/v1"

# This uses OPENAI_BASE_URL
unset OPENAI_BASE_URL
export OPENAI_API_BASE="https://proxy.com/v1"

Use .env files for local dev

Create a .env file in your project root for local development:

bash

# .env
MODEL_NAME=ollama/llama3.2
# OPENAI_BASE_URL not needed — ollama/ resolves it automatically.
# OPENAI_API_KEY not needed for Ollama

Load it in your Python code:

python

from dotenv import load_dotenv
load_dotenv()

from praisonaiagents import Agent
# Environment variables are now loaded

Realtime/WebSocket endpoints

For realtime features, WebSocket URLs are auto-derived from HTTP URLs. The system automatically:

Converts https:// to wss://
Strips /v1 suffix to avoid /v1/v1/realtime
Appends the appropriate realtime path

You only need to set OPENAI_BASE_URL – the realtime endpoint is handled automatically.

AutoGen Config List

Build an AutoGen-style config_list from the resolved endpoint

Standalone LLM Modules

All LLM modules available without the wrapper package

LLM Configuration

Complete LLM configuration options

Models

Supported models and providers

Provider Pages

Anthropic

Claude models and configuration

Groq

High-speed inference setup

Google

Gemini models and API access

Cohere

Language model configuration

OpenRouter

Multi-provider router setup

Ollama

Local model deployment

C7 note: As of PR #2550, the tool-discovery pipeline (tool_resolver), safe loader (_safe_loader), framework probes (_framework_availability), and plugin registry (tool_registry) also moved to praisonai_code — completing the same C7 arc as LLM config. See Tool Discovery Order and Local Tools Loading.

LLM Endpoint Configuration

Quick Start

How It Works

Provider-specific defaults

Environment Variables

Defaults

Common Patterns

Run against Ollama

bash

python

Run against a corporate OpenAI proxy

bash

python

Use Anthropic directly

bash

python

Use Groq for speed

bash

python

Use Google Gemini

bash

python

Use Azure OpenAI / Bedrock via LiteLLM

bash

python

User Interaction Flow

Best Practices

Related

Provider Pages