> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Endpoint Configuration

> Point your agents at any OpenAI-compatible endpoint with environment variables

Configure where your agents send LLM requests using environment variables, with automatic provider-specific routing for major LLM providers.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(name="assistant", llm="anthropic/claude-sonnet-4-6")
agent.start("Draft a release note for today's deploy.")
```

The user sets `MODEL_NAME` and provider keys; PraisonAI resolves the OpenAI-compatible endpoint before the agent runs.

<Note>
  As of PraisonAI 4.6.106, the endpoint resolver is available directly from `praisonai-code`:

  ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
  from praisonai_code.llm.env import LLMEndpoint, resolve_llm_endpoint
  ```

  The legacy path `from praisonai.llm.env import ...` still works via a `sys.modules` shim — both resolve to the same object.
</Note>

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Provider Resolution"
        A[📝 MODEL_NAME] --> B{🔍 Prefix?}
        B -->|anthropic/| C[🔑 ANTHROPIC_API_KEY]
        B -->|groq/| D[🔑 GROQ_API_KEY]
        B -->|google/| E[🔑 GOOGLE_API_KEY]
        B -->|No match| F[🔑 OPENAI_API_KEY]
    end
    
    subgraph "Agent Execution"
        C --> G[🤖 Agent]
        D --> G
        E --> G
        F --> G
        G --> H[📡 LLM Provider]
    end
    
    classDef env fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef resolved fill:#10B981,stroke:#7C90A0,color:#fff
    classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff
    
    class A env
    class B decision
    class C,D,E,F resolved
    class G agent
    class H resolved
```

## Quick Start

<Steps>
  <Step title="Use OpenAI (default)">
    Set your OpenAI API key and create an agent:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    if not os.getenv("OPENAI_API_KEY"):
        raise EnvironmentError("Set OPENAI_API_KEY in your environment")

    agent = Agent(
        name="Research Assistant",
        instructions="You are a helpful research assistant"
    )

    result = agent.start("Explain quantum computing in simple terms")
    ```
  </Step>

  <Step title="Use Anthropic directly">
    No base URL needed - automatic provider routing:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
    if not os.getenv("ANTHROPIC_API_KEY"):
        raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

    agent = Agent(
        name="Claude Assistant",
        instructions="You are Claude, an AI assistant"
    )

    result = agent.start("What makes you different from other AI models?")
    ```
  </Step>

  <Step title="Use Groq for fast inference">
    High-speed inference with automatic routing:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "groq/llama3-70b"
    if not os.getenv("GROQ_API_KEY"):
        raise EnvironmentError("Set GROQ_API_KEY in your environment")

    agent = Agent(
        name="Fast Assistant",
        instructions="You are a speed-optimized assistant"
    )

    result = agent.start("Generate a quick summary of machine learning")
    ```
  </Step>

  <Step title="Use Google Gemini">
    Access Google's latest models directly:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
    if not os.getenv("GOOGLE_API_KEY"):
        raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

    agent = Agent(
        name="Gemini Assistant",
        instructions="You are powered by Google Gemini"
    )

    result = agent.start("Analyze this complex dataset")
    ```
  </Step>
</Steps>

***

## How It Works

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    subgraph "Environment Variable Resolution"
        A[🔍 Check MODEL_NAME] --> B{Set?}
        B -->|Yes| C[✅ Use MODEL_NAME]
        B -->|No| D[🔍 Check OPENAI_MODEL_NAME]
        D --> E{Set?}
        E -->|Yes| F[✅ Use OPENAI_MODEL_NAME]
        E -->|No| G[📋 Use gpt-4o-mini]
    end
    
    classDef check fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef decision fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef result fill:#10B981,stroke:#7C90A0,color:#fff
    
    class A,D check
    class B,E decision
    class C,F,G result
```

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Which env var should I set?"
        A[🎯 Goal] --> B{Provider?}
        B -->|OpenAI| C[OPENAI_BASE_URL]
        B -->|Ollama| D[OLLAMA_API_BASE]
        B -->|Proxy| E[OPENAI_BASE_URL]
        B -->|Azure| F[OPENAI_BASE_URL]
    end
    
    classDef goal fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef var fill:#189AB4,stroke:#7C90A0,color:#fff
    
    class A goal
    class B decision
    class C,D,E,F var
```

***

## Provider-specific defaults

If you set `MODEL_NAME=anthropic/claude-3-5-sonnet`, you do not need to set `OPENAI_BASE_URL` — the right base URL is picked automatically.

| Model prefix      | API key env var      | Default base URL                                   |
| ----------------- | -------------------- | -------------------------------------------------- |
| `anthropic/`      | `ANTHROPIC_API_KEY`  | `https://api.anthropic.com/v1`                     |
| `google/`         | `GOOGLE_API_KEY`     | `https://generativelanguage.googleapis.com/v1beta` |
| `gemini/`         | `GEMINI_API_KEY`     | `https://generativelanguage.googleapis.com/v1beta` |
| `groq/`           | `GROQ_API_KEY`       | `https://api.groq.com/openai/v1`                   |
| `cohere/`         | `COHERE_API_KEY`     | `https://api.cohere.ai/v1`                         |
| `openrouter/`     | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/v1`                     |
| `ollama/`         | `OLLAMA_API_KEY`     | `http://localhost:11434/v1`                        |
| *no prefix match* | `OPENAI_API_KEY`     | `https://api.openai.com/v1`                        |

<Warning>
  Provider keys do not cross-fallback. If you use `anthropic/claude-3-5-sonnet` and only `OPENAI_API_KEY` is set, the call has no credentials. This is a security fix, not a bug — it prevents accidental credential exposure.
</Warning>

***

## Environment Variables

| Variable             | Purpose                                         | Precedence |
| -------------------- | ----------------------------------------------- | ---------- |
| `MODEL_NAME`         | Model name (highest priority)                   | 1          |
| `OPENAI_MODEL_NAME`  | Model name (legacy compat)                      | 2          |
| `OPENAI_BASE_URL`    | LLM endpoint URL (highest priority)             | 1          |
| `OPENAI_API_BASE`    | LLM endpoint URL (legacy compat)                | 2          |
| `OLLAMA_API_BASE`    | Ollama endpoint URL                             | 3          |
| `ANTHROPIC_API_KEY`  | Anthropic API key (for `anthropic/*` models)    | —          |
| `GOOGLE_API_KEY`     | Google API key (for `google/*` models)          | —          |
| `GEMINI_API_KEY`     | Gemini API key (for `gemini/*` models)          | —          |
| `GROQ_API_KEY`       | Groq API key (for `groq/*` models)              | —          |
| `COHERE_API_KEY`     | Cohere API key (for `cohere/*` models)          | —          |
| `OPENROUTER_API_KEY` | OpenRouter API key (for `openrouter/*` models)  | —          |
| `OLLAMA_API_KEY`     | Ollama API key (for `ollama/*` models)          | —          |
| `OPENAI_API_KEY`     | OpenAI API key (for OpenAI models and fallback) | —          |

### Defaults

| Setting  | Default Value                                    |
| -------- | ------------------------------------------------ |
| Model    | `gpt-4o-mini`                                    |
| Base URL | Provider-specific or `https://api.openai.com/v1` |
| API Key  | `None`                                           |

***

## Common Patterns

### Run against Ollama

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export OPENAI_BASE_URL="http://localhost:11434/v1"
    export MODEL_NAME="llama3"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    # Configure Ollama
    os.environ["OPENAI_BASE_URL"] = "http://localhost:11434/v1"
    os.environ["MODEL_NAME"] = "llama3"

    agent = Agent(
        name="Ollama Assistant",
        instructions="You are running on Ollama"
    )

    result = agent.start("Explain the benefits of local AI")
    ```
  </Tab>
</Tabs>

### Run against a corporate OpenAI proxy

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export OPENAI_BASE_URL="https://corporate-proxy.company.com/v1"
    export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
    export MODEL_NAME="gpt-4"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    # Configure corporate proxy
    os.environ["OPENAI_BASE_URL"] = "https://corporate-proxy.company.com/v1"
    if not os.getenv("OPENAI_API_KEY"):
        raise EnvironmentError("Set OPENAI_API_KEY in your environment")
    os.environ["MODEL_NAME"] = "gpt-4"

    agent = Agent(
        name="Corporate Assistant",
        instructions="You are using a corporate OpenAI proxy"
    )

    result = agent.start("Generate a business report")
    ```
  </Tab>
</Tabs>

### Use Anthropic directly

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export MODEL_NAME="anthropic/claude-3-5-sonnet"
    export ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:?Set ANTHROPIC_API_KEY in your shell}"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
    if not os.getenv("ANTHROPIC_API_KEY"):
        raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

    agent = Agent(
        name="Anthropic Assistant",
        instructions="You are powered by Claude"
    )

    result = agent.start("Write a thoughtful analysis")
    ```
  </Tab>
</Tabs>

### Use Groq for speed

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export MODEL_NAME="groq/llama3-70b"
    export GROQ_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "groq/llama3-70b"
    if not os.getenv("GROQ_API_KEY"):
        raise EnvironmentError("Set GROQ_API_KEY in your environment")

    agent = Agent(
        name="Speed Assistant",
        instructions="You provide fast responses"
    )

    result = agent.start("Quick summary of quantum physics")
    ```
  </Tab>
</Tabs>

### Use Google Gemini

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export MODEL_NAME="google/gemini-1.5-pro"
    export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
    if not os.getenv("GOOGLE_API_KEY"):
        raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

    agent = Agent(
        name="Gemini Assistant",
        instructions="You are powered by Google Gemini"
    )

    result = agent.start("Analyze this complex problem")
    ```
  </Tab>
</Tabs>

### Use Azure OpenAI / Bedrock via LiteLLM

<Tabs>
  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export OPENAI_BASE_URL="https://your-litellm-proxy/v1"
    export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
    export MODEL_NAME="azure/gpt-4"
    python your_agent.py
    ```
  </Tab>

  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    import os
    from praisonaiagents import Agent

    # Configure LiteLLM for Azure
    os.environ["OPENAI_BASE_URL"] = "https://your-litellm-proxy/v1"
    if not os.getenv("OPENAI_API_KEY"):
        raise EnvironmentError("Set OPENAI_API_KEY in your environment")
    os.environ["MODEL_NAME"] = "azure/gpt-4"

    agent = Agent(
        name="Azure Assistant",
        instructions="You are using Azure OpenAI via LiteLLM"
    )

    result = agent.start("Analyze this data")
    ```
  </Tab>
</Tabs>

***

## User Interaction Flow

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant CLI
    participant Resolver
    participant Agent
    participant Endpoint
    
    User->>CLI: Set environment variables
    User->>CLI: Start agent
    CLI->>Resolver: resolve_llm_endpoint()
    Resolver->>Resolver: Check precedence order
    Resolver-->>CLI: Return LLMEndpoint config
    CLI->>Agent: Initialize with config
    Agent->>Endpoint: Send LLM request
    Endpoint-->>Agent: Return response
    Agent-->>User: Return result
```

***

## Best Practices

<AccordionGroup>
  <Accordion title="Set OPENAI_BASE_URL, not OPENAI_API_BASE">
    `OPENAI_BASE_URL` is the standard OpenAI SDK environment variable and has the highest precedence. Use this for all new configurations rather than the legacy `OPENAI_API_BASE`.
  </Accordion>

  <Accordion title="Empty string ≠ unset">
    An empty string value is skipped during resolution, and the next variable in precedence order is tried. To disable a variable, unset it completely rather than setting it to an empty string.

    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    # This skips OPENAI_BASE_URL and tries OPENAI_API_BASE
    export OPENAI_BASE_URL=""
    export OPENAI_API_BASE="https://proxy.com/v1"

    # This uses OPENAI_BASE_URL
    unset OPENAI_BASE_URL
    export OPENAI_API_BASE="https://proxy.com/v1"
    ```
  </Accordion>

  <Accordion title="Use .env files for local dev">
    Create a `.env` file in your project root for local development:

    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    # .env
    OPENAI_BASE_URL=http://localhost:11434/v1
    MODEL_NAME=llama3
    # OPENAI_API_KEY not needed for Ollama
    ```

    Load it in your Python code:

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from dotenv import load_dotenv
    load_dotenv()

    from praisonaiagents import Agent
    # Environment variables are now loaded
    ```
  </Accordion>

  <Accordion title="Realtime/WebSocket endpoints">
    For realtime features, WebSocket URLs are auto-derived from HTTP URLs. The system automatically:

    * Converts `https://` to `wss://`
    * Strips `/v1` suffix to avoid `/v1/v1/realtime`
    * Appends the appropriate realtime path

    You only need to set `OPENAI_BASE_URL` - the realtime endpoint is handled automatically.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card title="AutoGen Config List" icon="list-tree" href="/docs/features/llm-autogen-config-list">
    Build an AutoGen-style config\_list from the resolved endpoint
  </Card>

  <Card title="Standalone LLM Modules" icon="plug" href="/docs/features/standalone-llm-modules">
    All LLM modules available without the wrapper package
  </Card>

  <Card title="LLM Configuration" icon="cog" href="/docs/configuration/llm-config">
    Complete LLM configuration options
  </Card>

  <Card title="Models" icon="brain" href="/docs/models">
    Supported models and providers
  </Card>
</CardGroup>

### Provider Pages

<CardGroup cols={2}>
  <Card title="Anthropic" icon="robot" href="/docs/models/anthropic">
    Claude models and configuration
  </Card>

  <Card title="Groq" icon="zap" href="/docs/models/groq">
    High-speed inference setup
  </Card>

  <Card title="Google" icon="brain" href="/docs/models/google">
    Gemini models and API access
  </Card>

  <Card title="Cohere" icon="text" href="/docs/models/cohere">
    Language model configuration
  </Card>

  <Card title="OpenRouter" icon="route" href="/docs/models/openrouter">
    Multi-provider router setup
  </Card>

  <Card title="Ollama" icon="server" href="/docs/models/ollama">
    Local model deployment
  </Card>
</CardGroup>

*C7 note: As of PR #2550, the tool-discovery pipeline (`tool_resolver`), safe loader (`_safe_loader`), framework probes (`_framework_availability`), and plugin registry (`tool_registry`) also moved to `praisonai_code` — completing the same C7 arc as LLM config. See [Tool Discovery Order](/docs/features/tool-discovery-order) and [Local Tools Loading](/docs/features/local-tools-loading).*