> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Module

> LLM client and model utilities for agent interactions

# LLM Module

The LLM module provides the core language model client and utilities for agent interactions, supporting multiple providers through LiteLLM.

## Param Cluster Map

| Cluster | Legacy Params                               | Consolidated To | Status |
| ------- | ------------------------------------------- | --------------- | ------ |
| LLM     | `llm`, `llm_config`, `function_calling_llm` | `llm=`          | ✅ Done |

**Note:** `base_url` and `api_key` remain **separate** parameters (connection/auth constraint).

## Precedence Ladder

`Instance > Config > Array > Dict > String > Bool > Default`

## Ways to Use `llm=`

| Form         | Example                                       | Description    |
| ------------ | --------------------------------------------- | -------------- |
| **String**   | `llm="gpt-4o"`                                | Model name     |
| **Alias**    | `model="gpt-4o"`                              | Same as `llm=` |
| **Dict**     | `llm={"model": "gpt-4o", "temperature": 0.7}` | Config dict    |
| **Instance** | `llm=LLM(model="gpt-4o")`                     | LLM instance   |

### Legacy (Deprecated)

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# ⚠️ DEPRECATED - still works but emits warning
Agent(llm={"temperature": 0.7})
Agent(function_calling_llm="gpt-4o-mini")

# ✅ NEW - use llm= instead
Agent(llm="gpt-4o")
Agent(model="gpt-4o")  # alias
```

## Installation

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
pip install praisonaiagents
```

## Quick Start

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("What is machine learning?")
print(response)
```

## Classes

### LLM

Main LLM client class supporting multiple providers.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

llm = LLM(
    model="gpt-4o-mini",
    temperature=0.7,
    max_tokens=1000
)
```

#### Constructor

| Parameter     | Type    | Default         | Description                       |
| ------------- | ------- | --------------- | --------------------------------- |
| `model`       | `str`   | `"gpt-4o-mini"` | Model identifier                  |
| `temperature` | `float` | `0.7`           | Sampling temperature              |
| `max_tokens`  | `int`   | `None`          | Maximum tokens in response        |
| `api_key`     | `str`   | `None`          | API key (uses env var if not set) |
| `base_url`    | `str`   | `None`          | Custom API base URL               |

#### Methods

| Method                             | Description                |
| ---------------------------------- | -------------------------- |
| `chat(prompt, **kwargs)`           | Send a chat message        |
| `chat_async(prompt, **kwargs)`     | Async chat message         |
| `stream(prompt, **kwargs)`         | Stream response            |
| `get_response(messages, **kwargs)` | Get response from messages |

### LLMContextLengthExceededException

Exception raised when context length is exceeded.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLMContextLengthExceededException

try:
    response = llm.chat(very_long_prompt)
except LLMContextLengthExceededException as e:
    print(f"Context too long: {e}")
```

### OpenAIClient

OpenAI-compatible client for direct API access.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import OpenAIClient, get_openai_client

client = get_openai_client(model="gpt-4o-mini")
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}]
)
```

### ModelRouter

Intelligent model routing based on task complexity.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "gpt-4-turbo"
    }
)

model = router.select(task="Simple greeting")
```

#### TaskComplexity

| Level    | Description                     |
| -------- | ------------------------------- |
| `LOW`    | Simple tasks, quick responses   |
| `MEDIUM` | Moderate complexity             |
| `HIGH`   | Complex reasoning, long context |

### ModelProfile

Profile for a model's capabilities.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ModelProfile

profile = ModelProfile(
    name="gpt-4o-mini",
    context_length=128000,
    supports_tools=True,
    supports_vision=True,
    cost_per_1k_input=0.00015,
    cost_per_1k_output=0.0006
)
```

## Response Types

### ChatCompletion

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ChatCompletion

# Response structure
completion.id           # Completion ID
completion.choices      # List of choices
completion.usage        # Token usage
completion.model        # Model used
```

### ChatCompletionMessage

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ChatCompletionMessage

message.role      # "assistant", "user", etc.
message.content   # Message content
message.tool_calls  # Tool calls if any
```

### ToolCall

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ToolCall

tool_call.id        # Tool call ID
tool_call.type      # "function"
tool_call.function  # Function details
```

### CompletionUsage

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import CompletionUsage

usage.prompt_tokens      # Input tokens
usage.completion_tokens  # Output tokens
usage.total_tokens       # Total tokens
```

## Utility Functions

### supports\_structured\_outputs

Check if a model supports structured outputs.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import supports_structured_outputs

if supports_structured_outputs("gpt-4o-mini"):
    # Use structured output mode
    pass
```

### supports\_streaming\_with\_tools

Check if a model supports streaming with tool calls.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import supports_streaming_with_tools

if supports_streaming_with_tools("gpt-4o"):
    # Enable streaming with tools
    pass
```

### process\_stream\_chunks

Process streaming response chunks.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import process_stream_chunks

for chunk in process_stream_chunks(stream_response):
    print(chunk.content, end="")
```

### create\_routing\_agent

Create an agent with model routing.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import create_routing_agent

agent = create_routing_agent(
    name="Smart Agent",
    router=model_router
)
```

## Usage Examples

### Basic Chat

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("Explain quantum computing in simple terms")
print(response)
```

### Streaming Response

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
for chunk in llm.stream("Write a short story"):
    print(chunk, end="", flush=True)
```

### With Tools

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

llm = LLM(model="gpt-4o-mini")
response = llm.chat(
    "What's the weather in Tokyo?",
    tools=tools
)
```

### Model Routing

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "claude-3-opus"
    }
)

# Router automatically selects appropriate model
model = router.select(task="Complex mathematical proof")
llm = LLM(model=model)
```

### Different Providers

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.llm import LLM

# OpenAI
openai_llm = LLM(model="gpt-4o-mini")

# Anthropic
claude_llm = LLM(model="claude-3-sonnet-20240229")

# Ollama (local)
ollama_llm = LLM(model="ollama/llama3")

# Google
gemini_llm = LLM(model="gemini/gemini-pro")

# Groq
groq_llm = LLM(model="groq/llama3-70b-8192")
```

## Environment Variables

| Variable             | Description        |
| -------------------- | ------------------ |
| `OPENAI_API_KEY`     | OpenAI API key     |
| `ANTHROPIC_API_KEY`  | Anthropic API key  |
| `GOOGLE_API_KEY`     | Google AI API key  |
| `GROQ_API_KEY`       | Groq API key       |
| `OPENROUTER_API_KEY` | OpenRouter API key |

## Related

* [Agent](/docs/sdk/praisonaiagents/agent/agent) - Using LLM with agents
* [Telemetry](/docs/sdk/praisonaiagents/telemetry) - Token tracking