LLM Module

LLM client and model utilities for agent interactions

LLM Module

The LLM module provides the core language model client and utilities for agent interactions, supporting multiple providers through LiteLLM.

Param Cluster Map

Cluster	Legacy Params	Consolidated To	Status
LLM	`llm`, `llm_config`, `function_calling_llm`	`llm=`	✅ Done

Note: base_url and api_key remain separate parameters (connection/auth constraint).

Precedence Ladder

Instance > Config > Array > Dict > String > Bool > Default

Ways to Use `llm=`

Form	Example	Description
String	`llm="gpt-4o"`	Model name
Alias	`model="gpt-4o"`	Same as `llm=`
Dict	`llm={"model": "gpt-4o", "temperature": 0.7}`	Config dict
Instance	`llm=LLM(model="gpt-4o")`	LLM instance

Legacy (Deprecated)

python

# ⚠️ DEPRECATED - still works but emits warning
Agent(llm={"temperature": 0.7})
Agent(function_calling_llm="gpt-4o-mini")

# ✅ NEW - use llm= instead
Agent(llm="gpt-4o")
Agent(model="gpt-4o")  # alias

Installation

bash

pip install praisonaiagents

Quick Start

python

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("What is machine learning?")
print(response)

Classes

LLM

Main LLM client class supporting multiple providers.

python

from praisonaiagents.llm import LLM

llm = LLM(
    model="gpt-4o-mini",
    temperature=0.7,
    max_tokens=1000
)

Constructor

Parameter	Type	Default	Description
`model`	`str`	`"gpt-4o-mini"`	Model identifier
`temperature`	`float`	`0.7`	Sampling temperature
`max_tokens`	`int`	`None`	Maximum tokens in response
`api_key`	`str`	`None`	API key (uses env var if not set)
`base_url`	`str`	`None`	Custom API base URL

Methods

Method	Description
`chat(prompt, **kwargs)`	Send a chat message
`chat_async(prompt, **kwargs)`	Async chat message
`stream(prompt, **kwargs)`	Stream response
`get_response(messages, **kwargs)`	Get response from messages

LLMContextLengthExceededException

Exception raised when context length is exceeded.

python

from praisonaiagents.llm import LLMContextLengthExceededException

try:
    response = llm.chat(very_long_prompt)
except LLMContextLengthExceededException as e:
    print(f"Context too long: {e}")

LLMResponseError

Raised by LLM.get_response() when the tool-calling loop fails mid-iteration and cannot produce a response. The original exception is chained via raise … from e, so e.__cause__ holds the underlying error.

python

from praisonaiagents import Agent
from praisonaiagents.llm import LLMResponseError

agent = Agent(name="Analyst", instructions="Analyse files using tools")

try:
    agent.start("Analyse this file")
except LLMResponseError as e:
    print(f"LLM loop failed: {e}")
    print(f"Underlying error: {e.__cause__}")

Before this behaviour landed, the same failure was silently swallowed and returned as an empty string, so agent.chat() persisted an empty assistant message and retries burned through their budget on a phantom success. Now the failure surfaces as a distinguishable exception.

Diagram

OpenAIClient

OpenAI-compatible client for direct API access.

python

from praisonaiagents.llm import OpenAIClient, get_openai_client

client = get_openai_client(model="gpt-4o-mini")
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}]
)

ModelRouter

Intelligent model routing based on task complexity.

python

from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "gpt-4-turbo"
    }
)

model = router.select(task="Simple greeting")

TaskComplexity

Level	Description
`LOW`	Simple tasks, quick responses
`MEDIUM`	Moderate complexity
`HIGH`	Complex reasoning, long context

ModelProfile

Profile for a model’s capabilities.

python

from praisonaiagents.llm import ModelProfile

profile = ModelProfile(
    name="gpt-4o-mini",
    context_length=128000,
    supports_tools=True,
    supports_vision=True,
    cost_per_1k_input=0.00015,
    cost_per_1k_output=0.0006
)

Response Types

ChatCompletion

python

from praisonaiagents.llm import ChatCompletion

# Response structure
completion.id           # Completion ID
completion.choices      # List of choices
completion.usage        # Token usage
completion.model        # Model used

ChatCompletionMessage

python

from praisonaiagents.llm import ChatCompletionMessage

message.role      # "assistant", "user", etc.
message.content   # Message content
message.tool_calls  # Tool calls if any

ToolCall

python

from praisonaiagents.llm import ToolCall

tool_call.id        # Tool call ID
tool_call.type      # "function"
tool_call.function  # Function details

CompletionUsage

python

from praisonaiagents.llm import CompletionUsage

usage.prompt_tokens      # Input tokens
usage.completion_tokens  # Output tokens
usage.total_tokens       # Total tokens

Utility Functions

supports_structured_outputs

Check if a model supports structured outputs.

python

from praisonaiagents.llm import supports_structured_outputs

if supports_structured_outputs("gpt-4o-mini"):
    # Use structured output mode
    pass

supports_streaming_with_tools

Check if a model supports streaming with tool calls.

python

from praisonaiagents.llm import supports_streaming_with_tools

if supports_streaming_with_tools("gpt-4o"):
    # Enable streaming with tools
    pass

process_stream_chunks

Process streaming response chunks.

python

from praisonaiagents.llm import process_stream_chunks

for chunk in process_stream_chunks(stream_response):
    print(chunk.content, end="")

create_routing_agent

Create an agent with model routing.

python

from praisonaiagents.llm import create_routing_agent

agent = create_routing_agent(
    name="Smart Agent",
    router=model_router
)

Usage Examples

Basic Chat

python

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("Explain quantum computing in simple terms")
print(response)

Streaming Response

python

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
for chunk in llm.stream("Write a short story"):
    print(chunk, end="", flush=True)

With Tools

python

from praisonaiagents.llm import LLM

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

llm = LLM(model="gpt-4o-mini")
response = llm.chat(
    "What's the weather in Tokyo?",
    tools=tools
)

Model Routing

python

from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "claude-3-opus"
    }
)

# Router automatically selects appropriate model
model = router.select(task="Complex mathematical proof")
llm = LLM(model=model)

Different Providers

python

from praisonaiagents.llm import LLM

# OpenAI
openai_llm = LLM(model="gpt-4o-mini")

# Anthropic
claude_llm = LLM(model="claude-3-sonnet-20240229")

# Ollama (local)
ollama_llm = LLM(model="ollama/llama3")

# Google
gemini_llm = LLM(model="gemini/gemini-pro")

# Groq
groq_llm = LLM(model="groq/llama3-70b-8192")

Environment Variables

Variable	Description
`OPENAI_API_KEY`	OpenAI API key
`ANTHROPIC_API_KEY`	Anthropic API key
`GOOGLE_API_KEY`	Google AI API key
`GROQ_API_KEY`	Groq API key
`OPENROUTER_API_KEY`	OpenRouter API key

Agent – Using LLM with agents
Telemetry – Token tracking

LLM Module

LLM Module

Param Cluster Map

Precedence Ladder

Ways to Use llm=

Legacy (Deprecated)

Installation

Quick Start

Classes

LLM

Constructor

Methods

LLMContextLengthExceededException

LLMResponseError

OpenAIClient

ModelRouter

TaskComplexity

ModelProfile

Response Types

ChatCompletion

ChatCompletionMessage

ToolCall

CompletionUsage

Utility Functions

supports_structured_outputs

supports_streaming_with_tools

process_stream_chunks

create_routing_agent

Usage Examples

Basic Chat

Streaming Response

With Tools

Model Routing

Different Providers

Environment Variables

Related

Ways to Use `llm=`