Ollama

Use local Ollama models with PraisonAI Agents

Diagram

Common pitfall: naming the model matters

Setting only OPENAI_BASE_URL is not enough. With no model named, the OpenAI default gpt-4o-mini is sent to Ollama, which answers:

404 - {'error': {'message': "model 'gpt-4o-mini' not found", 'type': 'not_found_error'}}

Always name the model with the ollama/ prefix — it selects the OllamaAdapter (tool-call repair, tool-result formatting, small-model streaming rules) instead of the plain OpenAIClient.

python

from praisonaiagents import Agent

# Simplest — discovery finds Ollama and picks the model.
Agent(instructions="You are a helpful assistant", llm="local").start("Hello!")

# Or name the model yourself.
Agent(instructions="You are a helpful assistant", llm="ollama/llama3.2").start("Why is the sky blue?")

bash

export OPENAI_MODEL_NAME=ollama/llama3.2

Custom Ollama host

If Ollama runs on a non-default host or port, pass base_url= or set OLLAMA_API_BASE (preferred) or OLLAMA_HOST:

python

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2",
    base_url="http://192.168.1.10:11434",
)
agent.start("Why is the sky blue?")

When the CLI runs an explicit ollama/<model>, it needs no key — it checks the first of these endpoints answers:

The agent file’s llm.base_url (only when --model is not passed)
OLLAMA_API_BASE
OLLAMA_HOST
http://127.0.0.1:11434

bash

export OLLAMA_API_BASE="http://gpu-box:11434"
praisonai run --model ollama/llama3.2 "Hello"

If Ollama isn’t running

The CLI stops with a clear error instead of asking for an API key:

Error: Ollama isn't reachable at http://127.0.0.1:11434 — start it with `ollama serve` (or set OLLAMA_API_BASE).

Start the server with ollama serve, or point OLLAMA_API_BASE at the right host.

Other local runtimes

The same provider/model shape works for other local runtimes:

Runtime	Prefix	Notes
Ollama	`ollama/`	Tool-call repair via `OllamaAdapter`
Ollama (tool calling)	`ollama_chat/`	Recommended for tool-using models
LM Studio	`lm_studio/`	Point `base_url` at the LM Studio server
vLLM	`vllm/` or `hosted_vllm/`	Depending on server mode

Diagnostic warning

When a non-OpenAI endpoint is configured via OPENAI_BASE_URL or OPENAI_API_BASE and no model is named, the SDK emits this one-time warning per process:

No model specified, so the OpenAI default 'gpt-4o-mini' will be sent to
http://localhost:11434/v1. That endpoint probably does not serve it. Set
OPENAI_MODEL_NAME to a model it does serve (e.g. OPENAI_MODEL_NAME=ollama/llama3.2),
or pass llm= explicitly.

Silence it by naming a model. The warning does not fire for real OpenAI, for an explicit llm=, or when a provider credential (e.g. OLLAMA_HOST) already resolves a prefixed default. It fires once per process, so building many agents in a loop does not repeat it.

Environment Variables

Setting MODEL_NAME=ollama/llama3.2 is enough — the default base URL http://localhost:11434/v1 is used automatically. The OLLAMA_API_KEY environment variable is consulted; OPENAI_API_KEY is not used as a fallback (though most local Ollama setups don’t need an API key).

If all you want is Ollama’s default llama3.2, setting OLLAMA_HOST alone is enough — no MODEL_NAME, no llm=. Agent(...) picks ollama/llama3.2 and routes it through litellm. See Provider Auto-Detection.

python

import os
from praisonaiagents import Agent

os.environ["OLLAMA_HOST"] = "http://localhost:11434"

# No MODEL_NAME, no llm= — the Agent picks ollama/llama3.2.
agent = Agent(instructions="You run locally via Ollama")
agent.start("2+2")

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "ollama/llama3.2"
# API key usually not needed for local Ollama

agent = Agent(name="Local Agent", instructions="You run locally via Ollama")

Direct provider-registry route

Since PR #4800, the ollama and ollama_chat prefixes are registered providers — you can build a provider directly, no environment variables needed:

python

from praisonai.llm import create_llm_provider

provider = create_llm_provider("ollama/llama3.2")
# or, for tool-calling models:
provider = create_llm_provider("ollama_chat/llama3.2")

Use ollama_chat/… when the model needs to call tools — it is LiteLLM’s recommended prefix for Ollama tool calling. Everything else keeps using ollama/….

Tool-calling models: use `ollama_chat/…`

For any Ollama model you plan to use with tools, prefer the ollama_chat/… prefix over the bare ollama/…. Both hit the same Ollama server, but ollama_chat/… is LiteLLM’s recommended spelling for tool calling and — since PR #4798 — PraisonAI routes it through the same OllamaAdapter used by ollama/…, with max_tool_repairs=2. A small local model that emits a malformed tool call now gets two automatic repair attempts instead of failing the conversation.

python

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",   # tool-calling prefix
    tools=[get_weather],
)
agent.start("What is the weather in Paris?")

No environment variables or manual adapter config required — the prefix alone selects the right adapter.

Argument filtering on every response path

When a small local model emits tool call arguments that don’t belong to the tool’s signature (a common failure mode of weak Ollama models), PraisonAI drops the extra arguments before dispatch on all three response paths — sync, async, and streaming. PR #4810 closed the last streaming-path gap, so a streamed tool call is now filtered the same way sync and async already were. The filter is a no-op for every other provider.

python

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",
    tools=[get_weather],
    stream=True,  # streaming is now protected too
)
# If the model emits {"city": "Paris", "stock_symbol": "AAPL"},
# the tool is called with just {"city": "Paris"} — the stray argument is dropped.
agent.start("What is the weather in Paris?")

String arguments that match a tool name are preserved

Ollama’s same-turn chaining substitutes a previous tool’s name — when it appears in a later argument value — with that earlier tool’s result. Since PraisonAI PR #5359, the substitution respects the tool’s schema: a parameter declared as string (including Optional[str] and Union[str, ...] shapes produced by tools/schema.py) keeps its literal value even when it equals a prior tool’s name. Non-string parameters (integer, number, boolean, object) still get the reference resolved as before.

python

# Tool 'lookup' declares city as a string and count as an integer:
#   {"properties": {"city": {"type": "string"}, "count": {"type": "integer"}}}
# An earlier turn produced get_weather -> 42, so the mapping is {"get_weather": 42}
# and the model sent {"city": "get_weather", "count": "get_weather"}.

# Before PR #5359: city was overwritten to 42 -> lookup(city=42) went out.
# After PR #5359:  city (declared string) stays "get_weather";
#                  count (integer) is still resolved to 42.

This fires only for ollama/… and ollama_chat/… routes — every other provider is unaffected.

Tool calls emitted as plain text

Some local models answer a tool call as text in the response body instead of using the tool-call field. OllamaAdapter recovers that text and dispatches the tool; other providers (OpenAI, Anthropic, Gemini) do not — they treat a JSON-looking answer as plain text.

Since PraisonAI PR #5215, three text dialects are recovered — whole-content JSON, <tool_call>{...}</tool_call> blocks, and <function=name>{...}</function> blocks — all guarded by a fenced-code mask (a call shown as documentation never runs). The two tag dialects also require an advertised-name allowlist (a hallucinated name never runs); the whole-content JSON path dropped that requirement in PR #5359, so any dict with a name field is recoverable there.

Recovery fires on both the sync path (agent.start() / agent.chat()) and the async path (agent.astart() / agent.achat()). PraisonAI PR #4940 closed the async-path gap — before that release, the same agent silently stopped calling the tool the moment the caller awaited instead of called. The streaming path does not (yet) recover text-emitted tool calls on any adapter.

See OpenAI-Compatible → Local prefix route for how this compares across adapters, and Tool Call Self-Repair → Text-based tool-call recovery for the full behaviour spec.

How Ollama is handled differently

Ollama’s protocol has known quirks. The OllamaAdapter compensates so your agent code doesn’t have to know about any of them.

Ollama behaviour	What the adapter does
Returns empty content after a tool call	Detects it and prompts the model for a final answer instead of ending the turn
Doesn’t reliably stream tool calls	Disables streaming-with-tools for Ollama; falls back to non-streaming for that path
Small models emit tool calls as JSON text	Parses the JSON and dispatches the tool call anyway (up to 2 repair attempts) on sync and async response paths
Sends unsupported tool arguments during streaming	Strips them before the tool runs
Tool results delivered as `role: "tool"` confuse small models	Rewrites the result as a `role: "user"` natural-language turn so the model reads it correctly
Tool returned an error	Rewrites as an apology-oriented instruction, not the raw error string

All of this fires automatically when the llm= string routes to the OllamaAdapter (any ollama/…, ollama_chat/…, or the local auto-discovery when it finds Ollama).

Diagram

Small tool-capable model (verified)

For CI, demos, and slower machines, qwen3:0.6b is the smallest Ollama model verified to complete a tool round-trip reliably.

python

from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

agent = Agent(
    name="Calculator",
    llm={
        "model": "ollama/qwen3:0.6b",
        "force_tool_usage": "always",
        "temperature": 0,
    },
    tools=[add],
)
agent.chat("Compute 17 + 25. You MUST use the calculator tool.")

It is 522 MB and carries the tools capability, unlike some sub-1B alternatives (smollm2:135m has no tools capability and cannot call tools at all).

Measured behaviour against a real Ollama server (PR #4803):

Configuration	Trials	Tool invoked + correct
Default	4	3 / 4
`force_tool_usage="always"` + `temperature=0`	6	6 / 6

The 6/6 configuration is the same one the CI contract test uses.

Multi-step arithmetic (e.g. (17+25)+(8+9)) at this size is unreliable — 0/2 in the same measurements, one trial hung > 90 s. For chained tool calls, prefer a larger model or split the request.

Models

Run models locally with Ollama. Popular options:

Recommended: ollama/llama3.2 (latest Llama)
Reasoning: ollama/deepseek-r1 (reasoning model)
Small: ollama/qwen3 (efficient)
Code: ollama/codellama (coding tasks)
Tool calling: ollama_chat/qwen3 (LiteLLM-recommended prefix; auto-repair on)
Tool calling (small, verified): ollama/qwen3:0.6b (522 MB; use force_tool_usage="always" + temperature=0 — see “Small tool-capable model” above)

Setup

bash

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Start Ollama server
ollama serve

# Pull a model
ollama pull llama3.2

Python

python

# No API key needed - runs locally
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2"
)
agent.start("Explain deep learning")

With Tools

python

from praisonaiagents import Agent

def read_file(path: str) -> str:
    """Read a file's contents."""
    with open(path, 'r') as f:
        return f.read()

agent = Agent(
    instructions="You are a code assistant",
    llm="ollama/codellama",
    tools=[read_file]
)
agent.start("Read and explain main.py")

Multi-Agent

python

from praisonaiagents import Agent, Task, AgentTeam

researcher = Agent(
    instructions="You research topics thoroughly",
    llm="ollama/llama3.2"
)
writer = Agent(
    instructions="You write clear summaries",
    llm="ollama/qwen3"
)

task1 = Task(description="Research Python best practices", agent=researcher)
task2 = Task(description="Write a guide", agent=writer)

agents = AgentTeam(agents=[researcher, writer], tasks=[task1, task2])
agents.start()

DeepSeek Reasoning

python

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a problem solver",
    llm="ollama/deepseek-r1"
)
agent.start("Solve this math problem: What is 15% of 240?")

CLI

bash

# Basic prompt
python -m praisonai "Explain AI" --ollama llama3.2

# With specific model
python -m praisonai "Write code" --llm ollama/codellama

# Run agents.yaml
python -m praisonai

YAML

yaml

framework: praisonai
topic: Local AI development
agents:
  coder:
    role: Software Developer
    goal: Write clean code
    instructions: You are an expert programmer
    llm:
      model: ollama/codellama
    tasks:
      code_task:
        description: Write a Python function to sort a list
        expected_output: Clean, documented Python code

  reviewer:
    role: Code Reviewer
    goal: Review and improve code
    instructions: You review code for best practices
    llm:
      model: ollama/llama3.2
    tasks:
      review_task:
        description: Review the code and suggest improvements
        expected_output: Code review with suggestions

Ollama

Common pitfall: naming the model matters

python

bash

Custom Ollama host

If Ollama isn’t running

Other local runtimes

Diagnostic warning

Environment Variables

Direct provider-registry route

Tool-calling models: use ollama_chat/…

Argument filtering on every response path

String arguments that match a tool name are preserved

Tool calls emitted as plain text

How Ollama is handled differently

Small tool-capable model (verified)

Models

Setup

Python

With Tools

Multi-Agent

DeepSeek Reasoning

CLI

YAML

Tool-calling models: use `ollama_chat/…`