llm="local" and the agent finds a running local model server, picks the best model on it, and configures itself — no URL, no model name, no key.
llm="local". Passing gpt-4o, ollama/llama3.2, or anthropic/… runs zero probes.
Quick Start
1
Discover whatever is running
Start any local server (Ollama, llama.cpp, LM Studio, vLLM), then run four lines.No
OPENAI_BASE_URL, no ollama/ prefix, no model name.2
Pin an engine, let it pick the model
Name the engine and the resolver chooses the best model on it.Swap
ollama for llama_cpp, lm_studio, vllm, mlx_lm, or transformers_serve.3
Pin engine and model
Name both for reproducibility.
How It Works
The agent calls the resolver, which probes loopback ports, identifies the engine, selects a model, and returns a target the normal LLM path uses unchanged. The resolver never infers identity from a port — a server matches only when its probe rules hold. This prevents silently talking to the wrong server.Precedence ladder
The resolver checks sources in a fixed order. The first three are authoritative: if one names a server that does not answer, resolution raises instead of quietly probing another port.Which option should I pick?
Pick the shortest spec that gives you the certainty you need.Supported local engines
Identity is decided by probe rules, never by the port — several engines share:8080 and :8000.
llama_cpp and mlx_lm both default to :8080; vllm and transformers_serve both default to :8000. The “absent” rules (e.g. /props absent, /version absent) separate the co-tenants on each port.Environment Variables
Every source is optional. The three authoritative sources raise when they name an unreachable server.How model selection works
When you do not name a model, the resolver ranks the server’s models and picks the best one to chat with.- Tools-capable models rank first — an agent that can call tools is preferred.
- Embedding-only models rank last — an embedder cannot hold a conversation, so it is never picked by accident.
- Ties break by recency — among equally-capable models, the newest wins.
qwen3:0.6b, all-minilm, nomic-embed-text, and mxbai-embed-large, the resolver picks qwen3:0.6b — even when the embedders are newer — because the three embed-only models are ranked last.
An explicitly named model must actually be served.
llm="local:ollama/does-not-exist" raises ModelNotAvailableError and lists what is available, rather than sending a phantom model to the server.Errors and gotchas
The resolver fails loudly and specifically instead of guessing.NoLocalEngineError — nothing answered
NoLocalEngineError — nothing answered
No server responded on any probed port. Start one (
ollama serve, then ollama pull qwen3:0.6b) or set PRAISONAI_LOCAL_BASE_URL to its address.EngineUnreachableError — a named server is silent
EngineUnreachableError — a named server is silent
An authoritative source (the
spec, PRAISONAI_LOCAL_BASE_URL, or OLLAMA_HOST) named a server that did not answer. Because it was named explicitly, the resolver does not fall through to another port — it raises so you fix the real target.HostHeaderRejectedError — Ollama refused the Host header
HostHeaderRejectedError — Ollama refused the Host header
The server returned a bodyless HTTP 403. Ollama rejects any request whose
Host header is not localhost or an IP address. Set OLLAMA_HOST on the server to allow the origin.ModelNotAvailableError — that model is not served
ModelNotAvailableError — that model is not served
A specific model was requested but the engine does not have it. Pull it or name one from the list the error prints.
InvalidLocalSpecError — the spec could not be parsed
InvalidLocalSpecError — the spec could not be parsed
The
llm="local:..." string was malformed. Expected forms: "local", "local:<engine>", "local:<engine>/<model>", a base URL, or "<url>#<model>".OLLAMA_HOST port-80 trap
OLLAMA_HOST port-80 trap
Setting Use
OLLAMA_HOST to a scheme with no port means port 80, not 11434. The resolver names this exact trap in the error:OLLAMA_HOST=127.0.0.1:11434 (bare host defaults to 11434) instead.Performance notes
Resolution is opt-in and cheap.- Discovery fires only for
llm="local"— cloud models trigger zero probes. - Importing
praisonaiagentsdoes not import the resolver, so import time is unchanged. - A full resolve issues a handful of loopback HTTP requests and caches the result for 30 s (
PRAISONAI_LOCAL_TTL). - The whole scan is budgeted (
PRAISONAI_LOCAL_TIMEOUT, default 1.5 s; per-probe 0.4 s), so nothing stalls when no server is listening.
Explicit vs implicit local
llm="local" is the explicit request — you ask for a local model by name. The keyless local-first fallback is the implicit form — it only kicks in when no cloud key is set and you named no model at all.
Related
Models
Provider auto-detection and the full model-selection precedence.
Keyless Local-First Run
The implicit local fallback when no cloud key is set.
Local Models
Point PraisonAI at Ollama or any OpenAI-compatible server.
Ollama
Use Ollama models with PraisonAI.