> ## Documentation Index > Fetch the complete documentation index at: https://praison.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Local Model Resolver > Point an Agent at whatever local model server is running with llm="local" — no URL, no model name, no key Set `llm="local"` and the agent finds a running local model server, picks the best model on it, and configures itself — no URL, no model name, no key. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent(instructions="You are helpful", llm="local") agent.start("Why is the sky blue?") ``` ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph LR subgraph "Local Model Resolver" You[📝 llm="local"] --> Resolve[🔍 resolve] Resolve --> Engine[🖥️ Discovered engine] Engine --> Answer[✅ Answer] end classDef input fill:#8B0000,stroke:#7C90A0,color:#fff classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff classDef engine fill:#189AB4,stroke:#7C90A0,color:#fff classDef output fill:#10B981,stroke:#7C90A0,color:#fff class You input class Resolve process class Engine engine class Answer output ``` Discovery fires **only** when `llm="local"`. Passing `gpt-4o`, `ollama/llama3.2`, or `anthropic/…` runs **zero** probes. ## Quick Start Start any local server (Ollama, llama.cpp, LM Studio, vLLM), then run four lines. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent(instructions="You are helpful", llm="local") agent.start("Why is the sky blue?") ``` No `OPENAI_BASE_URL`, no `ollama/` prefix, no model name. Name the engine and the resolver chooses the best model on it. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent(instructions="You are helpful", llm="local:ollama") agent.start("Explain gravity in one sentence") ``` Swap `ollama` for `llama_cpp`, `lm_studio`, `vllm`, `mlx_lm`, or `transformers_serve`. Name both for reproducibility. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent(instructions="You are helpful", llm="local:ollama/qwen3:0.6b") agent.start("Summarize the water cycle") ``` *** ## How It Works The agent calls the resolver, which probes loopback ports, identifies the engine, selects a model, and returns a target the normal LLM path uses unchanged. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} sequenceDiagram participant User participant Agent participant Resolver as resolve() participant Engine as Local Engine User->>Agent: Agent(llm="local") Agent->>Resolver: resolve(spec) Resolver->>Engine: probe loopback ports Engine-->>Resolver: identity + model list Resolver-->>Agent: LocalTarget (model, base_url, key) User->>Agent: agent.start("...") Agent->>Engine: chat completion Engine-->>User: Answer %% classDef not applied to sequence participants ``` The resolver never infers identity from a port — a server matches only when its probe rules hold. This prevents silently talking to the wrong server. ### Precedence ladder The resolver checks sources in a fixed order. The **first three are authoritative**: if one names a server that does not answer, resolution **raises** instead of quietly probing another port. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph TB A[spec argument
llm="local:..."] -->|not set| B[PRAISONAI_LOCAL_BASE_URL] B -->|not set| C[OLLAMA_HOST] C -->|not set| D[OPENAI_BASE_URL / OPENAI_API_BASE
only when local] D -->|not set| E[probe-table scan
Ollama, llama.cpp, LM Studio, vLLM, …] A -.authoritative.-> X[❌ raises EngineUnreachableError
if named server is silent] B -.authoritative.-> X C -.authoritative.-> X classDef auth fill:#8B0000,stroke:#7C90A0,color:#fff classDef opt fill:#6366F1,stroke:#7C90A0,color:#fff classDef scan fill:#189AB4,stroke:#7C90A0,color:#fff classDef err fill:#F59E0B,stroke:#7C90A0,color:#fff class A,B,C auth class D opt class E scan class X err ``` *** ## Which option should I pick? Pick the shortest spec that gives you the certainty you need. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph TB Q1{Only one server
running?} -->|Yes| S1[llm="local"] Q1 -->|No / unsure| Q2{Care which
engine?} Q2 -->|Yes| Q3{Care which
model?} Q2 -->|No| S1 Q3 -->|No| S2[llm="local:vllm"] Q3 -->|Yes, reproducible| S3[llm="local:ollama/qwen3:0.6b"] classDef q fill:#6366F1,stroke:#7C90A0,color:#fff classDef s fill:#10B981,stroke:#7C90A0,color:#fff class Q1,Q2,Q3 q class S1,S2,S3 s ``` *** ## Supported local engines Identity is decided by probe rules, never by the port — several engines share `:8080` and `:8000`. | Engine (`llm="local:"`) | Default port | Identity probe | | ------------------------------- | ------------ | ------------------------------------------------------------------------ | | `ollama` | `11434` | `GET /` returns `Ollama is running` | | `llama_cpp` | `8080` | `GET /health` ok **and** `GET /props` has `build_info` | | `mlx_lm` | `8080` | `GET /health` ok, `GET /props` **absent**, `GET /v1/models` has `data` | | `lm_studio` | `1234` | `GET /api/v0/models` has `data` | | `vllm` | `8000` | `GET /version` has `version` | | `transformers_serve` | `8000` | `GET /version` **absent**, `GET /health` ok, `GET /v1/models` has `data` | `llama_cpp` and `mlx_lm` both default to `:8080`; `vllm` and `transformers_serve` both default to `:8000`. The "absent" rules (e.g. `/props` absent, `/version` absent) separate the co-tenants on each port. *** ## Environment Variables Every source is optional. The three authoritative sources raise when they name an unreachable server. | Variable | Purpose | Default | | ------------------------------------- | ---------------------------------------------------------------------------- | -------- | | `PRAISONAI_LOCAL_BASE_URL` | Authoritative base URL — if set and unreachable, resolution **raises** | *(none)* | | `PRAISONAI_LOCAL_ENGINE` | Pin the expected engine (`ollama`, `llama_cpp`, `lm_studio`, `vllm`, …) | *(none)* | | `PRAISONAI_LOCAL_MODEL` | Pin the model id | *(none)* | | `PRAISONAI_LOCAL_TTL` | Positive-result cache seconds | `30` | | `PRAISONAI_LOCAL_NEG_TTL` | Negative-result cache seconds | `5` | | `PRAISONAI_LOCAL_TIMEOUT` | Total resolve budget in seconds (clamped `0.05`–`30`; per-probe is `0.4`) | `1.5` | | `OLLAMA_HOST` | Authoritative Ollama host (see the port-80 gotcha below) | *(none)* | | `OPENAI_BASE_URL` / `OPENAI_API_BASE` | Honoured **only** when the host is local (`127.0.0.1`, `localhost`, private) | *(none)* | *** ## How model selection works When you do not name a model, the resolver ranks the server's models and picks the best one to chat with. * **Tools-capable models rank first** — an agent that can call tools is preferred. * **Embedding-only models rank last** — an embedder cannot hold a conversation, so it is never picked by accident. * **Ties break by recency** — among equally-capable models, the newest wins. **Worked example.** On a machine serving `qwen3:0.6b`, `all-minilm`, `nomic-embed-text`, and `mxbai-embed-large`, the resolver picks `qwen3:0.6b` — even when the embedders are newer — because the three embed-only models are ranked last. An explicitly named model must actually be served. `llm="local:ollama/does-not-exist"` raises `ModelNotAvailableError` and lists what is available, rather than sending a phantom model to the server. *** ## Errors and gotchas The resolver fails loudly and specifically instead of guessing. No server responded on any probed port. Start one (`ollama serve`, then `ollama pull qwen3:0.6b`) or set `PRAISONAI_LOCAL_BASE_URL` to its address. An authoritative source (the `spec`, `PRAISONAI_LOCAL_BASE_URL`, or `OLLAMA_HOST`) named a server that did not answer. Because it was named explicitly, the resolver does **not** fall through to another port — it raises so you fix the real target. The server returned a bodyless HTTP 403. Ollama rejects any request whose `Host` header is not `localhost` or an IP address. Set `OLLAMA_HOST` on the server to allow the origin. A specific model was requested but the engine does not have it. Pull it or name one from the list the error prints. The `llm="local:..."` string was malformed. Expected forms: `"local"`, `"local:"`, `"local:/"`, a base URL, or `"#"`. Setting `OLLAMA_HOST` to a scheme with no port means **port 80**, not 11434. The resolver names this exact trap in the error: ``` Local runtime at http://127.0.0.1:80 did not answer (refused). It was named explicitly by OLLAMA_HOST='http://127.0.0.1', so no other port was probed. Note that OLLAMA_HOST with an explicit http:// scheme and no port means port 80, not 11434. ``` Use `OLLAMA_HOST=127.0.0.1:11434` (bare host defaults to 11434) instead. *** ## Performance notes Resolution is opt-in and cheap. * Discovery fires **only** for `llm="local"` — cloud models trigger zero probes. * Importing `praisonaiagents` does not import the resolver, so import time is unchanged. * A full resolve issues a handful of loopback HTTP requests and caches the result for 30 s (`PRAISONAI_LOCAL_TTL`). * The whole scan is budgeted (`PRAISONAI_LOCAL_TIMEOUT`, default 1.5 s; per-probe 0.4 s), so nothing stalls when no server is listening. *** ## Explicit vs implicit local `llm="local"` is the **explicit** request — you ask for a local model by name. The [keyless local-first fallback](/docs/features/keyless-local-first-run) is the **implicit** form — it only kicks in when no cloud key is set and you named no model at all. | | Explicit `llm="local"` | Implicit keyless fallback | | ------------------ | ---------------------------------------------------------------------------------- | ------------------------------------- | | Trigger | You pass `llm="local"` | No cloud key **and** no `--model` | | Silent fallthrough | Never — authoritative sources raise | Falls back to `gpt-4o-mini` | | Engine choice | Full probe table (Ollama, llama.cpp, LM Studio, vLLM, mlx\_lm, transformers serve) | Ollama `/api/tags`, then `/v1/models` | *** ## Related Provider auto-detection and the full model-selection precedence. The implicit local fallback when no cloud key is set. Point PraisonAI at Ollama or any OpenAI-compatible server. Use Ollama models with PraisonAI.