Local Memory & Knowledge

Run agent memory and knowledge fully local with Ollama embeddings

Keep both memory and knowledge on your own machine — a local LLM answers, a local Ollama embedder builds the vectors, and a local vector store holds them.

Auto-selection with llm="local"

Since PR #4870, Agent(llm="local", memory=..., knowledge=[...]) auto-configures a local embedder — no embedder block needed.

from praisonaiagents import Agent

agent = Agent(
    instructions="Answer from the handbook",
    llm="local",
    knowledge=["handbook.pdf"],
)
agent.start("What's the refund policy?")

The agent:

1. Resolves a running local server (Ollama, llama.cpp, LM Studio, vLLM).
2. Picks an embedding-capable model from the server’s listing (preferring nomic-embed-text, then mxbai-embed-large, bge-m3, snowflake-arctic-embed, all-minilm). Fourteen locally-served embedders now carry measured dimensions — see Vector store is now sized correctly.
3. Sets the embedder to point at the same local server.

Pin a specific embedder with PRAISONAI_LOCAL_EMBED_MODEL=<name> (accepts a bare name like all-minilm or the tagged id all-minilm:latest).

If the local server serves no embedding model, the agent logs a warning and does not silently fall back to a remote provider — pull one (ollama pull nomic-embed-text) or configure the block manually below.

sequenceDiagram
    participant Agent
    participant Resolver as local resolver
    participant Server as local server
    participant Store as vector store

    Agent->>Resolver: llm="local", knowledge=[...]
    Resolver->>Server: list models
    Server-->>Resolver: models[]
    alt embedding model present
        Resolver-->>Agent: pick nomic-embed-text
        Agent->>Server: embed(chunks)
        Server-->>Agent: vectors
        Agent->>Store: write
    else no embedder available
        Resolver-->>Agent: warn: no local embedder
        Note over Agent: does NOT silently fall back to OpenAI
    end

    classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef success fill:#10B981,stroke:#7C90A0,color:#fff
    classDef warn fill:#F59E0B,stroke:#7C90A0,color:#fff

    class Agent agent
    class Resolver,Server process
    class Store success

An explicitly configured embedder always wins — auto-selection only fills a gap.

Quick Start

1. Pull the embedder

   ollama pull nomic-embed-text

2. Local memory

   The embedder block below is only needed when the LLM is not "local", or when you want to override the auto-selected model.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="Remember facts across sessions.",
       memory={
           "provider": "mongodb",
           "config": {
               "connection_string": "mongodb://localhost:27017/",
               "database": "praisonai",
               "use_vector_search": True,
               "embedder": {
                   "provider": "ollama",
                   "config": {"model": "nomic-embed-text"},
               },
           },
       },
   )
   agent.start("Remember that my favourite colour is teal")

3. Add local knowledge

   The embedder block below is only needed when the LLM is not "local", or when you want to override the auto-selected model.

   from praisonaiagents import Knowledge
   
   kb = Knowledge(config={
       "vector_store": {
           "provider": "mongodb",
           "config": {
               "connection_string": "mongodb://localhost:27017/",
               "database": "praisonai",
               "collection": "handbook",
           },
       },
       "embedder": {
           "provider": "ollama",
           "config": {"model": "nomic-embed-text"},
       },
   })
   
   kb.add("Refunds are processed within 14 days.")
   kb.search("refund window")

How It Works

The same embedder block drives memory and knowledge — embed, store, and query all stay on your machine.

The SDK now sizes the vector index at the model’s real dimension — nomic-embed-text is 768, not the 1536 default it was silently written as before PR #4802.

Vector store is now sized correctly

Since PR #4924, the local embedder config carries embedding_dims all the way to the vector store — sourced from the running server first (via /api/show), then from PraisonAI’s dimension table, and omitted rather than guessed when unknown.

A guessed width is worse than none: an index built at the wrong width either rejects every write or silently returns stale nearest neighbours. Before the fix, a store built for bge-m3 (1024) was silently sized at 1536 — every write to a fresh Chroma collection failed.

Measured widths for locally-served embedders:

The mem0 provider config for a local embedder now carries embedding_dims alongside model:

embedder = {
    "provider": "ollama",
    "config": {"model": "bge-m3", "embedding_dims": 1024},
}

When the probe supplies a width, PraisonAI uses it; when it doesn’t, the table fills the gap; when the model is unknown, the field is omitted so the store infers the width from the first vector — never the 1536 default.

Choosing a Local Embedder

Pick by what matters most: speed, balance, or quality.

Common Patterns

Local memory with a local LLM:

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="ollama/llama3.2",
    memory={
        "provider": "mongodb",
        "config": {
            "connection_string": "mongodb://localhost:27017/",
            "database": "praisonai",
            "use_vector_search": True,
            "embedder": {"provider": "ollama", "config": {"model": "nomic-embed-text"}},
        },
    },
)
agent.start("Remember I prefer metric units")

Local knowledge with the Knowledge class:

from praisonaiagents import Knowledge

kb = Knowledge(config={
    "vector_store": {
        "provider": "mongodb",
        "config": {
            "connection_string": "mongodb://localhost:27017/",
            "database": "praisonai",
            "collection": "handbook",
        },
    },
    "embedder": {"provider": "ollama", "config": {"model": "nomic-embed-text"}},
})
kb.add("Onboarding takes three steps: sign in, verify email, join a team.")
kb.search("onboarding steps")

Both together — one local LLM for memory, one local embedder for knowledge:

from praisonaiagents import Agent, Knowledge

local_embedder = {"provider": "ollama", "config": {"model": "nomic-embed-text"}}

kb = Knowledge(config={
    "vector_store": {
        "provider": "mongodb",
        "config": {
            "connection_string": "mongodb://localhost:27017/",
            "database": "praisonai",
            "collection": "handbook",
        },
    },
    "embedder": local_embedder,
})
kb.add("Support hours are 9-5 UTC.")

agent = Agent(
    name="assistant",
    llm="ollama/llama3.2",
    memory={
        "provider": "mongodb",
        "config": {
            "connection_string": "mongodb://localhost:27017/",
            "database": "praisonai",
            "use_vector_search": True,
            "embedder": local_embedder,
        },
    },
)
agent.start("What did we agree on last session?")

Best Practices

Related