Knowledge Troubleshooting

Diagnose common knowledge/RAG failure modes: indexed but not grounded, missing chonkie, wrong chunks

Knowledge troubleshooting maps the most common “my agent has documents but ignores them” symptoms to fixes you can verify in seconds.

from praisonaiagents import Agent

agent = Agent(
    name="Docs Assistant",
    instructions="Answer using the knowledge base only.",
    knowledge=["policy.txt"],
)

agent.start("What is the daily API-call limit?")

The user passes files and asks a question; this guide confirms the answer actually came from those files — not from the model’s memory.

graph TB
    START[👤 knowledge=... configured]
    INSTALL{pip install<br/>praisonaiagents[knowledge]<br/>succeeded?}
    SEARCH{agent.knowledge.search<br/>returns the marker chunk?}
    CHAT{agent.chat echoes<br/>the marker + fact?}
    CHONKIE[❌ ModuleNotFoundError: chonkie]
    INDEX[❌ Indexing / embedding issue]
    GROUND[❌ Grounding failure]
    OK[✅ RAG is grounded]

    START --> INSTALL
    INSTALL -->|No| CHONKIE
    INSTALL -->|Yes| SEARCH
    SEARCH -->|No| INDEX
    SEARCH -->|Yes| CHAT
    CHAT -->|No| GROUND
    CHAT -->|Yes| OK

    classDef start fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef warn fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef ok fill:#10B981,stroke:#7C90A0,color:#fff

    class START start
    class INSTALL,SEARCH,CHAT decision
    class CHONKIE,INDEX,GROUND warn
    class OK ok

Quick Start

1. Install the knowledge extra

   pip install "praisonaiagents[knowledge]"

2. Run the three-check marker test

   Follow the decision tree top to bottom: install → search → chat. All three must pass before you trust RAG.

Symptom: “Indexed OK, but agent answers generically”

Search returns the right chunks, but the agent answers from its own memory — a silent hallucination against your own docs.

Before (broken):

Q: What is policy ZEBRA-QUOTA-9917 daily limit?
A: I don't have access to a specific policy with that identifier.

After (grounded):

Q: What is policy ZEBRA-QUOTA-9917 daily limit?
A: Policy ZEBRA-QUOTA-9917 limits daily API calls to 42.

Verification recipe

Run this marker test against a fresh install. It proves both that search finds the chunk and that chat() actually uses it.

from pathlib import Path
from praisonaiagents import Agent

Path("policy.txt").write_text(
    "Company policy ZEBRA-QUOTA-9917 limits daily API calls to 42."
)
agent = Agent(
    instructions="Answer using the knowledge base only.",
    llm="gpt-4o-mini",
    knowledge=["policy.txt"],
)
search_ok = any("ZEBRA-QUOTA-9917" in str(r) for r in agent.knowledge.search("ZEBRA-QUOTA-9917"))
answer = agent.chat("What is policy ZEBRA-QUOTA-9917 daily limit?")
chat_ok = "ZEBRA-QUOTA-9917" in answer and "42" in answer
print(f"SEARCH_OK={search_ok}  CHAT_OK={chat_ok}")

Why a unique marker matters

Generic questions can get a lucky correct guess from the model’s parametric knowledge, giving a false pass. Use a non-memorisable token like ZEBRA-QUOTA-9917 so a correct answer can only come from your documents.

Interim workaround (pinned to an older release)

If you cannot upgrade yet, inject the retrieved context into the prompt yourself:

chunks = agent.knowledge.search(user_query)
context = "\n".join(str(c) for c in chunks)
answer = agent.chat(f"Context:\n{context}\n\nQuestion: {user_query}")

Symptom: ModuleNotFoundError: No module named 'chonkie'

The knowledge extra (which bundles the chonkie chunker) is not installed.

pip install "praisonaiagents[knowledge]"

Symptom: chat is grounded but the wrong chunks come back

CHAT_OK passes, but the cited chunks are off-topic — a retrieval-quality problem, not a grounding one. Tune reranking and chunker settings.

Best Practices

Related