> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Knowledge Base
> Advanced knowledge management system for AI agents
# Knowledge Base System
The knowledge system provides sophisticated document processing and semantic search capabilities, enabling agents to access and utilise information from various sources.
```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
subgraph Input Sources
PDF[π PDFs]
DOC[π Documents]
TXT[π Text Files]
IMG[πΌοΈ Images]
URL[π URLs]
end
subgraph Processing
CHUNK[βοΈ Chunking]
EMB[π’ Embedding]
META[π·οΈ Metadata]
end
subgraph Storage
VECTOR[(ποΈ Vector Store)]
GRAPH[(πΈοΈ Graph Store)]
end
subgraph Retrieval
SEARCH[π Semantic Search]
RERANK[π Reranking]
end
PDF --> CHUNK
DOC --> CHUNK
TXT --> CHUNK
IMG --> CHUNK
URL --> CHUNK
CHUNK --> EMB
EMB --> META
META --> VECTOR
META --> GRAPH
VECTOR --> SEARCH
GRAPH --> SEARCH
SEARCH --> RERANK
classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
classDef storage fill:#2E8B57,stroke:#7C90A0,color:#fff
classDef retrieval fill:#FF6B6B,stroke:#7C90A0,color:#fff
class PDF,DOC,TXT,IMG,URL input
class CHUNK,EMB,META process
class VECTOR,GRAPH storage
class SEARCH,RERANK retrieval
```
## Key Features
Process PDFs, documents, spreadsheets, images, and web content
Multiple strategies for optimal text segmentation
Vector-based search with optional reranking
User, agent, and run-specific knowledge scoping
Optional relationship extraction and storage
Automatic quality assessment for stored knowledge
## Quick Start
```python Agent with Knowledge theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
agent = Agent(
name="Research Assistant",
instructions="Answer questions using the knowledge base.",
knowledge={
"sources": ["research_paper.pdf", "data.txt"],
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "research_docs",
"path": ".praison"
}
}
}
)
response = agent.start("What are the key findings?")
```
```python Direct Knowledge Use theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Knowledge
# Initialise knowledge base
kb = Knowledge({
"vector_store": {
"provider": "chroma",
"config": {"collection_name": "my_knowledge"}
}
})
# Add documents
kb.add("document.pdf", user_id="user123")
kb.add("https://example.com/article", user_id="user123")
# Store text directly
kb.store("Important fact about AI", user_id="user123")
# Search
results = kb.search("AI applications", user_id="user123")
```
```python Advanced Search theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Search with reranking
results = kb.search(
query="machine learning applications",
user_id="user123",
rerank=True,
top_k=10
)
# Filter by metadata
results = kb.search(
query="technical specs",
user_id="user123",
filters={"category": "engineering"}
)
```
## Configuration Options
### Basic Configuration
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
knowledge_config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "knowledge_base",
"path": ".praison",
"distance_metric": "cosine"
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small"
}
}
}
```
### Advanced Configuration with Graph Store
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
knowledge_config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "knowledge_base",
"path": ".praison"
}
},
"graph_store": {
"provider": "neo4j",
"config": {
"url": "bolt://localhost:7687",
"username": "neo4j",
"password": "password"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini",
"temperature": 0
}
},
"reranker": {
"enabled": True,
"default_rerank": False
}
}
```
## Chunking Strategies
```python Token-based Chunking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
kb = Knowledge({
"chunker": {
"name": "token",
"chunk_size": 500,
"chunk_overlap": 50
}
})
```
Best for: Consistent chunk sizes, token-limited models
```python Sentence-based Chunking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
kb = Knowledge({
"chunker": {
"name": "sentence",
"chunk_size": 10, # sentences per chunk
"min_chunk_size": 3
}
})
```
Best for: Natural text boundaries, Q\&A systems
```python Semantic Chunking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
kb = Knowledge({
"chunker": {
"name": "semantic",
"threshold": 0.7,
"min_chunk_size": 100
}
})
```
Best for: Topic-based segmentation, research papers
```python SDPM Chunking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
kb = Knowledge({
"chunker": {
"name": "sdpm",
"max_chunk_size": 1000
}
})
```
Best for: Document structure preservation
## Document Processing
### Supported File Types
* PDF (.pdf)
* Word (.doc, .docx)
* Text (.txt)
* Markdown (.md)
* RTF (.rtf)
* Excel (.xls, .xlsx)
* CSV (.csv)
* JSON (.json)
* XML (.xml)
* Images (OCR)
* HTML pages
* Web URLs
* YouTube videos
### Processing Options
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Add with metadata
kb.add(
"research.pdf",
user_id="user123",
metadata={
"category": "AI Research",
"year": 2024,
"author": "Dr. Smith"
}
)
# Batch processing
documents = ["doc1.pdf", "doc2.txt", "doc3.md"]
for doc in documents:
kb.add(doc, user_id="user123")
# URL processing
kb.add("https://arxiv.org/pdf/2301.00000.pdf", user_id="user123")
```
## Search Features
### Basic Search
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Simple search
results = kb.search("artificial intelligence", limit=5)
# User-scoped search
results = kb.search(
query="machine learning",
user_id="user123",
limit=10
)
```
### Advanced Search Options
```python With Reranking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Enable Mem0 reranking for better relevance
results = kb.search(
query="neural networks",
user_id="user123",
rerank=True,
top_k=20 # Retrieve more before reranking
)
```
```python Hybrid Search theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Combine keyword and semantic search
results = kb.search(
query="transformer architecture",
user_id="user123",
keyword_search=True, # Better recall
filter_memories=True # Better precision
)
```
```python Metadata Filtering theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Filter by metadata
results = kb.search(
query="implementation details",
user_id="user123",
filters={
"category": "technical",
"year": {"$gte": 2023}
}
)
```
## Memory Integration
When used with agents, knowledge automatically integrates with memory:
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
agent = Agent(
name="Research Assistant",
knowledge={
"sources": ["papers/"],
"vector_store": {
"provider": "chroma",
"config": {"collection_name": "research_docs"}
}
},
memory=True # Enable memory integration
)
# Knowledge is automatically searched during conversations
response = agent.chat("What does the research say about transformers?")
```
## Graph Store Features
Graph stores enable relationship extraction and complex queries beyond simple semantic search.
### Configuration
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
knowledge_config = {
"graph_store": {
"provider": "neo4j", # or "memgraph"
"config": {
"url": "bolt://localhost:7687",
"username": "neo4j",
"password": "password"
}
},
"extract_relationships": True
}
```
### Relationship Queries
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Find related concepts
results = kb.search_graph(
"What concepts are related to transformers?",
user_id="user123"
)
# Explore connections
results = kb.search_graph(
"How is attention mechanism connected to BERT?",
user_id="user123"
)
```
## Best Practices
* Smaller chunks (100-200 tokens): Better precision
* Larger chunks (500-1000 tokens): Better context
* Match chunk size to query complexity
* Separate collections by domain
* Use metadata for filtering
* Regular cleanup of outdated content
* Enable caching for repeated queries
* Use appropriate embedding models
* Batch document processing
* Verify document processing
* Monitor search relevance
* Regular reindexing if needed
## Example: Research Assistant
```python Complete Example theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
from praisonaiagents import Knowledge
# Configure knowledge base
knowledge_config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "research_papers",
"path": "./knowledge_db"
}
},
"chunker": {
"name": "semantic",
"threshold": 0.7
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small"
}
},
"reranker": {
"enabled": True
}
}
# Create research assistant
research_agent = Agent(
name="Research Assistant",
instructions="""You are an expert research assistant.
Use the knowledge base to provide accurate, well-sourced answers.
Always cite the specific documents you reference.""",
knowledge={
"sources": ["research_papers"],
**knowledge_config
}
)
# Use the assistant
response = research_agent.chat(
"What are the main approaches to AI alignment?"
)
# Direct knowledge queries
kb = research_agent.knowledge_instance
papers = kb.search("alignment techniques", limit=5)
for paper in papers:
print(f"- {paper['text'][:100]}...")
print(f" Source: {paper.get('metadata', {}).get('source')}")
```
## Next Steps
Learn about memory integration
Build RAG applications