Memory Advanced Search

Learn how to use advanced search parameters for memory retrieval including reranking and relevance filtering.

Memory search in PraisonAI Agents provides advanced parameters for better control over search results, including reranking for improved relevance and cutoff thresholds for quality control.

Quick Start

Agent with Advanced Search

from praisonaiagents import Agent

# Agent with memory for advanced search
agent = Agent(
    instructions="Search and retrieve with quality filtering.",
    memory={
        "backend": "rag",
        "use_embedding": True
    }
)

# Chat stores to memory automatically
agent.start("Paris is the capital of France")
agent.start("Tokyo is the capital of Japan")

# Subsequent queries use memory with relevance filtering
response = agent.start("What is the capital of France?")

Direct Memory API

from praisonaiagents import Memory

# Initialize memory with ChromaDB (local storage)
memory = Memory(config={
    "provider": "rag",
    "use_embedding": True,
    "rag_db_path": ".praison/memory_db"
})

# Store information
memory.store_long_term("Paris is the capital of France")

# Search with relevance cutoff
results = memory.search_long_term(
    "What is the capital of France?",
    relevance_cutoff=0.7,
    limit=5
)

How SQLite recall matches queries

The SQLite memory adapter matches on the significant words of your query as well as the whole phrase, so natural-language questions recall the right rows.

Diagram

python

from praisonaiagents.memory.adapters.sqlite_adapter import SqliteMemoryAdapter

adapter = SqliteMemoryAdapter(short_db="s.db", long_db="l.db")
adapter.store_long_term("The user prefers dark mode and works in Python.")

# All four queries recall the same row:
adapter.search_long_term("dark mode")
adapter.search_long_term("user prefers Python")
adapter.search_long_term("Which mode does the user prefer?")
adapter.search_long_term("What language does the user work in?")

Stop-words such as a, an, and, the, is, of, to, what, which, who, why, how, where, when, with, your, you, my, me and i are dropped. At most 32 terms are used, to stay well under SQLite’s bound-parameter limit. The user_id filter is unchanged, and so is the schema; no new dependency is needed. The same matching applies to search_short_term.

Advanced Search Parameters

The search_long_term method supports several advanced parameters:

Method Signature

python

def search_long_term(
    self, 
    query: str, 
    limit: int = 5, 
    relevance_cutoff: float = 0.0,
    min_quality: float = 0.0,
    rerank: bool = False,
    **kwargs
) -> List[Dict[str, Any]]:

Parameter Details

query string

The search query to find relevant memories

limit integer

Maximum number of results to return

relevance_cutoff float

Minimum relevance score (0.0 to 1.0) for results to be included

min_quality float

Minimum quality score for results (used with quality tracking)

rerank boolean

Enable reranking for improved relevance (only works with Mem0 provider)

Provider-Specific Features

ChromaDB (Local Storage)

ChromaDB is the default local storage provider that supports relevance filtering:

python

from praisonaiagents import Memory

# Initialize ChromaDB memory
memory = Memory(config={
    "provider": "rag",
    "use_embedding": True,
    "rag_db_path": ".praison/memory_db"
})

# Store memories with metadata
memory.store_long_term(
    "The Eiffel Tower is 330 meters tall",
    metadata={"category": "landmarks", "city": "Paris"}
)

memory.store_long_term(
    "The Statue of Liberty is 93 meters tall",
    metadata={"category": "landmarks", "city": "New York"}
)

# Search with relevance cutoff
results = memory.search_long_term(
    "How tall is the Eiffel Tower?",
    relevance_cutoff=0.6,  # Filter out low-relevance results
    limit=10
)

# ChromaDB calculates score as: 1.0 - distance
# Higher scores mean better relevance

Mem0 (Cloud Provider)

Mem0 is a cloud-based provider that supports both relevance filtering and reranking:

python

from praisonaiagents import Memory

# Initialize Mem0 memory
mem0_memory = Memory(config={
    "provider": "mem0",
    "config": {
        "api_key": "your-mem0-api-key",
        "org_id": "your-org-id",  # Optional
        "project_id": "your-project-id"  # Optional
    }
})

# Search with reranking enabled
results = mem0_memory.search(
    query="What are the key features of our product?",
    agent_id="agent-123",  # Required for Mem0
    rerank=True,          # Enable reranking for better results
    limit=5
)

# Reranking adds 150-200ms latency but improves result quality

Relevance Scoring

How Relevance Scores Work

ChromaDB Scoring

Uses vector similarity (cosine distance)
Score = 1.0 – distance
Range: 0.0 to 1.0
Higher scores = better matches

Mem0 Scoring

Uses proprietary scoring algorithm
Includes semantic understanding
Reranking uses additional context
Optimized for accuracy

Setting Appropriate Cutoffs

python

# Conservative cutoff - only very relevant results
high_quality_results = memory.search_long_term(
    "important company policies",
    relevance_cutoff=0.8
)

# Moderate cutoff - balanced results
balanced_results = memory.search_long_term(
    "product features",
    relevance_cutoff=0.6
)

# Low cutoff - more inclusive results
inclusive_results = memory.search_long_term(
    "general information",
    relevance_cutoff=0.3
)

Complete Examples

Example 1: Knowledge Base Search

python

from praisonaiagents import Memory

# Create a knowledge base
knowledge_memory = Memory(config={
    "provider": "rag",
    "use_embedding": True
})

# Store various facts
facts = [
    "Python was created by Guido van Rossum in 1991",
    "JavaScript was created by Brendan Eich in 1995",
    "Java was created by James Gosling in 1995",
    "C++ was created by Bjarne Stroustrup in 1985",
    "Ruby was created by Yukihiro Matsumoto in 1995"
]

for fact in facts:
    knowledge_memory.store_long_term(fact)

# Search with different relevance thresholds
query = "Who created Python?"

# High relevance - only direct matches
strict_results = knowledge_memory.search_long_term(
    query,
    relevance_cutoff=0.8,
    limit=3
)
print(f"Strict search found {len(strict_results)} results")

# Medium relevance - related programming languages
related_results = knowledge_memory.search_long_term(
    query,
    relevance_cutoff=0.5,
    limit=5
)
print(f"Related search found {len(related_results)} results")

Example 2: Agent Memory with Quality Tracking

python

from praisonaiagents import Agent, Memory, Task, AgentTeam

# Create agent with memory
agent = Agent(
    name="Research Assistant",
    role="Information specialist",
    goal="Provide accurate information from memory",
    backstory="An AI with perfect recall and organization skills"
)

# Create memory instance
memory = Memory(config={
    "provider": "rag",
    "use_embedding": True
})

# Task that stores high-quality information
def research_and_store(topic: str):
    # Simulate research with quality score
    research_data = f"Comprehensive research on {topic}"
    quality_score = 0.85  # High quality

    # Store with quality metadata
    memory.store_long_term(
        research_data,
        metadata={
            "topic": topic,
            "quality_score": quality_score,
            "agent_id": agent.id
        }
    )
    return f"Stored research on {topic}"

# Search with quality filtering
def search_quality_info(query: str):
    results = memory.search_long_term(
        query,
        relevance_cutoff=0.6,
        min_quality=0.8,  # Only high-quality results
        limit=3
    )
    return results

# Create tasks
store_task = Task(
    description="Research and store information about artificial intelligence",
    expected_output="Confirmation of stored research",
    agent=agent,
    execute_function=lambda: research_and_store("artificial intelligence")
)

search_task = Task(
    description="Find high-quality information about AI",
    expected_output="Top quality search results",
    agent=agent,
    execute_function=lambda: search_quality_info("artificial intelligence")
)

# Run workflow
workflow = AgentTeam(
    agents=[agent],
    tasks=[store_task, search_task],
    process="sequential"
)

results = workflow.start()

Example 3: Multi-Provider Setup

python

from praisonaiagents import Memory
import os

# Setup both providers
local_memory = Memory(config={
    "provider": "rag",
    "use_embedding": True
})

cloud_memory = Memory(config={
    "provider": "mem0",
    "config": {
        "api_key": os.getenv("MEM0_API_KEY")
    }
})

# Function to search both providers
def search_all_memory(query: str, use_rerank: bool = True):
    # Search local memory with relevance cutoff
    local_results = local_memory.search_long_term(
        query,
        relevance_cutoff=0.6,
        limit=5
    )

    # Search cloud memory with reranking
    cloud_results = cloud_memory.search(
        query=query,
        agent_id="global",
        rerank=use_rerank,  # Only works with Mem0
        limit=5
    )

    # Combine and deduplicate results
    all_results = []
    seen_content = set()

    for result in local_results + cloud_results:
        content = result.get('memory', '')
        if content not in seen_content:
            seen_content.add(content)
            all_results.append(result)

    # Sort by relevance score
    all_results.sort(
        key=lambda x: x.get('score', 0), 
        reverse=True
    )

    return all_results[:10]  # Top 10 results

# Use the multi-provider search
results = search_all_memory(
    "What are the main features of our product?",
    use_rerank=True
)

for i, result in enumerate(results, 1):
    print(f"{i}. {result['memory']}")
    print(f"   Score: {result.get('score', 'N/A')}")
    print(f"   Provider: {result.get('provider', 'unknown')}")

Performance Considerations

Reranking Impact

Adds 150-200ms latency
Improves result quality by 20-30%
Best for critical searches
Not suitable for real-time applications

Relevance Cutoff

No performance impact
Reduces result set size
Improves signal-to-noise ratio
Can filter out useful edge cases

Best Practices

Choose the Right Provider
Use ChromaDB for local, fast searches
Use Mem0 for cloud-based with reranking needs

Set Appropriate Cutoffs
Start with 0.6-0.7 for general searches
Use 0.8+ for precise matching
Use 0.3-0.5 for exploratory searches

Optimize for Your Use Case

python

# Fast, local search for UI
quick_results = memory.search_long_term(
    query,
    relevance_cutoff=0.7,
    limit=3
)

# Comprehensive search for analysis
detailed_results = cloud_memory.search(
    query=query,
    agent_id=agent_id,
    rerank=True,
    limit=20
)

Troubleshooting

Reranking not working

Verify you’re using Mem0 provider
Check API key is valid
Ensure agent_id is provided
Monitor API quota limits

No results returned

Lower relevance_cutoff threshold
Check if memories exist
Verify embedding model is working
Try broader search terms

Poor relevance scores

Ensure quality embeddings
Store more context with memories
Use more specific queries
Consider reranking (Mem0 only)

Next Steps

Memory Management

Learn about memory storage and retrieval basics

Knowledge Base

Explore knowledge management features