Skip to main content
SDPM (Semantic Double-Pass Merging) combines sentence-level chunking with semantic analysis for optimal chunk boundaries.

Quick Start

from praisonaiagents import Agent

agent = Agent(
    instructions="Answer questions from research papers.",
    knowledge={
        "sources": ["papers/"],
        "chunker": {
            "type": "sdpm",
            "chunk_size": 512,
            "embedding_model": "all-MiniLM-L6-v2"
        }
    }
)

response = agent.start("Summarize the findings")
from praisonaiagents.knowledge.chunking import Chunking

chunker = Chunking(
    chunker_type="sdpm",
    chunk_size=512,
    embedding_model="all-MiniLM-L6-v2"
)

chunks = chunker.chunk("Your research paper content...")

When to Use

  • Research papers with complex structure
  • Technical documents with multiple topics
  • Content where both sentence flow AND semantic coherence matter

Parameters

ParameterTypeDefaultDescription
chunk_sizeint512Max tokens per chunk
embedding_modelstrautoModel for semantic analysis