SDPM Chunking

Sentence-level chunking with semantic double-pass merging

SDPM (Semantic Double-Pass Merging) combines sentence-level chunking with semantic analysis for optimal chunk boundaries.

Quick Start

Agent with SDPM Chunking

from praisonaiagents import Agent

agent = Agent(
    instructions="Answer questions from research papers.",
    knowledge={
        "sources": ["papers/"],
        "chunker": {
            "type": "sdpm",
            "chunk_size": 512,
            "embedding_model": "all-MiniLM-L6-v2"
        }
    }
)

response = agent.start("Summarize the findings")

Direct API

from praisonaiagents.knowledge.chunking import Chunking

chunker = Chunking(
    chunker_type="sdpm",
    chunk_size=512,
    embedding_model="all-MiniLM-L6-v2"
)

chunks = chunker.chunk("Your research paper content...")

When to Use

Research papers with complex structure
Technical documents with multiple topics
Content where both sentence flow AND semantic coherence matter

Parameters

Parameter	Type	Default	Description
`chunk_size`	int	512	Max tokens per chunk
`embedding_model`	str	auto	Model for semantic analysis