> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Recursive Chunking

> Hierarchical document splitting using customizable rules

Splits text hierarchically using multiple separators (paragraphs → sentences → words). Ideal for structured documents like markdown.

## Quick Start

<CodeGroup>
  ```python Agent with Recursive Chunking theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
  from praisonaiagents import Agent

  agent = Agent(
      instructions="Answer questions from documentation.",
      knowledge={
          "sources": ["docs/"],
          "chunker": {
              "type": "recursive",
              "chunk_size": 512
          }
      }
  )

  response = agent.start("How do I configure the settings?")
  ```

  ```python Direct API theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
  from praisonaiagents.knowledge.chunking import Chunking

  chunker = Chunking(
      chunker_type="recursive",
      chunk_size=512
  )

  markdown_doc = """
  # Introduction
  This is the intro paragraph.

  ## Section 1
  Content for section 1.

  ## Section 2  
  Content for section 2.
  """

  chunks = chunker.chunk(markdown_doc)
  for chunk in chunks:
      print(chunk.text)
  ```
</CodeGroup>

## When to Use

<CardGroup cols={2}>
  <Card title="Good For" icon="check">
    * Markdown documentation
    * Technical manuals
    * Structured content
    * Code with comments
  </Card>

  <Card title="Consider Alternatives" icon="xmark">
    * Unstructured prose
    * Stream of consciousness
    * Very short documents
    * Topic-based splitting needed
  </Card>
</CardGroup>

## Parameters

| Parameter                    | Type | Default  | Description            |
| ---------------------------- | ---- | -------- | ---------------------- |
| `chunk_size`                 | int  | 512      | Max tokens per chunk   |
| `tokenizer_or_token_counter` | str  | `"gpt2"` | Tokenizer for counting |

## Examples

### Documentation

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
agent = Agent(
    instructions="Help users find answers in docs.",
    knowledge={
        "sources": ["README.md", "docs/"],
        "chunker": {
            "type": "recursive",
            "chunk_size": 512
        }
    }
)
```

### Large Codebase

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
agent = Agent(
    instructions="Explain code and architecture.",
    knowledge={
        "sources": ["src/"],
        "chunker": {
            "type": "recursive",
            "chunk_size": 1024
        }
    }
)
```

## How It Works

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
flowchart TD
    A[Document] --> B{Size > Limit?}
    B -->|Yes| C[Split by \\n\\n]
    C --> D{Size > Limit?}
    D -->|Yes| E[Split by \\n]
    E --> F{Size > Limit?}
    F -->|Yes| G[Split by sentence]
    G --> H{Size > Limit?}
    H -->|Yes| I[Split by word]
    
    B -->|No| J[Keep as chunk]
    D -->|No| J
    F -->|No| J
    H -->|No| J
    I --> J
    
    style A fill:#8B0000,color:#fff
    style J fill:#2E8B57,color:#fff
```

The recursive approach tries larger separators first (paragraphs), then falls back to smaller ones (sentences, words) only when needed.

## Best Practices

1. **Match chunk size to content density** - Dense technical docs need smaller chunks
2. **Use with markdown** - Recursive chunking respects markdown structure well
3. **Combine with semantic search** - The hierarchical splits provide logical boundaries for retrieval