Token Budgeting

Dynamic token budget management for large context handling

Token budgeting keeps retrieved context within each model’s window so agents never overflow the context limit.

from praisonaiagents import Agent, KnowledgeConfig

agent = Agent(
    name="Researcher",
    instructions="Answer from the knowledge base.",
    knowledge=KnowledgeConfig(sources=["./docs"], retrieval_k=5),
)
agent.start("What are the key features?")

The user asks a knowledge-heavy question; token budgeting trims retrieved chunks so the prompt stays within the model window.

How It Works

Quick Start

1. Agent with Knowledge

   Knowledge retrieval applies token budgets automatically when indexing and retrieving.

   from praisonaiagents import Agent, KnowledgeConfig
   
   agent = Agent(
       name="BudgetAwareAgent",
       instructions="Answer questions using the knowledge base.",
       knowledge=KnowledgeConfig(sources=["./docs"], retrieval_k=5),
   )
   agent.start("What are the key features?")

2. Long-Context Model (gpt-5)

   Pass a model name and the window resolves automatically — gpt-5 unlocks the full 1M context.

   from praisonaiagents import Agent
   from praisonaiagents.rag import TokenBudget
   
   # Simplest agent-centric usage — model resolves to the 1M window
   agent = Agent(
       name="LongContextAgent",
       instructions="Answer using the full document.",
       llm="gpt-5",
   )
   
   # Under the hood
   budget = TokenBudget(model="gpt-5")
   print(budget.model_context_window)  # 1047576

3. Direct Budget Control

   Use TokenBudget when building custom RAG pipelines.

   from praisonaiagents.rag import TokenBudget, DefaultBudgetEnforcer
   
   budget = TokenBudget(model="gpt-4o-mini")
   available = budget.dynamic_budget(system_tokens=500, history_tokens=1000)
   
   enforcer = DefaultBudgetEnforcer()
   chunks = ["chunk one...", "chunk two...", "chunk three..."]
   enforced = enforcer.enforce(budget, chunks)
   print(f"Kept {len(enforced)} of {len(chunks)} chunks ({available} tokens available)")

Configuration

TokenBudget Options

from praisonaiagents.rag import TokenBudget

budget = TokenBudget(
    model="gpt-4o",
    reserved_response=4096,
    reserved_system=1000,
    reserved_history=2000,
)
print(budget.model_context_window)  # 128000
print(budget.max_context_tokens)

Model Context Windows

Budget Enforcement

from praisonaiagents.rag import TokenBudget, DefaultBudgetEnforcer

budget = TokenBudget(model="gpt-4o-mini")
enforcer = DefaultBudgetEnforcer()

chunks = ["chunk1 content...", "chunk2 content...", "chunk3 content..."]
enforced_chunks = enforcer.enforce(budget, chunks)

Implement BudgetEnforcerProtocol for priority-based or custom selection strategies.

CLI Usage

praisonai knowledge stats --json
praisonai knowledge index ./docs --verbose

Best Practices

Related