Token Budgeting

Token budgeting keeps retrieved context within each model’s window so agents never overflow the context limit.

from praisonaiagents import Agent, KnowledgeConfig

agent = Agent(
    name="Researcher",
    instructions="Answer from the knowledge base.",
    knowledge=KnowledgeConfig(sources=["./docs"], retrieval_k=5),
)
agent.start("What are the key features?")

The user asks a knowledge-heavy question; token budgeting trims retrieved chunks so the prompt stays within the model window.

How It Works

Async enforcement (1.6.88+): ExecutionConfig(max_budget=...) is now enforced on async agent runs (astart()) as well as sync runs. BudgetExceededError is raised on both paths when the cap is exceeded. Gateway bots also honour the budget.

Quick Start

Agent with Knowledge

Knowledge retrieval applies token budgets automatically when indexing and retrieving.

from praisonaiagents import Agent, KnowledgeConfig

agent = Agent(
    name="BudgetAwareAgent",
    instructions="Answer questions using the knowledge base.",
    knowledge=KnowledgeConfig(sources=["./docs"], retrieval_k=5),
)
agent.start("What are the key features?")

Long-Context Model (gpt-5)

Pass a model name and the window resolves automatically — gpt-5 unlocks the full 1M context.

from praisonaiagents import Agent
from praisonaiagents.rag import TokenBudget

# Simplest agent-centric usage — model resolves to the 1M window
agent = Agent(
    name="LongContextAgent",
    instructions="Answer using the full document.",
    llm="gpt-5",
)

# Under the hood
budget = TokenBudget(model="gpt-5")
print(budget.model_context_window)  # 1047576

Direct Budget Control

Use TokenBudget when building custom RAG pipelines.

from praisonaiagents.rag import TokenBudget, DefaultBudgetEnforcer

budget = TokenBudget(model="gpt-4o-mini")
available = budget.dynamic_budget(system_tokens=500, history_tokens=1000)

enforcer = DefaultBudgetEnforcer()
chunks = ["chunk one...", "chunk two...", "chunk three..."]
enforced = enforcer.enforce(budget, chunks)
print(f"Kept {len(enforced)} of {len(chunks)} chunks ({available} tokens available)")

Configuration

TokenBudget Options

Option	Type	Default	Description
`model`	`str`	`"gpt-4o-mini"`	Model name for context window lookup
`reserved_response`	`int`	`4096`	Tokens reserved for the LLM response
`reserved_system`	`int`	`500`	Tokens reserved for system prompt
`reserved_history`	`int`	`1000`	Tokens reserved for chat history

from praisonaiagents.rag import TokenBudget

budget = TokenBudget(
    model="gpt-4o",
    reserved_response=4096,
    reserved_system=1000,
    reserved_history=2000,
)
print(budget.model_context_window)  # 128000
print(budget.max_context_tokens)

Model Context Windows

Model	Context Window
gpt-5, gpt-5-mini, gpt-5-nano	1,047,576
gpt-4.1, gpt-4.1-mini, gpt-4.1-nano	1,047,576
gpt-4o, gpt-4o-mini	128,000
gpt-4-turbo	128,000
gpt-4	8,192
gpt-4-32k	32,768
gpt-3.5-turbo	16,385
o3, o3-mini, o4-mini	200,000
o1	200,000
o1-mini, o1-preview	128,000
claude-3-5-sonnet, claude-3-5-haiku	200,000
claude-3-opus, claude-3-sonnet, claude-3-haiku	200,000
gemini-1.5-pro	2,097,152
gemini-1.5-flash, gemini-2.0-flash	1,048,576

RAG context-window values are sourced from a single canonical MODEL_LIMITS table shared with the rest of the SDK; RAG-only extras (Mistral, Llama, DeepSeek, Cohere, o1 variants, gpt-4-32k) are layered on top. Values here match get_model_context_window() exactly — pass any model name and get the same window everywhere.

As of PraisonAI PR #3796, the shared context-window lookup consults litellm’s model_cost registry first, so any litellm-known model (Mistral, DeepSeek, xAI, Qwen, Bedrock, Azure, OpenRouter, versioned OpenAI/Anthropic/Google ids) resolves to its true window even when it is not in the table below. The static table is the offline fallback. See How the context window is resolved for the full order.

Unknown or versioned model names resolve via partial match (e.g. gpt-4o-2024-05-13 → gpt-4o). The default 128,000 tokens only applies when both litellm and the static partial-match miss.

Budget Enforcement

from praisonaiagents.rag import TokenBudget, DefaultBudgetEnforcer

budget = TokenBudget(model="gpt-4o-mini")
enforcer = DefaultBudgetEnforcer()

chunks = ["chunk1 content...", "chunk2 content...", "chunk3 content..."]
enforced_chunks = enforcer.enforce(budget, chunks)

Implement BudgetEnforcerProtocol for priority-based or custom selection strategies.

CLI Usage

praisonai knowledge stats --json
praisonai knowledge index ./docs --verbose

Best Practices

Reserve adequate response tokens

Leave enough headroom for complete answers — undersized reserved_response truncates outputs mid-sentence.

Account for chat history

Multi-turn conversations consume tokens quickly; increase reserved_history or enable context compaction on long sessions.

Match budget to model

Different models have different context windows — always pass the actual model name so dynamic_budget() calculates correctly.

Monitor with verbose indexing

Run praisonai knowledge index ./docs --verbose to see chunk counts and token estimates before production retrieval.

Thinking Budgets

Extended reasoning token limits

Token Usage Protocol

Track token consumption across runs

Getting Started

Install

Core Concepts

Guides

Features

Models

Databases

Observability

Memory

Telemetry & Analytics

Knowledge

RAG

Persistence

Tools

Other Features

Developers

Configuration

Best Practices

Getting Started (No Code)

How It Works

Quick Start

Configuration

TokenBudget Options

Model Context Windows

Budget Enforcement

CLI Usage

Best Practices

Thinking Budgets

Token Usage Protocol

​How It Works

​Quick Start

​Configuration

​TokenBudget Options

​Model Context Windows

​Budget Enforcement

​CLI Usage

​Best Practices

​Related

Thinking Budgets

Token Usage Protocol

How It Works

Quick Start

Configuration

TokenBudget Options

Model Context Windows

Budget Enforcement

CLI Usage

Best Practices

Related