Skip to main content
The Context Budgeter allocates token budgets across context segments based on model limits and configurable priorities.
The user configures segment priorities; the budgeter allocates tokens before each model call.

Quick Start

1

Configure via Agent

2

Use the budgeter directly

How the context window is resolved

The model’s context window and output reserve resolve in three steps — litellm first, then the static table, then a safe default.
Any model in litellm’s model_cost registry — 100+ providers including Mistral, DeepSeek, xAI, Qwen, Bedrock, Azure OpenAI, OpenRouter, and versioned OpenAI/Anthropic/Google ids — resolves to its real window automatically. No config needed.
Provider-prefixed ids (e.g. openai/gpt-4o, anthropic/claude-3-5-sonnet, azure/gpt-4o) fall back to the base model name after the prefix. Lookup is case-insensitive.
When litellm is not installed, the static MODEL_LIMITS / OUTPUT_RESERVES tables below supply the answer — everything still works offline. pip install litellm unlocks the full registry.

Offline fallback table

Budget Allocation

Default segment budgets:

Custom Budgets

Overflow Detection

Threshold-Based Triggers

CLI Configuration

Environment Variables

Serialization

How It Works


Best Practices

Set output_reserve for the model’s reply so retrieval and history do not consume the full window.
Pass the actual model id (including any provider prefix like openai/ or azure/) — litellm auto-resolution handles the rest, so limits match the provider’s real context window without a manual override.
Trigger compaction or retrieval trimming before hard overflow — do not wait for API errors.
Allocate per-segment budgets when tools return bulky JSON or file contents.

Context Ledger

Track actual token usage by segment

Context Optimizer

Reduce context when over budget