Quick Start
1
Configure via Agent
2
Use the budgeter directly
How the context window is resolved
The model’s context window and output reserve resolve in three steps — litellm first, then the static table, then a safe default.Any model in litellm’s
model_cost registry — 100+ providers including Mistral, DeepSeek, xAI, Qwen, Bedrock, Azure OpenAI, OpenRouter, and versioned OpenAI/Anthropic/Google ids — resolves to its real window automatically. No config needed.Provider-prefixed ids (e.g.
openai/gpt-4o, anthropic/claude-3-5-sonnet, azure/gpt-4o) fall back to the base model name after the prefix. Lookup is case-insensitive.Offline fallback table
Budget Allocation
Default segment budgets:Custom Budgets
Overflow Detection
Threshold-Based Triggers
CLI Configuration
Environment Variables
Serialization
How It Works
Best Practices
Reserve output tokens explicitly
Reserve output tokens explicitly
Set
output_reserve for the model’s reply so retrieval and history do not consume the full window.Align budget with your model limit
Align budget with your model limit
Pass the actual model id (including any provider prefix like
openai/ or azure/) — litellm auto-resolution handles the rest, so limits match the provider’s real context window without a manual override.Monitor utilisation above 80%
Monitor utilisation above 80%
Trigger compaction or retrieval trimming before hard overflow — do not wait for API errors.
Segment large tool results
Segment large tool results
Allocate per-segment budgets when tools return bulky JSON or file contents.
Related
Context Ledger
Track actual token usage by segment
Context Optimizer
Reduce context when over budget

