LLM Context Compression

Intelligent LLM-driven compression of conversation history with session lineage and head/tail protection

LLM Context Compression intelligently summarises long conversation history while preserving the system prompt and recent context, with session lineage for traceability.

from praisonaiagents import Agent, ManagerConfig

agent = Agent(
    name="Researcher",
    instructions="Research topics in depth across many turns.",
    context=ManagerConfig(
        auto_compact=True,
        compact_threshold=0.8,
        strategy="summarize",
        llm_summarize=True,
    ),
)
agent.start("Continue our multi-turn research on vector databases.")

The user keeps chatting; when context fills up, the agent compacts middle turns while keeping head and tail intact.

Quick Start

1. Simplest — enable via Agent

   from praisonaiagents import Agent, ManagerConfig
   
   agent = Agent(
       name="Researcher",
       instructions="Research topics in depth across many turns.",
       context=ManagerConfig(
           auto_compact=True,
           compact_threshold=0.8,
           strategy="summarize",  # uses LLM compression when available
           llm_summarize=True,
       ),
   )
   
   agent.start("Walk me through the entire history of AI safety research...")

2. Full control with LLMContextCompressorOptimizer

   from praisonaiagents.context.optimizer import LLMContextCompressorOptimizer
   
   optimizer = LLMContextCompressorOptimizer(
       llm_client=agent.llm,                # reuse the agent's LLM
       auxiliary_model="gpt-4o-mini",       # cheaper model for summarization
       protect_last_n_tokens=20_000,
       summary_target_tokens=750,
       enable_session_tracking=True,
   )
   
   optimized_messages, result = optimizer.optimize(messages, target_tokens=8_000)
   print(f"Saved {result.tokens_saved} tokens — strategy: {result.strategy_used}")

How It Works

Configuration Options

Session Lineage

Track compression history and audit trails across repeated compactions:

# Access session history
compressor = ContextCompressor(llm=agent.llm)
result = await compressor.compress(messages)

# View compression sessions
for session in compressor.get_session_history():
    print(f"Session {session.session_id[:8]}: {session.original_tokens} → {session.compressed_tokens}")

# Chain sessions across compactions
next_result = await compressor.compress(
    result.messages,
    parent_session_id=result.session_id  # maintain audit trail
)

CompressionSession shape:

• session_id: Unique identifier for this compression
• parent_session_id: ID of previous compression for lineage
• created_at: Timestamp
• original_message_count / compressed_message_count: Message counts
• original_tokens / compressed_tokens: Token counts
• summary_text: The LLM-generated summary content

CompressResult

Common Patterns

Use a cheap auxiliary model:

optimizer = LLMContextCompressorOptimizer(
    llm_client=agent.llm,           # Main agent uses gpt-4o
    auxiliary_model="gpt-4o-mini",  # Compression uses cheaper model
)

When auxiliary_model is omitted, the optimizer resolves it via PRAISONAI_AUXILIARY_MODEL → OPENAI_MODEL_NAME → "gpt-4o-mini" (see Auxiliary model resolution). This is what lets a fully-local setup point compaction at a small model without editing a config file.

Tune for tool-heavy loops:

# Preserve more recent context when tools are frequently used
optimizer = LLMContextCompressorOptimizer(
    protect_last_n_tokens=30_000,  # Keep more recent tool results
    summary_target_tokens=1000,    # Slightly longer summaries
)

Chain sessions across compactions:

result1 = await compressor.compress(messages)
result2 = await compressor.compress(
    result1.messages,
    parent_session_id=result1.session_id  # maintain lineage
)

Best Practices

Related