Praison AI

Get Started

Gateway Idle Compaction

Compact idle gateway sessions off the critical path so returning users pay no first-message latency

Idle-session compaction runs in the background so a returning user resumes an already-compacted session with no latency spike, and idle transcripts stay bounded.

Diagram

Gateway Idle Compaction

💤 Idle Session
large transcript

🔍 Sweep
every N seconds

📊 Compact
SUMMARIZE off-path

🔖 Checkpoint
persisted

⚡ Next user turn
no latency spike

Quick Start

  1. Enable in gateway.yaml (minimal)
    yaml
    # gateway.yaml
lifecycle:
  idle_compaction:
    enabled: true
    bash
    praisonai gateway start --config gateway.yaml

    This uses the defaults (idle ≥ 30 min, ≥ 8000 tokens, sweep every 5 min). Requires a durable session store — the shipped default is durable, so no extra flags are needed.

  2. Tune for a busier gateway
    yaml
    # gateway.yaml
lifecycle:
  idle_compaction:
    enabled: true
    idle_after_seconds: 900      # 15 min
    min_tokens: 4000              # compact smaller transcripts
    sweep_interval_seconds: 120   # sweep every 2 min
    max_per_sweep: 50             # more work per pass
    bash
    praisonai gateway start --config gateway.yaml
  3. Verify it fires

    Grep the gateway logs for the arm line and per-session shrink lines:

    text
    Gateway idle_compaction enabled (idle_after=1800.0s, min_tokens=8000, sweep=300.0s)
Gateway idle-compaction sweep armed
[idle-compaction] user-42-chat: 12400->3100 tokens

How It Works

Each sweep enumerates persisted sessions off-thread, filters idle over-budget candidates, summarises them off the turn critical path, and persists a checkpoint the next user message resumes from.

Diagram
ContextCompactorCandidate FilterSession StoreSweep TickContextCompactorCandidate FilterSession StoreSweep Ticklist_sessions(limit) (off-thread)session listingidle + size + live-turn + cooldowncandidatesget_working_history(sid) (off-thread)summary + tailcompact_async (SUMMARIZE, off-path)summary + tokensdid the session grow during compaction?noappend_compaction_checkpoint (off-thread)
GuardBehaviour
Idle thresholdOnly sessions with updated_at ≥ idle_after_seconds old are candidates.
Size thresholdOnly sessions whose total_tokens ≥ min_tokens (unknown size is included — belt-and-braces).
Live-turn guardSessions currently _is_executing in this gateway are skipped.
Per-session cooldownAfter a sweep (whether the session shrank or not), the session is off-limits for cooldown_seconds.
Stale-checkpoint race guardIf the transcript grew between selection and summary completion, the checkpoint is skipped and the session is retried later.
DormancyThe sweep pauses when the gateway is _is_dormant (scale-to-zero quiesced).
Off-path I/Olist_sessions, get_working_history, and append_compaction_checkpoint all run via asyncio.to_thread so websocket/turn handling is never blocked.

The sweep reuses the same Context Compaction engine, constructed as ContextCompactor(max_tokens=min_tokens, target_tokens=int(min_tokens * 0.75), strategy=CompactionStrategy.SUMMARIZE).

Configuration Options

Every knob lives under lifecycle.idle_compaction: in gateway.yaml.

FieldTypeDefaultDescription
enabledboolfalseTurns the sweep on. The block is otherwise ignored. Requires a durable session store; otherwise disables itself with a warning.
idle_after_secondsfloat1800.0Minimum seconds since the session’s last updated_at before it becomes a compaction candidate. Must be > 0.
min_tokensint8000Minimum transcript size (in tokens, if the store reports total_tokens) before the session is considered for compaction. Must be > 0.
sweep_interval_secondsfloat300.0Seconds between sweeps. Must be > 0 (0/negative disables the feature — a zero interval would busy-spin).
cooldown_secondsfloat3600.0Per-session backoff after a sweep touches it (whether it shrank or not), so a session that cannot be shrunk isn’t retried every sweep. Must be > 0.
max_per_sweepint20Maximum sessions to compact per sweep pass. Bounds the work a single tick can queue. Must be > 0.
scan_limitint5000How many sessions the sweep enumerates from the store per pass. Larger caps prevent older idle sessions from being starved on busy gateways (stores return newest-first). Must be > 0.

Requirements

Durable session store

The feature needs list_sessions + get_working_history + append_compaction_checkpoint. The shipped SQLite transcript store (session.store: "sqlite", default) implements all three. The legacy "file" backend may not — see Gateway Session Persistence and SQLite Transcript Store. If no durable store is bound, the gateway logs Gateway idle_compaction requires a persistent session store; disabling.

Persistent sessions

Ephemeral (session.persist: false) gateways cannot use idle_compaction — there is nothing to enumerate.

Hot Reload

Enabling or disabling idle_compaction via a gateway.yaml reload takes effect without a full process restart — _reconcile_lifecycle cancels the old task and starts the new one. See Gateway Hot Reload.

Which Compaction Surface?

Four related-but-different compaction surfaces exist; this decision diagram shows which fits which scenario.

Diagram

What do you need
to compact?

Compact during
a single long run

Cheap –continue /
resume for one session

Sweep many idle gateway
sessions in background

Compact a bot
session on save

Context Compaction
execution=ExecutionConfig
context_compaction=True

Compacted Session Resume

This page
lifecycle.idle_compaction

Bot Session Compaction

Common Patterns

24/7 gateway with many long-lived users

yaml
# gateway.yaml
lifecycle:
  idle_compaction:
    enabled: true
    idle_after_seconds: 1800
    min_tokens: 8000
    sweep_interval_seconds: 300

Compact 30-minute-idle chats over 8k tokens every 5 minutes — cost drops and returning users pay no first-message latency.

Aggressive shrink for tight-context models

yaml
# gateway.yaml
lifecycle:
  idle_compaction:
    enabled: true
    min_tokens: 4000
    sweep_interval_seconds: 60

Shrink smaller transcripts more often to keep sessions under a tight model context window.

Very busy gateway (many idle sessions)

yaml
# gateway.yaml
lifecycle:
  idle_compaction:
    enabled: true
    scan_limit: 20000
    max_per_sweep: 50

Bump scan_limit and max_per_sweep so older idle sessions aren’t starved — stores return newest-first.

Best Practices

Only durable stores support this

Without persist: true and a store implementing get_working_history / append_compaction_checkpoint, the sweep disables itself. Verify with praisonai gateway start logs — a healthy start prints Gateway idle-compaction sweep armed.

Keep sweep_interval_seconds ≥ 60

Sub-minute sweeps rarely pay off on real workloads and can pressure the store. Start at the default 300 and lower only if you observe transcripts growing between sweeps.

Cool down long enough to avoid re-work

Default cooldown_seconds: 3600 matches the default idle threshold. If you lower idle_after_seconds, lower cooldown_seconds proportionally so cooled sessions become candidates again on a sensible cadence.

Watch for transcript grew during compaction debug logs

This is normal on active sessions — the sweep skips the checkpoint and retries. Persistent recurrence means your idle_after_seconds is too low for the traffic pattern; increase it so live sessions aren’t candidates.

Related

Scale to Zero

Sibling lifecycle policy — idle quiesce for serverless hosts.

 Drain Trigger

Sibling lifecycle policy — epoch-safe external drain marker.

 Compacted Session Resume

How the checkpoint this feature writes is consumed on the next user turn.

 Context Compaction

The in-run compactor this feature reuses.

 Session Persistence

The durable session store this feature requires.

 Gateway Overview

How lifecycle policies wire into WebSocketGateway.