Gateway Restart Continuation

Re-drive interrupted turns and proactively notify the originating channel when the gateway restarts

On boot, the gateway resumes turns that were in-flight when the previous process stopped — and proactively tells channel users what happened, so a Telegram or Slack user never sits in silence after a restart.

Quick Start

1. Start with an agent

   from praisonaiagents import Agent
   
   agent = Agent(name="assistant", instructions="Be helpful.")
   # Restart continuation activates automatically when session persistence is on.
   # No extra flags — see the gateway.yaml step below.

2. Enable session persistence (the only opt-in)

   # gateway.yaml
   gateway:
     session:
       persist: true

   praisonai gateway start --config gateway.yaml

3. What a channel user sees

   After a restart interrupts their request, the originating channel receives:

   I was interrupted by a restart - resuming your request. If you don't get a reply shortly, please resend.

4. What operators see

   The boot scan logs a one-line summary:

   INFO  Resumed 1 interrupted turn(s) on boot

How It Works

On boot — before serving new traffic — the gateway scans persisted sessions, notifies the originating channel of any interrupted turn, and re-drives it when the agent is still registered.

The whole scan is wrapped in a try / except at the start() site — a failure in the resume loop never blocks the gateway from serving new traffic.

Run ledger recovery

Alongside interrupted-turn resumption, boot also reconciles the durable Run Ledger. For background runs (long-running agent jobs kicked off from a channel), the gateway calls recover_orphans() right after the session scan, marking each still-active run LOST and posting a durable, exactly-once restart notice back to that thread’s origin.

The two mechanisms are complementary: the session scan re-drives turns that were mid-flight; the ledger recovery terminalises background runs that a crash left running and tells their origin.

What Triggers a Resume

A persisted session is resumed when all of these hold:

• is_executing == True or pending_inbox is non-empty (a turn was in flight).
• channel_target is set on the persisted snapshot — channel-originated only; direct-client sessions resume via reconnect.
• A durable session store is configured (the scan is a no-op without one).

The Notice Contract

The originating channel receives one verbatim message per interruption:

I was interrupted by a restart - resuming your request. If you don't get a reply shortly, please resend.

Fallback when the delivery router is unavailable. If WebSocketGateway.delivery_router is None, the notice falls back to bot.send_message(target, text) via get_channel_bot(channel) — best-effort, no idempotency. If you disable the delivery router, you also opt out of exactly-once: an ill-configured deployment can double-notify on a crash loop.

channel_target on Persisted Sessions

The persisted snapshot gains one field (default None):

{
  "session_id": "...",
  "messages": [],
  "events": [],
  "event_cursor": 42,
  "pending_inbox": ["..."],
  "is_executing": true,
  "channel_target": "telegram:12345"
}

Best-Effort Re-drive

Whether the turn actually resumes depends on the agent still being registered on the fresh process.

• Agent still registered (same agent_id in the config) → the session is rehydrated and _run_session_queue restarts, so a real reply follows the notice.
• Agent no longer registered (removed from the config) → the notice above still fires, and its wording already asks the user to resend. No exception, no crash-loop.

Configuration Options

No new user-facing config flags. The feature is enabled transitively by the presence of:

No operator action is required to opt in beyond enabling session persistence — the feature is on by default when persistence is on.

# gateway.yaml
gateway:
  session:
    persist: true

Operator Observability

Search a log stream for these lines to find this behaviour:

The boot scan calls list_sessions() with an explicit large limit, so interrupted turns beyond the store’s default 50-session pagination window are not silently skipped.

Common Patterns

Two deployment scenarios where restart continuation changes behaviour.

Rolling restarts. Restart continuation makes rolling deploys safer for channel users, provided session persistence is on and the delivery router is shared across instances.

Deliberate maintenance drop. To swallow all in-flight work on a restart (for example, a schema migration invalidates the persisted format), turn off gateway.session.persist for that boot — the scan is a no-op without a store.

Best Practices

Related