Durable Outbound Delivery

Retry-with-backoff and dead-letter queue for all bot channels so transient errors never drop agent replies

from praisonaiagents import Agent

agent = Agent(name="support", instructions="Reply helpfully on Slack or Telegram.")
agent.start("Tell the user their ticket was updated.")

Every bot reply now survives transient channel errors — 5xx responses, 429 rate limits, and network blips are automatically retried with bounded exponential backoff before parking in a Dead Letter Queue on permanent failure.

The user sends a bot reply; transient channel errors retry with backoff before a permanent failure lands in the dead-letter queue.

Quick Start

1. Zero Configuration (Retry + Durable Park)

   Every adapter now retries transient failures and parks permanent / exhausted failures in a per-platform DLQ by default — no config needed:

   from praisonaiagents import Agent
   from praisonai.bots import TelegramBot
   
   agent = Agent(
       name="Support Agent",
       instructions="Answer user questions helpfully.",
   )
   
   bot = TelegramBot(token="YOUR_TOKEN", agent=agent)
   await bot.start()
   # Transient errors retry with backoff; permanent / exhausted failures are
   # parked at ~/.praisonai/state/telegram/outbound_dlq.sqlite automatically.

   Slack, Discord, WhatsApp, Email, Linear, and AgentMail behave identically — each writes to its own ~/.praisonai/state/<platform>/outbound_dlq.sqlite file.

2. Override the default DLQ path or tune retries

   dlq_path is optional — set it only if you need the DLQ on a different filesystem (e.g. a shared volume, a specific bind mount). If you just want to tune retries, all the other keys work the same as before.

   from praisonaiagents import Agent
   from praisonai.bots import SlackBot, BotConfig, OutboundResilienceConfig
   
   config = BotConfig(
       outbound_resilience=OutboundResilienceConfig(
           # dlq_path is optional; default is ~/.praisonai/state/slack/outbound_dlq.sqlite
           dlq_path="/mnt/state/slack-outbound-dlq.sqlite",
           max_attempts=5,
           initial_ms=1000,
           max_ms=30000,
       )
   )
   
   agent = Agent(name="Slack Agent", instructions="Help the team.")
   bot = SlackBot(token="YOUR_TOKEN", agent=agent, config=config)
   await bot.start()

   To disable durable parking entirely for one channel:

   config = BotConfig(
       outbound_resilience=OutboundResilienceConfig(enabled=False)
   )

3. Tune Backoff Parameters

   Fine-tune retry behaviour per channel:

   from praisonai.bots import BotConfig, OutboundResilienceConfig
   
   config = BotConfig(
       outbound_resilience=OutboundResilienceConfig(
           initial_ms=500,       # first retry after 500ms
           max_ms=15000,         # cap at 15s
           factor=2.0,           # double each attempt
           max_attempts=4,       # give up after 4 tries
           jitter=0.3,           # add 30% random jitter
           dlq_path="~/.praisonai/state/dlq.db",
       )
   )

How It Works

The mixin (OutboundResilienceMixin) wraps each adapter’s raw send with deliver_outbound(). State is initialised lazily from self.config.outbound_resilience. When no dlq_path is configured, the mixin falls back to resolve_durable_store_dir(<platform>) / "outbound_dlq.sqlite" so the default is durable-by-default without touching existing adapter constructors.

Channel Support

All six channels now share the same durable delivery path:

What Gets Retried

Structured error classification

Each failure is tagged with a machine-readable SendErrorKind before the retry decision. Permanent kinds (forbidden, target_not_found, auth_fatal, invalid_request) short-circuit immediately rather than exhausting max_retries against a dead target; transient and rate-limited failures still retry with backoff.

See Send Error Taxonomy for the full kind reference and how to classify your own adapter’s native exceptions.

Automatic crash-recovery on start

Every gateway-managed channel redelivers any reply parked by a crash on a previous run — no operator step, symmetric with the inbound journal’s replay() sweep.

The gateway schedules WebSocketGateway._replay_outbound_dlq(bot) about 5 seconds after each channel task starts (a grace period for the transport to connect), then calls OutboundDLQ.redeliver(send) — draining parked replies oldest-first through the adapter’s own send_message, so the platform’s normal retry/backoff still applies. A redelivery that fails again is kept for the next boot, bounded by the DLQ’s existing TTL / max_size / attempts invariants. Failure never blocks channel start.

Users receive redelivered replies with a visible duplicate marker:

♻️ Recovered after restart — this reply may be a duplicate. <original reply text>

When something is recovered you’ll see this line in the gateway log:

Outbound crash-recovery: redelivered N parked repl(y/ies) on start (marked as possible duplicates)

When Held Replies Drain

The durable outbox re-attempts held replies on three triggers:

1. Adapter startup — replays anything queued before the last crash.
2. Channel recovery (Issue #4043) — when ChannelSupervisor sees a channel come back after a transient outage, it re-drains the outbox in the background so held replies go out promptly, without waiting for the next inbound turn.
3. Lazy on next inbound turn — the next chat() turn also drains opportunistically.

See Durable Delivery → When the Outbox Drains for the full explanation and behavioural notes.

Configuration Options

The default path is resolved by the shared resolve_durable_store_dir(<platform>) helper (the same store the inbound journal uses). Set the PRAISONAI_HOME environment variable to relocate the whole ~/.praisonai/ root — handy in tests or containers.

Best Practices

Related