Outbound Resilience

Retry transient send failures on every bot channel and park permanently-failed replies in a per-platform durable DLQ — safe by default, no config needed.

Outbound Resilience makes sure your agent’s reply actually reaches the user on every channel — retrying transient send errors with bounded backoff, and parking permanently-failed replies in a durable per-platform DLQ by default so they can be replayed instead of silently lost. No configuration is required.

from praisonaiagents import Agent
from praisonai.bots import SlackBot

agent = Agent(
    name="Support Bot",
    instructions="Answer customer questions politely.",
)
bot = SlackBot(token="xoxb-...", agent=agent)
bot.start()

# If the gateway crashes mid-send, the reply is parked in the outbound DLQ.
# On the next boot, it is automatically redelivered with a marker:
#
#   ♻️ Recovered after restart — this reply may be a duplicate.
#
#   <original reply text>
#
# No operator step, no code change — symmetric with the inbound journal.

The user sends a message on Slack; transient send failures retry with backoff until the reply is delivered or parked in the DLQ — and a parked reply is automatically redelivered on the next gateway boot.

Quick Start

1. Retry and durable parking are both on by default

   Every bot adapter retries transient failures with backoff and parks a permanent / exhausted failure at a canonical per-platform DLQ so nothing is silently lost — with no config on your side.

   from praisonaiagents import Agent
   from praisonai.bots import SlackBot
   
   agent = Agent(
       name="Support Bot",
       instructions="Answer customer questions politely.",
   )
   bot = SlackBot(token="xoxb-...", agent=agent)
   bot.start()
   # Transient Slack 5xx / rate-limit responses are retried.
   # A permanently-failed reply (bad channel, revoked bot, etc.) is parked at
   #   ~/.praisonai/state/slack/outbound_dlq.sqlite
   # for later replay — with no config on your side.

2. Override the defaults only when you need to

   Set outbound_resilience in praisonai.yml only to override defaults — e.g. more attempts, a longer cap, or a custom DLQ path.

   bots:
     slack:
       outbound_resilience:
         initial_ms: 1000
         max_ms: 30000
         factor: 2.0
         max_attempts: 5
         jitter: 0.25
         # dlq_path: /var/lib/praisonai/slack-dlq.sqlite  # optional override

   The default DLQ path is ~/.praisonai/state/<platform>/outbound_dlq.sqlite — only set dlq_path if you need it elsewhere (e.g. a shared volume in Docker).

3. Opt out with a single flag

   enabled: false disables both retry and durable parking on that channel — a permanently-failed reply on that channel is lost, matching the pre-safe-by-default behaviour.

   bots:
     email:
       outbound_resilience:
         enabled: false

How It Works

When a send fails, the adapter classifies the error:

Automatic crash-recovery on start

Every gateway-managed channel redelivers any reply parked by a crash on a previous run — no operator step, symmetric with the inbound journal’s replay() sweep.

The gateway schedules WebSocketGateway._replay_outbound_dlq(bot) about 5 seconds after each channel task starts (a grace period for the transport to connect), then calls OutboundDLQ.redeliver(send) — draining parked replies oldest-first through the adapter’s own send_message, so the platform’s normal retry/backoff still applies. A redelivery that fails again is kept for the next boot, bounded by the DLQ’s existing TTL / max_size / attempts invariants. Failure never blocks channel start — the behaviour degrades to today’s “parked until manual replay”.

Users receive redelivered replies with a visible duplicate marker so an ambiguous redelivery is never silent:

♻️ Recovered after restart — this reply may be a duplicate. <original reply text>

The marker is prepended idempotently — a parked reply that already starts with it is not double-marked. When something is recovered you’ll see this line in the gateway log:

Outbound crash-recovery: redelivered N parked repl(y/ies) on start (marked as possible duplicates)

Automatic crash-recovery introduced in PraisonAI #3862.

Separate from this boot-time DLQ drainer, ChannelSupervisor also re-drains the durable outbox when a channel recovers from a transient outage — see Channel supervision → Recovery-triggered outbox re-drain.

Which Channels Does This Apply To?

Both retry/backoff and durable DLQ parking are on by default for every adapter — no dlq_path opt-in required.

¹ Default DLQ location: ~/.praisonai/state/<platform>/outbound_dlq.sqlite. Override with outbound_resilience.dlq_path; disable with outbound_resilience.enabled = false.

Both text and media now inherit the same policy:

Media uploads share this policy

The same retry wrapper covers outbound media uploads (DeliveryRouter.send_media → Telegram send_photo, Slack files_upload_v2, Discord file send). A transient transport failure on an image or file attachment is retried with the same bounded backoff and Retry-After honouring as the text path.

• The adapter’s configured _outbound_backoff BackoffPolicy is reused;
• if the adapter has no policy, a default of initial_ms=1000,
• max_ms=10000, factor=1.5, max_attempts=3 applies.
• Adapters with no upload primitive still return False immediately —
• no wasted retries.
• Permanent errors surface as before after the attempt budget is spent.

See Outbound Media Delivery → Retry & backoff on transient failures for the media-specific sequence diagram and behaviour matrix.

Configuration Options

All settings live under outbound_resilience in your channel config (in praisonai.yml or via BotConfig).

The canonical default path is resolved via resolve_durable_store_dir(<platform>). Set the PRAISONAI_HOME environment variable to relocate the whole ~/.praisonai/ root (useful in tests or containers).

Full YAML example:

bots:
  slack:
    outbound_resilience:
      # dlq_path is optional — defaults to ~/.praisonai/state/slack/outbound_dlq.sqlite
      max_attempts: 5
      max_ms: 30000
      factor: 2.0
  discord:
    outbound_resilience:
      # Custom DLQ path for a shared volume
      dlq_path: /var/lib/praisonai/state/discord/outbound_dlq.sqlite
      max_attempts: 3

Transient DLQ Init Recovery

If the default DLQ path is briefly unwritable when the adapter tries to initialise it (SQLite lock, disk permission race, transient FS error), Outbound Resilience does not latch itself into a permanently-degraded state. The failed send this turn is retried at the transport level and the DLQ init is re-attempted on the next send — so a first-turn storage blip does not silently disable durable parking for the life of the process.

You will see a Failed to initialize outbound DLQ (will retry on next send): <error> warning in the log the first time this happens; a subsequent send will silently succeed if storage has recovered.

Common Patterns

Opt one channel out entirely:

bots:
  email:
    outbound_resilience:
      enabled: false   # one attempt, no retry, no DLQ park

Use a shared DLQ path (e.g. a Docker volume):

bots:
  slack:
    outbound_resilience:
      dlq_path: /var/lib/praisonai/state/slack/outbound_dlq.sqlite

Aggressive retry for a flaky upstream (defaults handle 99% of cases):

bots:
  whatsapp:
    outbound_resilience:
      max_attempts: 5
      max_ms: 30000
      factor: 2.0

DLQ replay — failed replies parked in the SQLite DLQ can be replayed with the same tooling as the inbound DLQ. See Inbound DLQ for the analogous replay command.

Best Practices

# ✅ Default — durable, per-platform, isolated
# (omit dlq_path)

# ✅ Shared volume — multi-replica setup
dlq_path: /var/lib/praisonai/state/slack/outbound_dlq.sqlite

# ❌ Bad — lost on container restart
dlq_path: /tmp/slack-dlq.sqlite

Related