gateway status / doctor โ so a cross-instance blackout is never silent.
When Redis drops, the adapter marks itself degraded, reconnects with bounded backoff, re-subscribes every channel, and clears the degraded record on recovery โ while counting any writes that hit the disconnected transport.
Quick Start
1
Run an agent behind an HA gateway
The agent code is unchanged โ the resilience lives in the fan-out transport:
2
Enable Redis fan-out in gateway.yaml
Point the push config at your Redis instance for cross-instance delivery:
3
Watch outage signals via gateway status
The three
redis_* fields tell you the transportโs health at a glance:How It Works
On a listener error the adapter drops its stale handles, marks itself degraded, then reconnects with exponential backoff (capped at 30s) and re-subscribes every tracked channel before resuming.- Bounded exponential backoff โ retries start at
1s, double each failure, and cap at30s. The loop only exits if the adapter is disconnected. - Automatic re-subscription โ every channel tracked before the outage is re-subscribed on the fresh pub/sub handle.
- Degraded surfacing โ the adapter records itself as owner
("route", "redis-pubsub")withstate="stale", a redacted reason (exception string only โ never the URL or password), and an actionableretry_hint. - Dropped-write counting โ
publish,set_presence,remove_presence,store_message, anddelete_messagecalls made while disconnected incrementdropped_writesinstead of vanishing. - Clears on recovery โ the degraded record is removed once Redis is reachable again.
Health Signals
health() (and gateway status) expose three fields on the push status block.
During an outage, the degraded transport is listed under
degraded_owners:
redis_* fields return to healthy and the route:redis-pubsub entry is gone. Watch for the Redis push adapter reconnected (server_id=<uuid>) log line.
Doctor Integration
The degraded record shows up inpraisonai gateway doctor (see Extensible Gateway Doctor) with the actionable retry_hint. No new plugin is needed โ the adapter registers itself with the shared DegradedCapabilityRegistry.
Best Practices
Alert on route:redis-pubsub degradation
Alert on route:redis-pubsub degradation
The
degraded_owners entry for route:redis-pubsub is the single hook for a cross-instance-transport pager. It appears for the outage duration and clears automatically on reconnect.Watch redis_dropped_writes after an outage
Watch redis_dropped_writes after an outage
A non-zero
push.redis_dropped_writes tells you how many events fanned out one-sided. Alert on it if replay matters โ those writes are gone from the transportโs perspective.Trust the redacted reason
Trust the redacted reason
The
reason only ever carries the exception message. The Redis URL and password are never surfaced, so the field is safe to log or page on.Let the adapter reconnect itself
Let the adapter reconnect itself
Backoff and re-subscription are automatic. Do not restart the gateway to recover Redis โ the listener re-establishes every channel on its own.
Related
Extensible Gateway Doctor
Where the degraded record surfaces with its retry hint
Push Notifications
Channel pub/sub and HA fan-out over Redis
Durable Polling
At-least-once delivery for the long-poll fallback transport
Gateway Liveness
Health and liveness signals for the gateway

