> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Redis Pub/Sub Outage Handling
> Multi-instance gateway HA transport reconnects, surfaces outages, and counts dropped writes
The Redis pub/sub adapter powering multi-instance gateway fan-out reconnects on a dropped connection, re-subscribes every channel, and reports the outage through `gateway status` / `doctor` ā so a cross-instance blackout is never silent.
```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
subgraph "Outage & Recovery"
L[š Listener] --> X{š„ Connection dropped?}
X -->|No| Fan[š” Fan-out]
X -->|Yes| Mark[ā ļø Mark degraded]
Mark --> Back[š Backoff 1sā30s]
Back --> Re[š Re-subscribe]
Re --> Clear[ā
Clear degraded]
end
classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
classDef warn fill:#F59E0B,stroke:#7C90A0,color:#fff
classDef success fill:#10B981,stroke:#7C90A0,color:#fff
class L input
class X,Re,Back process
class Mark warn
class Fan,Clear success
```
When Redis drops, the adapter marks itself degraded, reconnects with bounded backoff, re-subscribes every channel, and clears the degraded record on recovery ā while counting any writes that hit the disconnected transport.
## Quick Start
The agent code is unchanged ā the resilience lives in the fan-out transport:
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
agent = Agent(
name="alerts-agent",
instructions="Summarise incoming alert events in one sentence",
)
agent.start("Ready to fan out across gateway instances")
```
Point the push config at your Redis instance for cross-instance delivery:
```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# gateway.yaml
push:
enabled: true
redis:
host: redis.internal
port: 6379
```
The three `redis_*` fields tell you the transport's health at a glance:
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
praisonai gateway status --json
# push.redis_connected: bool (false while degraded)
# push.redis_degraded: bool (true during the outage window)
# push.redis_dropped_writes: int (cumulative)
```
***
## How It Works
On a listener error the adapter drops its stale handles, marks itself degraded, then reconnects with exponential backoff (capped at 30s) and re-subscribes every tracked channel before resuming.
```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
participant Listener
participant Redis
participant Registry as DegradedCapabilityRegistry
Listener->>Redis: get_message()
Redis--xListener: connection dropped
Listener->>Listener: drop stale client + pubsub handles
Listener->>Registry: mark(route:redis-pubsub, state=stale)
loop backoff 1s ā 30s cap
Listener->>Redis: reconnect attempt
end
Redis-->>Listener: reconnected
Listener->>Redis: re-subscribe every channel
Listener->>Registry: clear(route, redis-pubsub)
```
* **Bounded exponential backoff** ā retries start at `1s`, double each failure, and cap at `30s`. The loop only exits if the adapter is disconnected.
* **Automatic re-subscription** ā every channel tracked before the outage is re-subscribed on the fresh pub/sub handle.
* **Degraded surfacing** ā the adapter records itself as owner `("route", "redis-pubsub")` with `state="stale"`, a redacted reason (exception string only ā never the URL or password), and an actionable `retry_hint`.
* **Dropped-write counting** ā `publish`, `set_presence`, `remove_presence`, `store_message`, and `delete_message` calls made while disconnected increment `dropped_writes` instead of vanishing.
* **Clears on recovery** ā the degraded record is removed once Redis is reachable again.
***
## Health Signals
`health()` (and `gateway status`) expose three fields on the `push` status block.
| Field | Type | Meaning |
| --------------------------- | ------ | ------------------------------------------------------------------------------ |
| `push.redis_connected` | `bool` | `true` only when the client is live **and** not degraded |
| `push.redis_degraded` | `bool` | `true` while the reconnect is in flight |
| `push.redis_dropped_writes` | `int` | Cumulative publish/presence/store/delete calls that hit a disconnected adapter |
During an outage, the degraded transport is listed under `degraded_owners`:
```json theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
{
"push": {
"redis_connected": false,
"redis_degraded": true,
"redis_dropped_writes": 3,
"online_clients": 12
},
"degraded_owners": [
{
"owner_kind": "route",
"owner_id": "redis-pubsub",
"state": "stale",
"reason": "redis pub/sub disconnected: Connection refused",
"retry_hint": "check Redis connectivity; see `praisonai gateway doctor`"
}
]
}
```
After recovery the `redis_*` fields return to healthy and the `route:redis-pubsub` entry is gone. Watch for the `Redis push adapter reconnected (server_id=)` log line.
Writes counted in `redis_dropped_writes` are **not** replayed on reconnect. If you need durable cross-instance delivery, publish from a source of truth you can replay.
***
## Doctor Integration
The degraded record shows up in `praisonai gateway doctor` (see [Extensible Gateway Doctor](/docs/features/gateway-doctor-plugins)) with the actionable `retry_hint`. No new plugin is needed ā the adapter registers itself with the shared `DegradedCapabilityRegistry`.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
praisonai gateway doctor
# route:redis-pubsub stale
# redis pub/sub disconnected: Connection refused
# ā check Redis connectivity; see `praisonai gateway doctor`
```
***
## Best Practices
The `degraded_owners` entry for `route:redis-pubsub` is the single hook for a cross-instance-transport pager. It appears for the outage duration and clears automatically on reconnect.
A non-zero `push.redis_dropped_writes` tells you how many events fanned out one-sided. Alert on it if replay matters ā those writes are gone from the transport's perspective.
The `reason` only ever carries the exception message. The Redis URL and password are never surfaced, so the field is safe to log or page on.
Backoff and re-subscription are automatic. Do not restart the gateway to recover Redis ā the listener re-establishes every channel on its own.
***
## Related
Where the degraded record surfaces with its retry hint
Channel pub/sub and HA fan-out over Redis
At-least-once delivery for the long-poll fallback transport
Health and liveness signals for the gateway