Degraded Channel Isolation
One channel's missing credential no longer takes the whole gateway down — that channel is isolated as degraded, the others keep serving
One channel’s unavailable credential isolates just that channel as degraded — every healthy channel keeps serving. The same surface now covers two cases: a token that was unavailable at boot (this page), and a token the platform rejected at runtime (see Runtime credential rejection).
from praisonaiagents import Agent
agent = Agent(name="assistant", instructions="Be helpful.")
# Rotate Slack's token overnight — Telegram + Discord keep serving; Slack is
# marked degraded until the new token is available.
agent.start("Anyone there?")Before this change one channel’s ValueError aborted the whole GatewayConfigSchema, taking every channel offline. Now the schema returns an empty token for the unavailable credential, the runtime’s pre-existing “no token → skip” path isolates that one channel, and health() reports it as degraded.
Token-free-by-design channels
These channels legitimately run with no token: and are never marked degraded for the missing credential:
| Channel | Why token-free |
|---|---|
email | SMTP/IMAP credentials live in their own config fields, not a token: |
agentmail | AI-native inbox with its own auth surface |
signal | Talks to a locally linked device, not a bearer token |
local | Owner-trusted terminal; nothing to authenticate against |
What Changed
A single channel’s unavailable credential is isolated; structural errors and the gateway’s own ingress secret still fail closed.
| Scenario | Before | After |
|---|---|---|
| Single channel token env var unset at boot | GatewayConfigSchema raised; all channels dead | Only that channel is skipped and marked degraded; siblings keep serving |
{source: file, id: /run/secrets/x} file empty at boot | Same — whole gateway down | Same isolation as above |
Structurally invalid config (missing platform:) | Fails closed | Still fails closed (unchanged) |
| Gateway’s own ingress / auth secret unavailable | Fails closed | Still fails closed (unchanged) |
Preflight probe running (--preflight, default) | Hard-abort on failure | Isolate the failing channel and serve every healthy channel — changed by PR #4863. The CLI start-gate now defers to the same degradation contract: a bad token no longer takes healthy channels offline. Fails closed only when --strict-preflight / gateway.preflight.strict: true is set, or when no channel is left serviceable |
| Token accepted at boot, rejected mid-flight (revoked/rotated/expired → 401/403) | Whole channel went to FAILED; required a process restart to try a new token | Isolated as CREDENTIAL_UNAVAILABLE (degraded); reconnect() / hot-reload picks up the fresh token without a restart. See Channel Supervision. |
The single most important distinction: a channel credential fails open (isolated, degraded); the gateway’s own config and ingress secret fail closed (abort).
How to Observe the Degraded State
The degraded channel is visible in the startup log, gateway doctor, channels --probe, and GET /health — no value is ever printed.
Startup log
One redacted WARNING is logged per degraded channel. An unset ${ENV} token logs:
Channel token env var 'SLACK_BOT_TOKEN' not set — channel will be skipped (degraded). Set it with: export SLACK_BOT_TOKEN=your_tokenA {source, id} reference that resolves to configured-but-unavailable / missing logs:
Channel secret configured-unavailable — channel will be skipped (degraded): env var 'SLACK_BOT_TOKEN' emptyThe token value is never printed — resolved secrets are registered for redaction.
praisonai gateway doctor
The secrets block reports configured-but-unavailable or missing for the affected field.
$ praisonai gateway doctor
Credential availability (values never shown):
telegram token ✓ available
slack token ✓ available
slack app_token ✗ configured-but-unavailableSee Credential availability (without revealing values) for the full table.
praisonai gateway channels --probe
The channel listing surfaces the same per-channel verdict — a one-line probe you can run against a live gateway.
GET /health on the running gateway
The degraded channel appears under channels, distinguished from a running channel by status: "degraded" and a reason.
{
"status": "healthy",
"channels": {
"telegram": { "platform": "telegram", "running": true },
"discord": { "platform": "discord", "running": true },
"slack": {
"platform": "slack",
"running": false,
"status": "degraded",
"reason": "credential unavailable"
}
}
}The same degraded channel also appears in the unified degraded_owners list, alongside any degraded provider, capability, or route — see Unified Degraded-Capability Registry.
{
"degraded_owners": [
{
"owner_kind": "channel",
"owner_id": "slack",
"state": "cold",
"reason": "credential unavailable",
"retry_hint": "praisonai gateway doctor --fix"
}
]
}A degraded channel stays queryable in health() — it does not silently vanish, so a monitor can tell “configured-but-unavailable” apart from “never configured”.
How to Recover
- Fix the credential
Rotate the env var, refill the mounted secret file, or restore the secret manager so the token resolves to
availableagain. - Reload the config
An empty credential is resolved once at config load, so
praisonai gateway reconnect <channel>alone does not re-read it — reconnect drives the channel supervisor, not the secret resolver. Trigger a config reload (or restart) so the schema re-resolves the now-available token. See Channel Supervision for reconnect/pause/resume. - Confirm it recovered
On the next config load, a channel that previously degraded but now has a token is removed from the degraded set automatically. Verify with
praisonai gateway doctororGET /health.
When You Do NOT Want This Behaviour
Some operators prefer strict fail-closed — enforce it at the operator layer, not the schema.
Preflight (--preflight, the default) now applies the same isolation contract at start-time: a channel with a bad credential is reported, skipped, and parked in CREDENTIAL_UNAVAILABLE while every healthy channel keeps serving. Preflight fails closed only when the operator opts into strict mode (--strict-preflight or gateway.preflight.strict: true) or when no channel is left serviceable (nothing to serve). Operators who want the pre-PR-#4863 behaviour — one bad token aborts everything — should opt into strict mode explicitly.
There is intentionally no knob to make the schema raise again — that behaviour was removed precisely because it took healthy channels down. To gate a deploy on strict availability, fail the release if praisonai gateway doctor --json reports any configured-but-unavailable or missing:
praisonai gateway doctor --config gateway.yaml --json > doctor.json
python -c "import json,sys; d=json.load(open('doctor.json')); \
bad=[(c,f,v) for c,fs in d.get('secrets',{}).items() for f,v in fs.items() if v not in ('available','configured')]; \
sys.exit(1) if bad else None"Strict opt-in
Opt back into fail-closed preflight when one bad token should abort the whole gateway. Set it in YAML for every start, or pass the flag for a one-off.
# gateway.yaml — opt back into fail-closed preflight
gateway:
preflight:
strict: true# One-off override without touching YAML
praisonai gateway start --config gateway.yaml --strict-preflight--strict-preflight / --no-strict-preflight overrides the YAML strict key for that run. Even without strict mode, preflight still fails closed when no channel is left serviceable.
Best Practices
Gate deploys on praisonai gateway doctor –json
Read the secrets block and fail the deploy if any field is not available (or configured for exec sources). This gives you strict fail-closed at release time while keeping the resilient fail-open behaviour at runtime.
Use {source: file} for K8s / Docker secrets
A mounted secret file survives a container restart; a process env var may not. Cross-link: Secret References. File-sourced credentials keep resolving after a restart, so a channel recovers on the next config load without manual intervention.
Alert on the degraded WARNING log line
The channel will be skipped (degraded) WARNING is your operator canary. Wire it into your alerting so a rotated-but-not-yet-refilled credential pages you before users notice the channel is quiet.
Never rely on preflight alone for continuous safety
Preflight only checks credentials at start. A token can go stale after preflight passes — expiry, rotation, a secret-store blip. Degraded-channel isolation is what keeps the other channels serving when that happens.
Related
Load credentials from files, env vars, or secret managers.
Gateway CLIgateway doctor and channels --probe credential availability.
CREDENTIAL_UNAVAILABLE state and the refresh_credentials() hook.
Drain, admission, and fail-closed opt-outs.