Gateway Troubleshooting

Common gateway issues and step-by-step solutions

Common issues with the PraisonAI Gateway daemon and server, with step-by-step troubleshooting guides.

from praisonaiagents import Agent

agent = Agent(
    name="Gateway Helper",
    instructions="Diagnose PraisonAI Gateway connectivity issues.",
)
agent.start("Gateway health check: is the daemon reachable?")

The user checks gateway status, reads logs, and restarts the daemon until channels respond again.

Common Issues

Is this the core or a plugin/hook?

Symptom: The gateway misbehaves and you can’t tell whether the fault is in the gateway core or in an external plugin / inbound hook.

Boot core-only in one restart with Gateway Safe Mode:

praisonai gateway start --config gateway.yaml --safe-mode
# → logs: SAFE MODE: N inbound hook(s) disabled; external plugins skipped (PRAISONAI_NO_PLUGINS)

If the fault disappears, it is a plugin or inbound hook; if it persists, it is the gateway core. Aliases --pure / --no-plugins are equivalent, and PRAISONAI_NO_PLUGINS=1 works for crash loops. Safe mode is replayed by restart — exit with a fresh gateway start without the flag.

Port Already In Use / Two Gateways Running

Symptom: Gateway fails to start with a clear error message about port conflicts.

Error: Gateway port 8765 is already in use.

  Another gateway may be running (PID 8864).
  Stop it:  praisonai gateway stop

  Only ONE gateway process may poll each bot token. To run a second,
  isolated gateway give it its own state, bot tokens and port:
    HOME=/srv/gateway-b GATEWAY_PORT=8766 praisonai gateway start

A second gateway on another port that shares ~/.praisonai is refused by the state lock instead:

Error: another gateway already owns the state at /home/user/.praisonai (PID 12345 on 127.0.0.1:8765).

  Two gateways sharing state poll the same bot tokens and write the same
  queues, causing 409 Conflict errors and duplicate delivery.
  Stop it:  praisonai gateway stop
  Or run an isolated gateway with its own state and bot tokens:
    HOME=/srv/gateway-b praisonai gateway start

1. Check PID lock and port status

   praisonai gateway status

   Look for lines like:

   • Gateway PID lock: Process 12345 running (127.0.0.1:8765)
   • Port 127.0.0.1:8765: In use
2. Stop the existing gateway gracefully

   praisonai gateway stop

   This sends SIGTERM and waits for graceful shutdown. The gateway drains active sessions for up to 10 seconds — in-flight turns and queued inbox messages are persisted before exit. Sessions that exceed the timeout are force-persisted with a SESSION_END event (had_pending_work, was_executing). Use --force to skip the drain. See Gateway Session Continuity.

3. If gateway is stuck, force stop

   praisonai gateway stop --force

   This sends SIGKILL directly (SIGTERM on Windows) for immediate termination.

4. If port is used by non-gateway process

   # Find what's using the port
   lsof -i :8765
   # Linux alternative:
   netstat -ano | findstr :8765
   
   # Either kill that process or use a different port
   GATEWAY_PORT=8766 praisonai gateway start
   # Or:
   praisonai gateway start --port 8766

5. Clean up stale lock file (if needed)

   # Check the PID lock file
   cat ~/.praisonai/gateway-127_0_0_1-8765.pid
   
   # Check the state lock file
   cat ~/.praisonai/gateway.state.pid
   
   # Remove stale locks manually (auto-cleaned on next start)
   rm ~/.praisonai/gateway-127_0_0_1-8765.pid
   rm ~/.praisonai/gateway.state.pid ~/.praisonai/gateway.state.pid.guard

   Only remove lock files when no gateway is running.

   Stale locks are automatically detected and removed, but you can clean them manually if needed.

SSL Certificate Verify Failed on Start (Corporate Proxy / MITM)

Symptom: praisonai gateway start --config gateway.yaml fails with an SSLCertVerificationError in gateway doctor, but the same channel token works fine with --no-preflight.

Cause: Your network intercepts TLS with a corporate CA the probe’s HTTP client does not trust yet. The runtime bot adapter is often more permissive, so the token itself is usually fine.

Fix (pick one):

1. Point PraisonAI at your corporate CA (preferred)

   export PRAISONAI_SSL_CA_BUNDLE=/path/to/corp-ca.pem
   praisonai gateway start --config gateway.yaml

   PRAISONAI_SSL_CA_BUNDLE overrides any pre-existing SSL_CERT_FILE and REQUESTS_CA_BUNDLE for the probe.

2. Use the standard SSL env vars

   export SSL_CERT_FILE=/path/to/corp-ca.pem   # or REQUESTS_CA_BUNDLE
   praisonai gateway start --config gateway.yaml

3. Skip the preflight check

   praisonai gateway start --config gateway.yaml --no-preflight

   Skips channel-credential validation on start. Use only when you know the tokens are good.

Gateway start aborts with ‘Turn pre-flight failed’

Symptom: praisonai gateway start exits with:

Turn pre-flight failed — the agent did not produce a reply: <detail>

Cause: The channel credentials are valid, but the resolved agent’s model provider credential is missing / invalid / rate-limited / out of quota. The startup real-turn check (--verify-turn, on by default) sends one "ping" prompt to the agent and the model round-trip failed.

Fix (any of):

1. Add the model/provider API key

   # Add the key (e.g. OPENAI_API_KEY) to ~/.praisonai/.env, then:
   praisonai gateway start

2. Reproduce interactively

   praisonai gateway test --turn ping

3. Temporarily skip the turn check

   praisonai gateway start --no-verify-turn

   Or set gateway.preflight.verify_turn: false in bot.yaml. Use only when you know the model key is good (e.g. constrained/offline environments).

Telegram Bot Goes Silent After Restart

Symptom: Gateway /health endpoint returns healthy, but Telegram messages aren’t received after a restart or crash.

Cause: Two processes are polling the same Telegram bot token. Telegram delivers messages to only one poller, causing the bot to appear “silent” when the wrong process gets the messages.

1. Stop all gateway instances

   praisonai gateway stop --force

   This ensures all instances are terminated, even if PID files are corrupted.

2. Verify no processes are running

   praisonai gateway status

   Should show:

   • Gateway PID lock: No lock file found
   • Port 127.0.0.1:8765: Available
3. Start a single gateway instance

   praisonai gateway start

   Only start one instance to ensure exclusive bot token polling.

4. Test bot responsiveness

   Send a message to your Telegram bot. It should respond normally now that only one process is polling the token.

Daemon Running But Gateway Unreachable

Symptom: praisonai gateway status shows Daemon service: Running (launchd) but Gateway not reachable at http://127.0.0.1:8765/health.

1. Verify daemon is actually running

   praisonai gateway status --daemon-only

   Look for Running status and process ID.

2. Check daemon logs for errors

   praisonai gateway logs
   
   # Or check raw log files:
   # macOS: tail ~/.praisonai/logs/bot-stderr.log
   # Linux: journalctl --user -u praisonai-bot

   Look for Python tracebacks or port binding errors.

3. Check port and PID lock status

   praisonai gateway status

   This now reports both port usage and PID lock status directly. Look for:

   • Gateway PID lock: Process <pid> running or No lock file found
   • Port 127.0.0.1:8765: In use or Available

   If another process is using the port, use praisonai gateway stop to stop an existing gateway, or choose a different port.

4. Verify PraisonAI version

   praisonai --version

   Upgrade to ≥ v4.6.23 if you see older versions – earlier versions had IndentationError bugs fixed in PR #1484.

5. Restart the daemon

   # macOS
   launchctl kickstart -k gui/$(id -u)/ai.praison.bot
   
   # Linux  
   systemctl --user restart praisonai-bot
   
   # Or reinstall completely
   praisonai onboard

Rapidly Growing Log Files

Symptom: ~/.praisonai/logs/bot-stderr.log grows to multiple MB per minute.

1. Check log file size

   ls -lh ~/.praisonai/logs/bot-stderr.log

   If growing rapidly (>1MB/min), this indicates a crash loop.

2. View recent errors

   tail -50 ~/.praisonai/logs/bot-stderr.log

   Look for repeated Python tracebacks, especially IndentationError.

3. Stop the daemon

   # macOS
   launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist
   
   # Linux
   systemctl --user stop praisonai-bot

4. Clear logs and restart

   # Clear the log file
   > ~/.praisonai/logs/bot-stderr.log
   
   # Upgrade PraisonAI
   pip install --upgrade praisonai
   
   # Restart daemon
   praisonai gateway install --start

Daemon Not Installed

Symptom: praisonai gateway status shows Daemon service: Not installed (systemd).

1. Run the onboarding wizard

   praisonai onboard

   This installs the daemon service for your platform.

2. Verify installation

   praisonai gateway status --daemon-only

   Should show Installed but not running or Running.

3. Start the service

   # Manual start
   praisonai gateway install --start
   
   # Or check platform-specific commands
   praisonai gateway status

HTTP 500 on Health Endpoint

Symptom: curl http://127.0.0.1:8765/health returns 500 Internal Server Error.

1. Check PraisonAI version

   praisonai --version

   Versions before v4.6.23 had AttributeError bugs in the health endpoint.

2. Upgrade PraisonAI

   pip install --upgrade praisonai

3. Restart the gateway

   praisonai gateway install --start

4. Test health endpoint

   curl http://127.0.0.1:8765/health

   Should return JSON with status, uptime, agents, sessions, clients, and channels.

Windows: gateway status shows PID lock status: Unavailable

Symptom: praisonai gateway status prints an advisory PID-lock line, followed by a healthy gateway:

PID lock status: Unavailable (kill returned a result with an exception set)
Gateway Status: healthy
  Uptime: ...

On older versions (pre-v4.6.141) the same underlying cause produced a harder failure that hid the status entirely:

ERROR: Error checking gateway server status:  returned a result with an exception set

Root cause: On Windows, os.kill(pid, 0) — used by GatewayPIDLock._is_process_running to poll whether a lock is stale — can raise SystemError (“returned a result with an exception set”) instead of OSError / ProcessLookupError. Before PraisonAI v4.6.141 that exception propagated through GatewayHandler.status and short-circuited the /health probe, so operators saw no gateway status at all.

What v4.6.141 changes:

1. _is_process_running now catches SystemError and ValueError and returns False (process not running).
2. GatewayHandler.status treats PID-lock inspection as advisory only. Any unexpected exception prints PID lock status: Unavailable (<error>) and the command continues to the /health probe.

1. Check your version

   praisonai --version

   Upgrade to ≥ v4.6.141 if you still see ERROR: Error checking gateway server status: returned a result with an exception set.

2. Read the line as advisory

   If praisonai gateway status now prints PID lock status: Unavailable (<error>) followed by Gateway Status: healthy, the gateway is running — the PID line is advisory only.

3. Confirm with the health endpoint

   curl http://127.0.0.1:8765/health

   A 200 healthy response confirms the gateway is up regardless of the PID-lock line.

4. Inspect the lock file if needed

   # PowerShell
   Get-Content $env:USERPROFILE\.praisonai\gateway-127_0_0_1-8765.pid

   The lines are pid, host, port, timestamp, and (since PR #4197) a create_time start-time fingerprint. The 5th line may be empty on older 4-line locks or when psutil is unavailable — the lock then degrades to the previous PID-only check. A stale PID is cleaned up automatically on the next start/stop/status call.

Windows: ‘charmap’ codec error from Telegram bot replies

Symptom: Telegram users receive Error: 'charmap' codec can't encode character '⚠' in position N: character maps to <undefined> instead of the real error message.

Root cause: Windows default console encoding is cp1252, not UTF-8. When agent exceptions contained warning symbols (⚠), emoji, or accented text, the error formatter crashed before the real error could be reported.

1. Verify your version contains the fix

   python -c "from praisonai.gateway.unicode_utils import safe_error_message; print('OK')"

   If this command runs without error, you have the fix from PR #1754.

2. Upgrade to a fixed version

   pip install --upgrade praisonai

   Upgrade to PraisonAI version that contains PR #1754 — Gateway and Telegram bot error handlers now sanitize exception text to ASCII-safe form for transport, while preserving full Unicode in logs.

3. Restart the gateway

   praisonai gateway install --start

4. Test the fix

   Users will now see clean error messages instead of charmap crashes:

   • Error: API quota exceeded. Check billing. (was: charmap crash hiding OpenAI 429)
   • Error: Rate limit exceeded. Try again later.
   • Error: Authentication failed. Check API key.
   • Error: Request timeout. Try again.

Permission Denied Errors

Symptom: Daemon fails to start with permission errors in logs.

# Check LaunchAgent permissions
ls -la ~/Library/LaunchAgents/ai.praison.bot.plist

# Reload LaunchAgent
launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl load ~/Library/LaunchAgents/ai.praison.bot.plist

# Check systemd user service
systemctl --user status praisonai-bot

# Enable user lingering (allows services to run when not logged in)
sudo loginctl enable-linger $(whoami)

# Restart service
systemctl --user daemon-reload
systemctl --user restart praisonai-bot

# Run as administrator
# Check Task Scheduler for PraisonAI task
schtasks /Query /TN PraisonAIGateway

# Delete and recreate
praisonai gateway uninstall
praisonai gateway install

Clean Reinstall Process

When all else fails, perform a clean reinstall:

1. Stop and uninstall

   praisonai gateway uninstall

2. Clear configuration

   # Backup first if needed
   rm -rf ~/.praisonai/logs
   rm -rf ~/.praisonai/config

3. Upgrade PraisonAI

   pip install --upgrade praisonai

4. Run onboarding

   praisonai onboard

   Follow the wizard to reinstall the daemon service.

5. Verify installation

   praisonai gateway status

   Should show daemon running and gateway reachable.

Client stops reconnecting after agent_not_found

Symptom: GatewayClient logs Connection abandoned: agent_not_found (reconnect paused; not retrying) and stops attempting to connect, even though the gateway is healthy.

Cause: The agent_id supplied in the client’s hello frame is not registered on the gateway. The server emits agent_not_found with next_step: do_not_retry, and the client now correctly stops the reconnect loop instead of backing off forever.

1. Check the agent_id you pass

   GatewayClient(url="ws://localhost:8765", agent_id="assistant")
   #                                                  ^^^^^^^^^ must exist on the server

2. List agents registered on the gateway

   curl http://127.0.0.1:8765/health | jq '.agents'

   Compare against the id you’re passing.

3. Fix the id or register the agent

   Either update the client’s agent_id, or add the agent to the gateway configuration and restart it. Then call client.connect() again — the fresh call clears any stale backoff floor from the previous cycle.

4. Surface the failure without log scraping

   Setting client.on_reconnect_paused before connect() gives you an immediate callback with (code, next_step) so the failure surfaces without waiting for a log scrape. See Gateway Client → Terminal vs Transient Connect Errors.

Cross-instance messages not being delivered / Redis pub/sub outage

Symptom: Subscribers on gateway B stop seeing messages that were published on gateway A. praisonai gateway status shows push.redis_degraded: true and a growing push.redis_dropped_writes; health()["degraded_owners"] lists route:redis-pubsub.

Cause: The Redis pub/sub connection that fans out channel messages across gateway instances dropped. Cross-instance delivery pauses until the adapter reconnects.

1. Confirm the degraded owner and its retry_hint

   curl http://127.0.0.1:8765/health | jq '.push, .degraded_owners'

   Look for push.redis_degraded: true and a route:redis-pubsub entry whose retry_hint points you at praisonai gateway doctor.

2. Run gateway doctor

   praisonai gateway doctor

   Surfaces the same record with the actionable hint.

3. Verify Redis reachability from the gateway host

   redis-cli -h <redis-host> ping

   A PONG confirms Redis is reachable; if it fails, fix the Redis connection or network path.

Recovery: No manual intervention on the gateway is needed. Once Redis is reachable again the adapter reconnects with bounded exponential backoff (1s → 30s cap), re-subscribes every channel, and clears route:redis-pubsub from degraded_owners. The log line to look for is Redis push adapter reconnected (server_id=<uuid>).

See Real-Time Push Notifications → HA & cross-instance delivery for the full contract.

Auth lockout looser than expected on a multi-replica gateway

Symptom: Magic-link or approval lockout lets through roughly N× more attempts than the documented max_attempts on an N-replica gateway.

Cause: push.redis is not configured, so each replica keeps its own auth rate-limit bucket.

Fix: Point every replica at the same Redis in gateway.push.redis — the same block the turn lock and idempotency store already share. If the log shows gateway.push.redis set but Redis auth rate limiter unavailable, install the redis package. See Cluster-Wide Auth Rate Limiting.

Reading the hello_error Envelope

When the gateway rejects a connection, it sends a structured hello_error frame before closing. Each (code, next_step) pair maps to a specific operator action.

Example envelope received on rate-limit:

{
  "type": "hello_error",
  "code": "rate_limited",
  "message": "Too many connection attempts",
  "next_step": "wait_then_retry",
  "retry_after_seconds": 30,
  "next": "wait_then_retry"
}

See Gateway Handshake Protocol for the full field reference and ConnectRecoveryStep values.

Authentication Errors

GatewayStartupError: Cannot bind to 0.0.0.0 without an auth token

Symptom: Gateway fails to start when binding to external interfaces without authentication.

1. Use onboarding wizard

   praisonai onboard

   This automatically generates and saves a secure token.

2. Or set token manually

   export GATEWAY_AUTH_TOKEN=$(openssl rand -hex 16)
   praisonai gateway start --host 0.0.0.0

UIStartupError: Cannot bind to 0.0.0.0 with default admin/admin credentials

Symptom: Chainlit UI fails to start on external interface with default credentials.

1. Set custom credentials

   export CHAINLIT_USERNAME=myuser
   export CHAINLIT_PASSWORD=mypass
   praisonai chat --host 0.0.0.0

2. Or allow defaults for demos (unsafe)

   export PRAISONAI_ALLOW_DEFAULT_CREDS=1
   praisonai chat --host 0.0.0.0

Gateway refuses to start: gateway.auth_token is weak/placeholder

Symptom: doctor or start refuses with “gateway.auth_token is a known-weak/placeholder value”.

Preview the repair, then apply it — --fix mints a strong token and re-validates:

1. Preview (writes nothing, exits 1)

   praisonai gateway doctor --config gateway.yaml --fix --dry-run

   gateway_auth_token: weak → would mint a strong token (--dry-run)

2. Apply (mints, persists, re-validates)

   praisonai gateway doctor --config gateway.yaml --fix

   gateway_auth_token: weak → generated a strong token… done
   re-validated: gateway_auth_token now strong

An explicit weak auth_token in gateway.yaml is rewritten in place. ${ENV} references are left alone — they resolve from ~/.praisonai/.env, which --fix already updated. See Gateway CLI › Auto-repair.

My gateway logs show gw_****xxxx instead of the full token

This is intentional for security — tokens are fingerprinted in logs to prevent exposure.

1. Retrieve full token from environment file

   cat ~/.praisonai/.env | grep GATEWAY_AUTH_TOKEN

2. Or check environment variables

   echo $GATEWAY_AUTH_TOKEN

Magic-link login works in WebSocket but fails over HTTP (or vice versa)

Symptom: One auth method works but the other fails, even with the same token.

Cause (pre-fix): HTTP/magic-link and WebSocket used different secret sources before PR #1744.

1. Upgrade PraisonAI

   Upgrade to PraisonAI ≥ v4.6.47 (ships PR #1744). Config token now exports to env, unifying all auth paths.

2. Restart the gateway

   praisonai gateway restart

   Config token precedence is now enforced on restart.

Restart After Config Change

When you update bot configuration files, restart the daemon using these OS-specific commands (matching the onboard Done panel):

launchctl kickstart -k gui/$(id -u)/ai.praison.bot

systemctl --user restart praisonai-bot

schtasks /End /TN PraisonAIGateway && schtasks /Run /TN PraisonAIGateway

Diagnostic Commands

Quick commands for gathering diagnostic information:

# Full status check
praisonai gateway status

# Daemon-only check (for scripts)
praisonai gateway status --daemon-only

# Recent logs (last 50 lines)
praisonai gateway logs -n 50

# Check the PID lock file
cat ~/.praisonai/gateway-127_0_0_1-8765.pid

# Stop a running gateway
praisonai gateway stop
praisonai gateway stop --force

# Check port usage
lsof -i :8765

# Test health endpoint directly
curl -v http://127.0.0.1:8765/health

# Check PraisonAI version
praisonai --version

# Platform daemon status
# macOS: launchctl list | grep ai.praison
# Linux: systemctl --user status praisonai-bot
# Windows: schtasks /Query /TN PraisonAIGateway

Platform-Specific Notes

# Manual management
launchctl load ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl kickstart -k gui/$(id -u)/ai.praison.bot

# Check if loaded
launchctl list | grep ai.praison

# Manual management
systemctl --user start praisonai-bot
systemctl --user stop praisonai-bot
systemctl --user restart praisonai-bot
systemctl --user enable praisonai-bot

# Check status
systemctl --user status praisonai-bot

# Manual management via Task Scheduler or:
schtasks /Run /TN PraisonAIGateway
schtasks /End /TN PraisonAIGateway

# Check status
schtasks /Query /TN PraisonAIGateway

Config reload did not apply

Symptom: You edited gateway.yaml or sent SIGHUP, but channels or agents still run the old configuration.

See Gateway Config Reload for file-watch vs SIGHUP paths and YAML keys.

Multi-Channel Troubleshooting

Common failure modes when using multiple bots on the same platform:

_BOT_TOKEN (e.g. TELEGRAM_CFO_BOT_TOKEN) | | Missing token: <env> | Env var referenced in bot.yaml is unset | Add it to ~/.praisonai/.env` |

Example Fix:

# Problem: Both channels use the same token
# Error: "Token TELEGRAM_BOT_TOKEN is used by multiple channels"

# Solution: Create unique tokens per role
# 1. Create new bot in @BotFather for CFO role
# 2. Add unique environment variable
export TELEGRAM_CFO_BOT_TOKEN="987654321:DEF..."

# 3. Update bot.yaml
channels:
  telegram_cfo:
    platform: telegram
    token: ${TELEGRAM_CFO_BOT_TOKEN}  # Unique token
    routes:
      default: cfo

Validation:

# Check multi-channel configuration
praisonai doctor multi_channel_tokens

# ✅ Expected output:
# "Multi-channel configuration looks good (N channels)"

Group Policy Issues

Bot answers group @mentions with no context of the preceding conversation

You’re on group_policy: mention_only — unmentioned messages are dropped and leave no session record, so an @mention only sees its own text. Switch to group_policy: observe (Telegram) so the transcript retains unmentioned messages as passive context. Combine with session_scope: per_chat for shared group memory. Note that a reply to the bot is still admitted as an implicit mention, so the natural follow-up (tap Reply on the bot’s answer) works without re-typing @bot — see Implicit Mentions to disable this or enable quote/thread signals.

channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    group_policy: observe        # keep unmentioned messages as context
    session:
      session_scope: per_chat    # share that context across the group

Channel Supervision Issues

Channel goes silent but /health shows running: true

Symptom: Channel stops receiving messages but /health still reports "running": true.

Cause: Pre-PR-2041 behaviour, or the health: block is missing from gateway.yaml — hung sockets are not detected until an exception is raised.

Fix: Add the proactive health block:

gateway:
  health:
    interval: 300
    stale_after: 120
    enabled: true

See Proactive Health Monitoring.

Channel keeps restarting every 5 minutes

Symptom: Channel restarts on a regular interval visible in logs.

Cause: stale_after too low for a quiet channel, or interval too aggressive for a slow remote API.

Fix: Raise stale_after to e.g. 600, or lower max_restarts_per_hour so the cap kicks in and surfaces the issue:

gateway:
  health:
    stale_after: 600
    max_restarts_per_hour: 3

Every channel restarts at once — fleet crash-loop breaker tripped

Symptom: One of:

• Log line Fleet crash-loop breaker TRIPPED: holding channel restarts (...).
• praisonai gateway status shows fleet.breaker_tripped: true.
• list_degraded() / GET /health shows a gateway owner with owner_id: "fleet".

Cause: A systemic upstream fault — a bad shared provider, a network partition, or an org-wide expired token — is restarting every channel at once. The fleet-level breaker tripped one aggregate breaker instead of letting each channel silently burn its own max_restarts_per_hour budget.

1. Diagnose the shared cause

   praisonai gateway doctor --config gateway.yaml

   Look for the shared provider / token / network issue behind the storm — a single fault is affecting all channels.

2. Fix the underlying fault

   Rotate the shared token, restore the provider, or heal the network partition. While the breaker is tripped, restarts are held on purpose — you do not need to restart the gateway.

3. Wait one cooldown, then confirm recovery

   After the cause is fixed, wait one breaker_cooldown_s (default 120s). The breaker re-arms cleanly and the gateway/fleet degraded entry clears on the next monitor sweep — no manual poll needed. Confirm with:

   curl http://127.0.0.1:8765/health | jq '.channels'   # fleet owner should be gone

Channel Shows state: failed in /health

Symptom: When checking GET /health, a channel shows "state": "failed" with error details.

Common Causes:

• Telegram Conflict: Multiple bot instances using the same token
• Invalid Credentials: Bot token revoked or incorrect
• Permission Issues: Bot lacks required permissions

Investigation Steps:

1. Check Error Details

   curl http://127.0.0.1:8765/health | jq '.channels.telegram.supervision'

   Look for last_error and last_error_time fields.

2. Telegram Conflict Resolution

   If error contains “Conflict: terminated by other getUpdates”:

   # Stop any other bot instances using the same token
   # Then force reconnect to reset state
   praisonai gateway reconnect telegram

3. Credential Verification

   For invalid token errors, verify your bot token:

   # Test token directly
   curl "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/getMe"

Channel Keeps Retrying Forever

Symptom: High total_recoveries count, constant retry attempts visible in logs.

Investigation:

# Check supervision status
curl http://127.0.0.1:8765/health | jq '.channels'

# Look for:
# - High total_recoveries value
# - next_retry_at timestamp
# - last_error details

Resolution:

# Temporarily pause while investigating
praisonai gateway pause telegram

# Investigate root cause (network, DNS, etc.)
# Fix underlying issue

# Resume when ready
praisonai gateway resume telegram

Config Migration

When praisonai doctor --only gateway_config_migration reports Config can be migrated, your YAML uses a legacy shape that loads correctly but can be rewritten to the canonical format.

1. Detect migration opportunities

   praisonai doctor --only gateway_config_migration

   Example WARN output:

   ⚠ Gateway Config Migration [MEDIUM]
     Config can be migrated: 2 change(s)
     • telegram: allowed_users string → list migration available
     • Set telegram.group_policy to secure default 'mention_only'

2. Understand auto-normalisation

   At load time, GatewayConfigSchema already normalises legacy formats — your bot runs without rewriting the file. Persist the canonical form when you want the on-disk YAML to match what the schema produces.

3. Apply canonical YAML

   Legacy shape	Canonical form
   Top-level platform + token	channels: dict with one entry
   BotOS platforms: dict	channels: dict (same keys)
   allowed_users: "id1,id2" (string)	allowed_users: ["id1", "id2"] (list)

   See Gateway Config Migration for before/after YAML.

Environment Variables in Config

Gateway and bot configs support ${VAR} substitution in any string value. Resolution order:

1. Process environment variables
2. ~/.praisonai/.env (loaded automatically; override path with PRAISONAI_ENV_FILE)

# Verify all referenced variables resolve
praisonai doctor --only gateway_env_substitution

Unset variables cause validation to FAIL at load time and in the gateway_env_substitution doctor check.

Best Practices

Related