Gateway Troubleshooting

Common issues with the PraisonAI Gateway daemon and server, with step-by-step troubleshooting guides.

from praisonaiagents import Agent

agent = Agent(
    name="Gateway Helper",
    instructions="Diagnose PraisonAI Gateway connectivity issues.",
)
agent.start("Gateway health check: is the daemon reachable?")

The user checks gateway status, reads logs, and restarts the daemon until channels respond again.

Common Issues

Port Already In Use / Two Gateways Running

Symptom: Gateway fails to start with a clear error message about port conflicts.

Error: Gateway port 8765 is already in use.

  Another gateway may be running (PID 8864).
  Stop it:  praisonai gateway stop
  Or use a different port:  GATEWAY_PORT=8766 praisonai gateway start

  Only ONE gateway process should poll each Telegram bot token.

Check PID lock and port status

praisonai gateway status

Look for lines like:

Gateway PID lock: Process 12345 running (127.0.0.1:8765)
Port 127.0.0.1:8765: In use

Stop the existing gateway gracefully

praisonai gateway stop

This sends SIGTERM and waits for graceful shutdown. The gateway drains active sessions for up to 10 seconds — in-flight turns and queued inbox messages are persisted before exit. Sessions that exceed the timeout are force-persisted with a SESSION_END event (had_pending_work, was_executing). Use --force to skip the drain. See Gateway Session Continuity.

If gateway is stuck, force stop

praisonai gateway stop --force

This sends SIGKILL directly (SIGTERM on Windows) for immediate termination.

If port is used by non-gateway process

# Find what's using the port
lsof -i :8765
# Linux alternative:
netstat -ano | findstr :8765

# Either kill that process or use a different port
GATEWAY_PORT=8766 praisonai gateway start
# Or:
praisonai gateway start --port 8766

Clean up stale lock file (if needed)

# Check the PID lock file
cat ~/.praisonai/gateway-127_0_0_1-8765.pid

# Remove stale lock manually (auto-cleaned on next start)
rm ~/.praisonai/gateway-127_0_0_1-8765.pid

Stale locks are automatically detected and removed, but you can clean them manually if needed.

Telegram Bot Goes Silent After Restart

Symptom: Gateway /health endpoint returns healthy, but Telegram messages aren’t received after a restart or crash. Cause: Two processes are polling the same Telegram bot token. Telegram delivers messages to only one poller, causing the bot to appear “silent” when the wrong process gets the messages.

Stop all gateway instances

praisonai gateway stop --force

This ensures all instances are terminated, even if PID files are corrupted.

Verify no processes are running

praisonai gateway status

Should show:

Gateway PID lock: No lock file found
Port 127.0.0.1:8765: Available

Start a single gateway instance

praisonai gateway start

Only start one instance to ensure exclusive bot token polling.

Test bot responsiveness

Send a message to your Telegram bot. It should respond normally now that only one process is polling the token.

Daemon Running But Gateway Unreachable

Symptom: praisonai gateway status shows Daemon service: Running (launchd) but Gateway not reachable at http://127.0.0.1:8765/health.

Verify daemon is actually running

praisonai gateway status --daemon-only

Look for Running status and process ID.

Check daemon logs for errors

praisonai gateway logs

# Or check raw log files:
# macOS: tail ~/.praisonai/logs/bot-stderr.log
# Linux: journalctl --user -u praisonai-bot

Look for Python tracebacks or port binding errors.

Check port and PID lock status

praisonai gateway status

This now reports both port usage and PID lock status directly. Look for:

Gateway PID lock: Process <pid> running or No lock file found
Port 127.0.0.1:8765: In use or Available

If another process is using the port, use praisonai gateway stop to stop an existing gateway, or choose a different port.

Verify PraisonAI version

praisonai --version

Upgrade to ≥ v4.6.23 if you see older versions - earlier versions had IndentationError bugs fixed in PR #1484.

Restart the daemon

# macOS
launchctl kickstart -k gui/$(id -u)/ai.praison.bot

# Linux  
systemctl --user restart praisonai-bot

# Or reinstall completely
praisonai onboard

Rapidly Growing Log Files

Symptom: ~/.praisonai/logs/bot-stderr.log grows to multiple MB per minute.

Check log file size

ls -lh ~/.praisonai/logs/bot-stderr.log

If growing rapidly (>1MB/min), this indicates a crash loop.

View recent errors

tail -50 ~/.praisonai/logs/bot-stderr.log

Look for repeated Python tracebacks, especially IndentationError.

Stop the daemon

# macOS
launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist

# Linux
systemctl --user stop praisonai-bot

Clear logs and restart

# Clear the log file
> ~/.praisonai/logs/bot-stderr.log

# Upgrade PraisonAI
pip install --upgrade praisonai

# Restart daemon
praisonai gateway install --start

Daemon Not Installed

Symptom: praisonai gateway status shows Daemon service: Not installed (systemd).

Run the onboarding wizard

praisonai onboard

This installs the daemon service for your platform.

Verify installation

praisonai gateway status --daemon-only

Should show Installed but not running or Running.

Start the service

# Manual start
praisonai gateway install --start

# Or check platform-specific commands
praisonai gateway status

HTTP 500 on Health Endpoint

Symptom: curl http://127.0.0.1:8765/health returns 500 Internal Server Error.

Check PraisonAI version

praisonai --version

Versions before v4.6.23 had AttributeError bugs in the health endpoint.

Upgrade PraisonAI

pip install --upgrade praisonai

Restart the gateway

praisonai gateway install --start

Test health endpoint

curl http://127.0.0.1:8765/health

Should return JSON with status, uptime, agents, sessions, clients, and channels.

Windows: ‘charmap’ codec error from Telegram bot replies

Symptom: Telegram users receive Error: 'charmap' codec can't encode character '⚠' in position N: character maps to <undefined> instead of the real error message. Root cause: Windows default console encoding is cp1252, not UTF-8. When agent exceptions contained warning symbols (⚠), emoji, or accented text, the error formatter crashed before the real error could be reported.

Verify your version contains the fix

python -c "from praisonai.gateway.unicode_utils import safe_error_message; print('OK')"

If this command runs without error, you have the fix from PR #1754.

Upgrade to a fixed version

pip install --upgrade praisonai

Upgrade to PraisonAI version that contains PR #1754 — Gateway and Telegram bot error handlers now sanitize exception text to ASCII-safe form for transport, while preserving full Unicode in logs.

Restart the gateway

praisonai gateway install --start

Test the fix

Users will now see clean error messages instead of charmap crashes:

Error: API quota exceeded. Check billing. (was: charmap crash hiding OpenAI 429)
Error: Rate limit exceeded. Try again later.
Error: Authentication failed. Check API key.
Error: Request timeout. Try again.

No workaround needed on supported versions. The previously recommended PYTHONUTF8=1 / PYTHONIOENCODING=utf-8 workaround is no longer required for the bot reply path (still useful for general console output).

Permission Denied Errors

Symptom: Daemon fails to start with permission errors in logs.

macOS
Linux
Windows

# Check LaunchAgent permissions
ls -la ~/Library/LaunchAgents/ai.praison.bot.plist

# Reload LaunchAgent
launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl load ~/Library/LaunchAgents/ai.praison.bot.plist

# Check systemd user service
systemctl --user status praisonai-bot

# Enable user lingering (allows services to run when not logged in)
sudo loginctl enable-linger $(whoami)

# Restart service
systemctl --user daemon-reload
systemctl --user restart praisonai-bot

# Run as administrator
# Check Task Scheduler for PraisonAI task
schtasks /Query /TN PraisonAIGateway

# Delete and recreate
praisonai gateway uninstall
praisonai gateway install

Clean Reinstall Process

When all else fails, perform a clean reinstall:

Stop and uninstall

praisonai gateway uninstall

Clear configuration

# Backup first if needed
rm -rf ~/.praisonai/logs
rm -rf ~/.praisonai/config

Upgrade PraisonAI

pip install --upgrade praisonai

Run onboarding

praisonai onboard

Follow the wizard to reinstall the daemon service.

Verify installation

praisonai gateway status

Should show daemon running and gateway reachable.

Reading the hello_error Envelope

When the gateway rejects a connection, it sends a structured hello_error frame before closing. Each (code, next_step) pair maps to a specific operator action.

`code`	`next_step`	Action
`rate_limited`	`wait_then_retry`	Wait `retry_after_seconds` then reconnect; if persistent, raise the rate limiter ceiling
`origin_not_allowed`	`do_not_retry`	Add the client origin to `allowed_origins` in `gateway.yaml`
`configuration_error`	`do_not_retry`	External bind without `allowed_origins` configured; bind to loopback or set the allowlist
`auth_required`	`reauthenticate`	Fetch a token (CLI or pairing flow) and reconnect
`protocol_unsupported`	`upgrade_client`	Update the client SDK to a newer version
`protocol_unsupported`	`downgrade_client`	Server is older than the client; pin the client to an older release or upgrade the server
`agent_not_found`	`do_not_retry`	Fix the `agent_id` in the hello frame

Example envelope received on rate-limit:

{
  "type": "hello_error",
  "code": "rate_limited",
  "message": "Too many connection attempts",
  "next_step": "wait_then_retry",
  "retry_after_seconds": 30,
  "next": "wait_then_retry"
}

See Gateway Handshake Protocol for the full field reference and ConnectRecoveryStep values.

Authentication Errors

GatewayStartupError: Cannot bind to 0.0.0.0 without an auth token

Symptom: Gateway fails to start when binding to external interfaces without authentication.

Use onboarding wizard

praisonai onboard

This automatically generates and saves a secure token.

Or set token manually

export GATEWAY_AUTH_TOKEN=$(openssl rand -hex 16)
praisonai gateway start --host 0.0.0.0

UIStartupError: Cannot bind to 0.0.0.0 with default admin/admin credentials

Symptom: Chainlit UI fails to start on external interface with default credentials.

Set custom credentials

export CHAINLIT_USERNAME=myuser
export CHAINLIT_PASSWORD=mypass
praisonai chat --host 0.0.0.0

Or allow defaults for demos (unsafe)

export PRAISONAI_ALLOW_DEFAULT_CREDS=1
praisonai chat --host 0.0.0.0

My gateway logs show `gw_****xxxx` instead of the full token

This is intentional for security — tokens are fingerprinted in logs to prevent exposure.

Retrieve full token from environment file

cat ~/.praisonai/.env | grep GATEWAY_AUTH_TOKEN

Or check environment variables

echo $GATEWAY_AUTH_TOKEN

Symptom: One auth method works but the other fails, even with the same token. Cause (pre-fix): HTTP/magic-link and WebSocket used different secret sources before PR #1744.

Upgrade PraisonAI

Upgrade to PraisonAI ≥ v4.6.47 (ships PR #1744). Config token now exports to env, unifying all auth paths.

Restart the gateway

praisonai gateway restart

Config token precedence is now enforced on restart.

Restart After Config Change

When you update bot configuration files, restart the daemon using these OS-specific commands (matching the onboard Done panel):

macOS
Linux
Windows

launchctl kickstart -k gui/$(id -u)/ai.praison.bot

systemctl --user restart praisonai-bot

schtasks /End /TN PraisonAIGateway && schtasks /Run /TN PraisonAIGateway

Diagnostic Commands

Quick commands for gathering diagnostic information:

# Full status check
praisonai gateway status

# Daemon-only check (for scripts)
praisonai gateway status --daemon-only

# Recent logs (last 50 lines)
praisonai gateway logs -n 50

# Check the PID lock file
cat ~/.praisonai/gateway-127_0_0_1-8765.pid

# Stop a running gateway
praisonai gateway stop
praisonai gateway stop --force

# Check port usage
lsof -i :8765

# Test health endpoint directly
curl -v http://127.0.0.1:8765/health

# Check PraisonAI version
praisonai --version

# Platform daemon status
# macOS: launchctl list | grep ai.praison
# Linux: systemctl --user status praisonai-bot
# Windows: schtasks /Query /TN PraisonAIGateway

Platform-Specific Notes

macOS
Linux
Windows

LaunchAgent Path: ~/Library/LaunchAgents/ai.praison.bot.plist Log Path: ~/.praisonai/logs/bot-stderr.log

# Manual management
launchctl load ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl unload ~/Library/LaunchAgents/ai.praison.bot.plist
launchctl kickstart -k gui/$(id -u)/ai.praison.bot

# Check if loaded
launchctl list | grep ai.praison

Service Path: ~/.config/systemd/user/praisonai-bot.service Logs: journalctl --user -u praisonai-bot

# Manual management
systemctl --user start praisonai-bot
systemctl --user stop praisonai-bot
systemctl --user restart praisonai-bot
systemctl --user enable praisonai-bot

# Check status
systemctl --user status praisonai-bot

Task Path: Task Scheduler → PraisonAIGateway Logs: Windows Event Log

# Manual management via Task Scheduler or:
schtasks /Run /TN PraisonAIGateway
schtasks /End /TN PraisonAIGateway

# Check status
schtasks /Query /TN PraisonAIGateway

Config reload did not apply

Symptom: You edited gateway.yaml or sent SIGHUP, but channels or agents still run the old configuration.

Check	What to look for
Log line	A successful reload logs a summary such as `reload applied: agents; restart[telegram]` — if absent, the watcher may not have fired or SIGHUP was ignored (Windows has no SIGHUP)
Drain window	Channel restarts wait up to `gateway.reload_drain_timeout` (falls back to `drain_timeout`) — a slow turn can delay the visible restart
Install	`pip install "praisonai[gateway]"` enables `watchdog`; without it, mtime polling (about 5 s) still reloads but more slowly

See Gateway Config Reload for file-watch vs SIGHUP paths and YAML keys.

Multi-Channel Troubleshooting

Common failure modes when using multiple bots on the same platform:

Symptom	Cause	Fix
`Token ${X} is used by multiple channels`	Two channels reference the same env var in `bot.yaml`	Give each channel its own env var and create a separate bot in @BotFather
`Same bot token value is used by multiple channels`	Two different env vars resolve to the same actual token	Generate a fresh token from @BotFather for the duplicate channel
`Channel '<x>' token '<env>' doesn't follow naming convention`	Custom env var name	Rename to `PLATFORM_<ROLE>_BOT_TOKEN` (e.g. `TELEGRAM_CFO_BOT_TOKEN`)
`Missing token: <env>`	Env var referenced in `bot.yaml` is unset	Add it to `~/.praisonai/.env`

Example Fix:

# Problem: Both channels use the same token
# Error: "Token TELEGRAM_BOT_TOKEN is used by multiple channels"

# Solution: Create unique tokens per role
# 1. Create new bot in @BotFather for CFO role
# 2. Add unique environment variable
export TELEGRAM_CFO_BOT_TOKEN="987654321:DEF..."

# 3. Update bot.yaml
channels:
  telegram_cfo:
    platform: telegram
    token: ${TELEGRAM_CFO_BOT_TOKEN}  # Unique token
    routes:
      default: cfo

Validation:

# Check multi-channel configuration
praisonai doctor multi_channel_tokens

# ✅ Expected output:
# "Multi-channel configuration looks good (N channels)"

Channel Supervision Issues

Channel goes silent but /health shows running: true

Symptom: Channel stops receiving messages but /health still reports "running": true. Cause: Pre-PR-2041 behaviour, or the health: block is missing from gateway.yaml — hung sockets are not detected until an exception is raised. Fix: Add the proactive health block:

gateway:
  health:
    interval: 300
    stale_after: 120
    enabled: true

See Proactive Health Monitoring.

Channel keeps restarting every 5 minutes

Symptom: Channel restarts on a regular interval visible in logs. Cause: stale_after too low for a quiet channel, or interval too aggressive for a slow remote API. Fix: Raise stale_after to e.g. 600, or lower max_restarts_per_hour so the cap kicks in and surfaces the issue:

gateway:
  health:
    stale_after: 600
    max_restarts_per_hour: 3

Channel Shows state: failed in /health

Symptom: When checking GET /health, a channel shows "state": "failed" with error details. Common Causes:

Telegram Conflict: Multiple bot instances using the same token
Invalid Credentials: Bot token revoked or incorrect
Permission Issues: Bot lacks required permissions

Investigation Steps:

Check Error Details

curl http://127.0.0.1:8765/health | jq '.channels.telegram.supervision'

Look for last_error and last_error_time fields.

Telegram Conflict Resolution

If error contains “Conflict: terminated by other getUpdates”:

# Stop any other bot instances using the same token
# Then force reconnect to reset state
praisonai gateway reconnect telegram

Credential Verification

For invalid token errors, verify your bot token:

# Test token directly
curl "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/getMe"

Channel Keeps Retrying Forever

Symptom: High total_recoveries count, constant retry attempts visible in logs. Investigation:

# Check supervision status
curl http://127.0.0.1:8765/health | jq '.channels'

# Look for:
# - High total_recoveries value
# - next_retry_at timestamp
# - last_error details

Resolution:

# Temporarily pause while investigating
praisonai gateway pause telegram

# Investigate root cause (network, DNS, etc.)
# Fix underlying issue

# Resume when ready
praisonai gateway resume telegram

Config Migration

When praisonai doctor --only gateway_config_migration reports Config can be migrated, your YAML uses a legacy shape that loads correctly but can be rewritten to the canonical format.

Detect migration opportunities

praisonai doctor --only gateway_config_migration

Example WARN output:

⚠ Gateway Config Migration [MEDIUM]
  Config can be migrated: 2 change(s)
  • telegram: allowed_users string → list migration available
  • Set telegram.group_policy to secure default 'mention_only'

Understand auto-normalisation

At load time, GatewayConfigSchema already normalises legacy formats — your bot runs without rewriting the file. Persist the canonical form when you want the on-disk YAML to match what the schema produces.

Apply canonical YAML

Legacy shape	Canonical form
Top-level `platform` + `token`	`channels:` dict with one entry
BotOS `platforms:` dict	`channels:` dict (same keys)
`allowed_users: "id1,id2"` (string)	`allowed_users: ["id1", "id2"]` (list)

See Gateway Config Migration for before/after YAML.

Environment Variables in Config

Gateway and bot configs support ${VAR} substitution in any string value. Resolution order:

Process environment variables
~/.praisonai/.env (loaded automatically; override path with PRAISONAI_ENV_FILE)

# Verify all referenced variables resolve
praisonai doctor --only gateway_env_substitution

Unset variables cause validation to FAIL at load time and in the gateway_env_substitution doctor check.

Best Practices

Use praisonai gateway status first

Always run praisonai gateway status before diving into logs - it shows port, PID, and daemon state in one command.

Stop before restart

Always use praisonai gateway stop before restarting - this drains active sessions and prevents data loss.

Use onboard for fresh installs

When in doubt, run praisonai onboard - it handles daemon setup, credentials, and port configuration automatically.

Check logs for crash loops

If bot-stderr.log grows rapidly, you have a crash loop - stop the daemon, upgrade PraisonAI, then restart.

Gateway CLI

Gateway command reference

Gateway Server

Gateway configuration and setup

​Common Issues

​Port Already In Use / Two Gateways Running

​Telegram Bot Goes Silent After Restart

​Daemon Running But Gateway Unreachable

​Rapidly Growing Log Files

​Daemon Not Installed

​HTTP 500 on Health Endpoint

​Windows: ‘charmap’ codec error from Telegram bot replies

​Permission Denied Errors

​Clean Reinstall Process

​Reading the hello_error Envelope

​Authentication Errors

​GatewayStartupError: Cannot bind to 0.0.0.0 without an auth token

​UIStartupError: Cannot bind to 0.0.0.0 with default admin/admin credentials

​My gateway logs show gw_****xxxx instead of the full token

​Magic-link login works in WebSocket but fails over HTTP (or vice versa)

​Restart After Config Change

​Diagnostic Commands

​Platform-Specific Notes

​Config reload did not apply

​Multi-Channel Troubleshooting

​Channel Supervision Issues

​Channel goes silent but /health shows running: true

​Channel keeps restarting every 5 minutes

​Channel Shows state: failed in /health

​Channel Keeps Retrying Forever

​Config Migration

​Environment Variables in Config

​Best Practices

​Related

Gateway CLI

Gateway Server

Common Issues

Port Already In Use / Two Gateways Running

Telegram Bot Goes Silent After Restart

Daemon Running But Gateway Unreachable

Rapidly Growing Log Files

Daemon Not Installed

HTTP 500 on Health Endpoint

Windows: ‘charmap’ codec error from Telegram bot replies

Permission Denied Errors

Clean Reinstall Process

Reading the hello_error Envelope

Authentication Errors

GatewayStartupError: Cannot bind to 0.0.0.0 without an auth token

UIStartupError: Cannot bind to 0.0.0.0 with default admin/admin credentials

My gateway logs show `gw_****xxxx` instead of the full token

Magic-link login works in WebSocket but fails over HTTP (or vice versa)

Restart After Config Change

Diagnostic Commands

Platform-Specific Notes

Config reload did not apply

Multi-Channel Troubleshooting

Channel Supervision Issues

Channel goes silent but /health shows running: true

Channel keeps restarting every 5 minutes

Channel Shows state: failed in /health

Channel Keeps Retrying Forever

Config Migration

Environment Variables in Config

Best Practices

Related