Gateway Flow Control

Bounded inboxes and slow-consumer disconnect keep one misbehaving client from degrading the gateway

Flow control bounds per-session inboxes and disconnects slow consumers so a single misbehaving client can’t degrade the whole gateway.

from praisonaiagents import Agent, GatewayConfig

agent = Agent(
    name="assistant",
    instructions="You help users with tasks",
)
# Flow control is enabled by default — 256-message inbox, 1 MB buffer ceiling

The user sends bursts of messages; per-client inbox and outbound byte limits queue, drop, or close connections before the agent is overwhelmed.

Quick Start

1. Defaults work out of the box

   Flow control is enabled by default — no configuration needed. Every session gets a 256-message inbox and the gateway disconnects clients whose transport buffer exceeds 1 MB or whose outbound frame queue exceeds 1000 frames.

   from praisonaiagents import Agent, GatewayConfig
   
   agent = Agent(
       name="assistant",
       instructions="You help users with tasks"
   )

2. Tune the limits

   Adjust thresholds to match your workload.

   from praisonaiagents import Agent, GatewayConfig, SessionConfig
   
   config = GatewayConfig(
       max_buffered_bytes=4 * 1024 * 1024,
       max_queued_frames=2000,            # NEW: frame-count ceiling
       session_config=SessionConfig(
           max_inbox=512,
       )
   )
   
   agent = Agent(
       name="assistant",
       instructions="You help users with tasks"
   )

How It Works

Two checks protect the gateway from overload:

A frame is rejected when either ceiling would be exceeded, or a single frame already exceeds max_buffered_bytes on its own.

Configuration Options

max_inbox is set on SessionConfig; max_buffered_bytes and max_queued_frames are set on GatewayConfig.

from praisonaiagents import GatewayConfig, SessionConfig

config = GatewayConfig(
    max_buffered_bytes=1024 * 1024,
    max_queued_frames=1000,
    session_config=SessionConfig(
        max_inbox=256,
    )
)

These options are also available in gateway.yaml:

gateway:
  max_buffered_bytes: 1048576
  max_queued_frames: 1000      # frame-count ceiling
  session_config:
    max_inbox: 256

Client-Side Error Contract

When the inbox is full, the gateway sends this JSON frame before rejecting the message:

{
  "type": "error",
  "code": "inbox_full",
  "message": "Message queue is full. Please wait for current messages to be processed."
}

When either transport ceiling is exceeded for a critical event, the gateway closes the WebSocket with:

• Code: 1013 (Try Again Later)
• Reason: "slow_consumer" (the value of GatewayCloseCode.SLOW_CONSUMER)

Use the typed enum from Python clients:

from praisonaiagents.gateway import GatewayCloseCode

if close.reason == GatewayCloseCode.SLOW_CONSUMER.value:
    # reconnect and resume from the last known message
    ...

Clients should reconnect and resume from the last known message. The gateway preserves session state during the resume_window (default 24 h), so conversation context is not lost.

Common Patterns

Chat-heavy workloads — Tighten the inbox to shed load early and fail fast:

from praisonaiagents import GatewayConfig, SessionConfig

config = GatewayConfig(
    max_buffered_bytes=512 * 1024,
    session_config=SessionConfig(max_inbox=64)
)

Long-running tools — Loosen both limits to avoid spurious disconnections during slow AI responses:

from praisonaiagents import GatewayConfig, SessionConfig

config = GatewayConfig(
    max_buffered_bytes=8 * 1024 * 1024,
    session_config=SessionConfig(max_inbox=1024)
)

Stream-heavy workloads — Lower max_queued_frames to shed load on backpressure earlier:

from praisonaiagents import GatewayConfig

config = GatewayConfig(
    max_buffered_bytes=4 * 1024 * 1024,
    max_queued_frames=200,           # eager eviction for chatty streams
)

Local development — Disable slow-consumer checks so network hiccups don’t interrupt debugging:

from praisonaiagents import GatewayConfig, SessionConfig

config = GatewayConfig(
    max_buffered_bytes=0,
    session_config=SessionConfig(max_inbox=0)
)

Best Practices

Related