Gateway Concurrency Policy Protocol • AI Agent SDK

GatewayConcurrencyPolicyProtocol: Protocol for gateway-wide inbound admission decisions.

GatewayConcurrencyPolicyProtocol

Defined in the protocols module.

Protocol for gateway-wide inbound admission decisions.

Pure, import-free decision contract consumed by the wrapper’s run-dispatch path. The wrapper supplies the live aggregate counts (turns currently in flight and turns currently waiting) and the policy decides whether the next inbound turn may run now, must wait, or should be shed. Concrete enforcement (an `asyncio.Semaphore ceiling plus a bounded asyncio.Queue` with per-session fairness) lives in the wrapper, since it needs the running event loop and live session manager; this contract keeps the decision testable in isolation.

A config-driven default (:class:ConcurrencyLimitPolicy) is provided for the common “N concurrent runs, bounded wait queue, declared overflow” case.

On a shared/multi-tenant gateway the decision may additionally be scoped by a tenant/profile token (Issue #5168), mirroring the `scope axis of :class:RateLimitPolicyProtocol / :class:SpendBudgetPolicyProtocol — the wrapper passes the caller's scope` plus the live per-scope in-flight and queued counts so a single tenant cannot occupy every global slot. When no per-scope sub-limit is configured the scope arguments are ignored and the global-only decision is returned unchanged (backward compatible).

Properties

Methods

Source

Related Documentation