LLM Network Retry Budget

Keep long runs alive through brief network outages — DNS and host-unreachable errors get their own retry budget

Short network outages such as VPN reconnects, Wi-Fi drops, or a laptop waking from sleep no longer abort your run. DNS failures and “no route to host” errors get their own retry budget, separate from max_retries.

python
from praisonaiagents import Agent

# Default is 300s; nothing to configure for most users.
agent = Agent(
    name="Researcher",
    instructions="Research topics on the web",
    llm="openai/gpt-4o-mini",
)
agent.start("Find recent papers on diffusion models")
# A 2-minute Wi-Fi drop no longer aborts the run.

The user runs an agent during a flaky network. The LLM call waits out the outage and continues when connectivity returns.

Diagram
graph LR
    Call[📞 LLM Call] --> Try{🔄 Attempt}
    Try -->|Success| Done[✅ Return]
    Try -->|Server error 5xx / 429| General[📊 General budget<br/>max_retries=3]
    Try -->|DNS / no route / unreachable| Network[🌐 Network budget<br/>network_retry_budget=300s]
    Network -->|Budget left| Wait[⏱️ 2s → 4s → … → 30s]
    Wait --> Try
    Network -->|Budget exhausted| General
    General -->|Attempts left| Back[⏱️ General backoff]
    Back --> Try
    General -->|Exhausted| Fail[🛑 Raise]

    classDef call fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef gate fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef proc fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef cfg fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef ok fill:#10B981,stroke:#7C90A0,color:#fff
    classDef err fill:#8B0000,stroke:#7C90A0,color:#fff

    class Call call
    class Try gate
    class General,Network cfg
    class Wait,Back proc
    class Done ok
    class Fail err

Quick Start

  1. Do nothing — you already have 300s of network retries
    python
    from praisonaiagents import Agent
    
    agent = Agent(
        name="Assistant",
        instructions="Answer carefully",
        llm="openai/gpt-4o-mini",
    )
    agent.start("Summarise the attached report.")
    # DNS blips / unreachable errors retry for up to 5 minutes.
  2. Raise the budget for an unattended CI run
    python
    from praisonaiagents import Agent
    
    agent = Agent(
        name="CI Agent",
        instructions="Triage failing tests",
        llm={"model": "gpt-4o", "network_retry_budget": 1800},  # 30 minutes
    )
    agent.start("Summarise today's failing tests")
  3. Turn it off and restore the old behaviour
    python
    from praisonaiagents import Agent
    
    agent = Agent(
        name="Strict Agent",
        instructions="Fail fast",
        llm={"model": "gpt-4o", "network_retry_budget": 0},
    )
    agent.start("Hello")

How It Works

Diagram
ProviderClassifierLLMAgentProviderClassifierLLMAgentcallrequest❌ DNS failure (gaierror)unreachable?Truestart network budget (300s)sleep 2sretry❌ ENETUNREACHsleep 4s (doubling, cap 30s)retry✅ 200 OKresult
  • Backoff schedule: 2s, 4s, 8s, 16s, 30s, 30s, … until the deadline.
  • Clamped final wait: the last sleep is cut to the time left, so it never overshoots the budget.
  • max_retries is not consumed while on this path.
  • Budget exhausted: once the deadline passes, further errors fall back to the general retry policy.
  • Failover profiles first: when an auth-profile failover manager is configured, this path is skipped so profile rotation can run first.
  • Sync and async calls behave the same.

What counts as “unreachable”

Detection sourceExamples
Exception classsocket.gaierror
errno on OSErrorENETUNREACH, EHOSTUNREACH, ENETDOWN
Message text (only when there is no HTTP status, or for APIConnectionError)Temporary failure in name resolution, name or service not known, nodename nor servname, getaddrinfo failed, ENOTFOUND, EAI_AGAIN, network is unreachable, no route to host, ENETUNREACH, EHOSTUNREACH
Exception chainUp to 8 links via __cause__ / __context__; for grouped exceptions, every child must be unreachable

Deliberately excluded: ConnectionRefusedError, ConnectionResetError (mid-response resets), and any error carrying an HTTP status code. The one exception is litellm’s APIConnectionError, which carries a synthetic 500 but is still a pre-send failure. These keep their current behaviour because they can mean the request reached a server.

Which budget handles an error?

Diagram

Yes

No

Yes

Yes

No

No

LLM call failed

Failover profiles
configured?

rotate_profile
then general retry

Unreachable?

Network budget
remaining?

Network retry
2s→30s backoff

Fall back to
general retry policy


Configuration Options

OptionWhere to setTypeDefaultDescription
network_retry_budgetllm={...} dict / LLM(...) kwargint seconds300Total wall-clock seconds to keep retrying an unreachable-class error. 0 disables it.
max_retriesllm={...} dict / LLM(...) kwargint3Unchanged. Used only for other retryable errors and after the network budget expires.
retry_delayllm={...} dict / LLM(...) kwargint60Unchanged. Fixed general retry delay, used only when set explicitly.

Common Patterns

  • CI / unattended runs: raise network_retry_budget to 1800 (30 min) to ride out a gateway or VPN blip.
  • Local LLMs (LM Studio, vLLM, Ollama): keep the default. An endpoint that can’t be resolved is still a real failure, and the budget caps the wait at 300 s.
  • Strict fail-fast pipelines: set network_retry_budget=0 to surface DNS failures immediately.
  • Monitoring: the retry callback fires on every network retry with attempt=<n>, max_attempts=None (the path is time-budgeted, not counted), and retry_in_seconds=<delay>.

Best Practices

Keep the default unless you need another value

300 s covers most VPN and Wi-Fi outages without leaving a run hanging for long.

Pair it with a per-attempt timeout

Each attempt is still bounded by the LLM timeout (default 600 s). See LLM Timeouts.

With failover profiles, this path is skipped

If auth-profile failover is configured, the retry loop rotates to another profile instead of waiting for the network.

Setting 0 is a strict opt-out

With network_retry_budget=0, a DNS blip counts against max_retries again and can abort the run.


Related