Replay-Safe Retries

PraisonAI never silently retries a mid-response failure on a tool-calling turn

On a tool-calling turn, PraisonAI never silently retries a failure that happened after the request was dispatched — not from the retry driver, and not from the streaming → non-streaming fallback. The post-dispatch error surfaces to your code so you decide what to do.

Quick Start

1. Nothing to configure

   Replay-safety is default-on for tool-calling turns. No new config keys, no new Agent params.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="Bookings",
       instructions="Book flights on request.",
       tools=[book_flight],  # a side-effecting tool
   )
   
   # On a mid-response network failure you'll see:
   #   FailoverDecision(reason='provider_outcome_unknown', is_retryable=False)
   # instead of a silent retry that could book the flight twice.
   agent.start("Book me on the next flight to Tokyo")

2. Handle the surfaced outcome

   A surfaced provider_outcome_unknown means the provider may already have run your tool — you choose whether to verify state, ask the user, or retry deliberately.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="Bookings",
       instructions="Book flights on request.",
       tools=[book_flight],
   )
   
   try:
       agent.start("Book me on the next flight to Tokyo")
   except Exception as e:
       # reason='provider_outcome_unknown' — decide, don't blindly replay
       print("Provider outcome unknown; verifying booking state before retrying.")

The Invariant

On a tool-calling turn, a post-dispatch transient failure surfaces as provider_outcome_unknown instead of auto-retrying. Tool-less turns and pre-dispatch failures still auto-retry exactly as before.

Replaying a post-dispatch failure on a tool-calling turn could double-execute the tool — double billing, or a re-run of a side-effecting action. The gate stops that.

Streaming → Non-Streaming Fallback is Gated Too

When streaming fails, PraisonAI used to silently retry the same request with stream=False. On a tool-calling turn that is still a provider replay — the tool may already have run server-side mid-response. The fallback now respects the same replay-safety gate.

sequenceDiagram
    autonumber
    participant Agent
    participant LLM as LLM (streaming)
    participant Tool

    Agent->>LLM: request (stream=True, tools exposed)
    Note right of LLM: streaming started…<br/>bytes on the wire
    LLM--xAgent: read timeout mid-stream
    Note over Agent: provider may already have<br/>run a side-effecting tool call
    Agent-->>Agent: is_replay_unsafe_chain(err) == True<br/>+ side_effecting == True
    Agent-->>Agent: surface original error
    Note right of Agent: NO second request with stream=False

    classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef llm fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef tool fill:#F59E0B,stroke:#7C90A0,color:#fff

    class Agent agent
    class LLM llm
    class Tool tool

Both fallback sites are gated:

Tool-less turns and pre-dispatch failures (connect / DNS / TLS) still fall back to non-streaming exactly as before.

Agent example

from praisonaiagents import Agent

def book_flight(destination: str) -> str:
    """Side-effecting tool — charges a card, calls an API."""
    ...

agent = Agent(
    name="Bookings",
    instructions="Book flights on request.",
    tools=[book_flight],
)

try:
    # Streams by default; if the stream drops mid-response AFTER
    # the request reached the provider, you get the original error
    # instead of a silent stream=False retry that could book twice.
    agent.start("Book me on the next flight to Tokyo", stream=True)
except Exception as e:
    print("Streaming failed mid-response; verifying booking state before retrying.")

What Counts as Post-Dispatch (Replay-Unsafe)

Failures that happen once bytes are on the wire — the provider may already have produced output.

Message patterns treated as post-dispatch: read timeout, connection reset, reset by peer, peer closed, incomplete read, chunked, response ended prematurely, server disconnected.

What Counts as Pre-Dispatch (Replay-Safe)

Failures that prove the request never reached the provider — always safe to replay, retried as before.

Message patterns treated as pre-dispatch: connect timed out, connection refused, name resolution, getaddrinfo, handshake, certificate verify failed, cert verif.

The bare ssl and tls tokens are not pre-dispatch signals — ssl.SSLError is raised for both handshake failures and mid-response read/reset failures, so a transport label alone does not establish that the request failed before dispatch. Use the phase-specific handshake or certificate verify message to mark a TLS error as pre-dispatch.

Phase-Specific Precedence

A TLS handshake or certificate verification phase completes before any request byte is written, so an error naming either phase wins over a post-dispatch token.

Building Your Own Retry Wrapper

The classifier is public — import it if you write your own retry loop.

from praisonaiagents.llm.error_classifier import is_replay_unsafe

try:
    call_provider(...)
except Exception as e:
    if is_replay_unsafe(e) and turn_has_tools:
        raise  # surface — do not replay
    # otherwise: safe to retry

is_replay_unsafe(error) returns True when the failure signals the provider may already have processed the request.

Walk the Exception Chain

The streaming path re-raises wrapping the cause (raise Exception(...) from read_timeout), so the replay-unsafe signal lives on __cause__ / __context__ rather than the outermost exception. Use is_replay_unsafe_chain() to walk the chain:

from praisonaiagents.llm.error_classifier import is_replay_unsafe_chain

try:
    call_provider_stream(...)
except Exception as e:
    # Even if `e` is a wrapper, is_replay_unsafe_chain finds the
    # read-timeout underneath via __cause__/__context__.
    if is_replay_unsafe_chain(e) and turn_has_tools:
        raise  # surface — do not fall back to non-streaming or retry
    # otherwise: safe to retry / fall back

The chain is treated as unsafe if any link is unsafe — a pre-dispatch (safe) link does not override an unsafe one found elsewhere in the chain.

Common Patterns

A TLS-labelled read timeout or mid-response reset on a tool turn now surfaces instead of silently retrying.

import ssl
from praisonaiagents.llm.error_classifier import is_replay_unsafe

# Fixed in PR #5443 — previously returned False, retrying twice could re-run the tool.
assert is_replay_unsafe(Exception("TLS transport: read timeout")) is True
assert is_replay_unsafe(ssl.SSLError("SSL connection reset by peer")) is True

# Phase-specific pre-dispatch signals still auto-retry safely.
assert is_replay_unsafe(ssl.SSLError("TLS handshake failed")) is False
assert is_replay_unsafe(ssl.SSLError("certificate verify failed")) is False
assert is_replay_unsafe(ssl.SSLError(
    "connection reset by peer during TLS handshake"
)) is False  # phase signal wins over reset token

Best Practices

Related