Tool Retry Policy

Automatically retry failed tool calls with exponential backoff

Tool retry automatically re-runs a failing tool with exponential backoff so transient errors don’t break your agent.

from praisonaiagents import Agent, tool, ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool
def web_search(query: str) -> str:
    """Search the web."""
    return f"Results for: {query}"

agent = Agent(
    name="researcher",
    instructions="Research topics on the web",
    tools=[web_search],
    tool_config=ToolConfig(retry_policy=RetryPolicy()),
)
agent.start("Find information about renewable energy")

The user asks for web research; RetryPolicy re-runs retryable tool errors with backoff before surfacing a failure.

graph LR
    Call[🔧 Tool Call] --> Try{🔄 Attempt}
    Try -->|Success| Done[✅ Return]
    Try -->|Retryable Error| Backoff[⏱️ Wait + Backoff]
    Backoff --> Try
    Try -->|Non-Retryable / Max Attempts| Fail[❌ Raise]

    classDef call fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef success fill:#10B981,stroke:#7C90A0,color:#fff
    classDef fail fill:#8B0000,stroke:#7C90A0,color:#fff

    class Call call
    class Try,Backoff process
    class Done success
    class Fail fail

Quick Start

1. Enable with defaults

   Enable retry for all tools with safe defaults:

   from praisonaiagents import Agent, ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   agent = Agent(
       name="researcher",
       instructions="Research topics on the web",
       tools=[web_search],
       tool_config=ToolConfig(retry_policy=RetryPolicy())  # 3 attempts, exponential backoff
   )
   
   agent.start("Find information about renewable energy")

2. Tune attempts and backoff

   Configure specific retry behavior:

   from praisonaiagents import Agent, ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   agent = Agent(
       name="api_agent", 
       instructions="Call external APIs",
       tools=[api_tool],
       tool_config=ToolConfig(
           retry_policy=RetryPolicy(
               max_attempts=5,
               retry_on={"timeout", "rate_limit", "connection_error"},
               backoff_factor=2.0,
               initial_delay_ms=1000,
               jitter=True,
           ),
       )
   )

3. Override per tool

   Different tools may need different retry strategies:

   from praisonaiagents import Agent, tool, ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   @tool(retry_policy=RetryPolicy(max_attempts=5, backoff_factor=3.0))
   def unreliable_api_call(query: str) -> str:
       """Call an unreliable external API."""
       # This tool gets aggressive retry policy
       return call_external_api(query)
   
   agent = Agent(
       name="mixed_agent",
       tools=[local_tool, unreliable_api_call],
       tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=2))  # Default for other tools
   )

How It Works

An error is tagged rate_limit when its message contains either "rate…limit" or "too many requests" — the latter catches HTTP 429 responses whose bodies use the standard phrasing without the word “limit”:

connection_error matches the substring "connect" in the message or the exception type name, so both ConnectionError and httpx-style ConnectError (even with an empty message) are retried.

# Both of these classify as rate_limit and are retried by default:
raise Exception("Rate limit exceeded")
raise Exception("HTTP 429: Too Many Requests")

Sync / async parity

RetryPolicy applies identically on the sync path (agent.execute_tool(...), agent.start(...)) and the async path (agent.execute_tool_async(...), agent.achat(...)) — including tools that raise exceptions.

A tool that exhausts its per-tool retries raises ToolExecutionError, which propagates directly out of agent.chat() / agent.start() (fix in PR #4252) and agent.achat() (fix in PR #6109). It is not routed through the LLM error classifier — so a tool message that looks like a rate limit (e.g. "429 too many requests") does not trigger an LLM backoff time.sleep or a redundant model call. The per-tool retry wrapper (max_attempts) is what governs tool-error retries; retry / RetryBackoffConfig are LLM-error-driven and never fire on a tool failure. See Error Handling → Tool Failure Behavior.

import asyncio
from praisonaiagents import Agent, ToolConfig
from praisonaiagents.tools import RetryPolicy

calls = []

def flaky():
    """Simulate a transient failure."""
    calls.append(1)
    raise RuntimeError("temporary glitch")

agent = Agent(
    instructions="Use the flaky tool.",
    tools=[flaky],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3, initial_delay_ms=0)),
)

# Sync path — retries 3 times
try: agent.execute_tool("flaky", {})
except Exception: pass

# Async path — also retries 3 times (parity restored in PR #4141)
try: asyncio.run(agent.execute_tool_async("flaky", {}))
except Exception: pass

The veto is now honoured on both paths too (PR #4299) — a declared-unsafe tool runs exactly once whether called sync or async:

import asyncio
from praisonaiagents import Agent, tool, ToolConfig
from praisonaiagents.tools import RetryPolicy

calls = []

@tool(restart_safe=False)
def charge_card(amount: str) -> str:
    """Charge a card — must never be silently re-driven."""
    calls.append(amount)
    raise RuntimeError("gateway 502")

agent = Agent(
    instructions="Charge the customer.",
    tools=[charge_card],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3, initial_delay_ms=0)),
)

# Sync path — runs once (was already correct)
try: agent.execute_tool("charge_card", {"amount": "100"})
except Exception: pass
assert len(calls) == 1

calls.clear()
# Async path — also runs once (fixed in PR #4299)
try: asyncio.run(agent.execute_tool_async("charge_card", {"amount": "100"}))
except Exception: pass
assert len(calls) == 1

Both paths share the same terminal rule for raised exceptions:

On the async path a raised exception is flattened into an error dict carrying the private _praison_retryable verdict, then the retry loop honours it and strips it. Once retries stop, the raised exception surfaces as ToolExecutionError (PR #6109) — a tool that returned an error dict is still handed to the model:

Your tool’s own retryable field is safe

Many HTTP-API wrappers return payloads shaped like {"error": "...", "retryable": true}. That field is the tool’s payload and is preserved untouched on the surfaced result. The framework’s own verdict lives on the private _praison_retryable key, which is stripped before the result reaches the model or the caller — the two never collide.

To make a returned error retry, tag its error type via RetryPolicy(retry_on=...). A word in the tool’s payload is not how retry is controlled.

from praisonaiagents import Agent, tool

@tool
def call_upstream(endpoint: str) -> dict:
    """Return a payload that includes its own retryable field."""
    return {"error": "upstream 503", "retryable": True}

agent = Agent(name="api", tools=[call_upstream])

result = agent.execute_tool("call_upstream", {"endpoint": "/status"})

assert result.get("retryable") is True        # tool payload survives
assert "_praison_retryable" not in result      # framework control key stripped

The same holds on the async path — the body still runs exactly once:

import asyncio
from praisonaiagents import Agent, tool

@tool
def call_upstream(endpoint: str) -> dict:
    """Return a payload that includes its own retryable field."""
    return {"error": "upstream 503", "retryable": True}

agent = Agent(name="api", tools=[call_upstream])

result = asyncio.run(agent.execute_tool_async("call_upstream", {"endpoint": "/status"}))

assert result.get("retryable") is True
assert "_praison_retryable" not in result

Returned error dicts run once

A tool that returns an error dict has already reached its own decision — the body ran to completion. Re-running it would duplicate side effects (charge money, send a message, write a row) for a decision the tool already made. The sync outer loop therefore surfaces such a dict exactly once, regardless of what fields it contains.

To make a returned error retry-eligible, register its error type via RetryPolicy(retry_on={...}) — the inner loop honours that classification, and the outer loop will not double-drive it.

Outer wall-clock timeouts retry only for idempotent tools

The outer loop’s own wall-clock timeout (_outer_timeout) is now gated on the tool’s idempotency: a timed-out tool is re-driven only if it is classified as idempotent. The timed-out body is still running in an orphaned worker thread (Future.cancel() is a no-op once started), so auto-retrying a mutating tool would duplicate its side effects. Idempotency comes from an explicit idempotent attribute on the tool, the global registry (with ToolConfig(allow_global_tools=True)), or the shared IDEMPOTENT_TOOLS / MUTATING_TOOLS name registry.

RetryPolicy is the single budget

The effective RetryPolicy is the one budget for tool-body runs. RetryPolicy.max_attempts genuinely caps how many times a tool runs, and the ExecutionConfig spelling (max_retry_limit / retry_*) is translated into that same policy in one place.

from praisonaiagents import Agent, tool, ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool
def charge_card(amount: int) -> str:
    """Charge a payment — must run at most once."""
    return f"Charged {amount}"

# Run a non-idempotent tool exactly once — no duplicate charges
agent = Agent(
    name="billing",
    tools=[charge_card],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=1)),
)
agent.start("Charge the customer $50")

How many times a failing tool body actually runs:

Precedence Ladder

When no retry_policy is set, the fallback is the policy translated from ExecutionConfig (max_retry_limit / retry_*), which defaults to RetryPolicy(max_attempts=3) when ExecutionConfig is untouched. Users who never set retry_policy= see no behaviour change — the translated policy reproduces max_retry_limit + 1 attempts and the same backoff.

retry_policy=None on @tool(...) means use the fallback — the tool-level slot is treated as empty, not as “no retries”. To disable retries for a specific tool, pass an explicit RetryPolicy(max_attempts=1) instead.

Tool-level (highest priority):

@tool(retry_policy=RetryPolicy(max_attempts=5))
def flaky_api():
    pass

Agent-level (medium priority):

from praisonaiagents import Agent, ToolConfig
from praisonaiagents.tools import RetryPolicy

agent = Agent(tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3)))

Translated ExecutionConfig fallback (lowest priority):

from praisonaiagents import Agent, ExecutionConfig

# max_retry_limit / retry_* are translated into the effective RetryPolicy.
# Untouched ExecutionConfig → RetryPolicy(max_attempts=3).
agent = Agent(tools=[tool], execution=ExecutionConfig(max_retry_limit=4))

Task-scoped tools

A tool passed to a task via tools_override (not registered on the agent) participates in the same idempotency lookup as an agent-level tool. Its restart_safe=False / idempotent=False declaration is honoured on both sync and async paths (PR #4299). Task-scoped tools shadow same-named agent tools when both are present, so a safer per-task override wins.

from praisonaiagents import Agent, tool

@tool(restart_safe=False)
def charge_card(amount: str) -> str:
    """Charge a customer — the per-task safe version, wins over any agent-level tool."""
    ...

agent = Agent(instructions="Billing.", tools=[])  # no agent-level tools
await agent.execute_tool_async(
    "charge_card", {"amount": "100"}, tools_override=[charge_card]
)  # runs exactly once

Choosing a Retry Policy

Configuration Options

Non-retryable error types (always short-circuit):

• approval_denied, permission_denied, approval_error, circuit_open, loop_blocked
• Python exceptions from tool code: ValueError, TypeError, AttributeError, LookupError (incl. KeyError / IndexError), FileNotFoundError, FileExistsError, IsADirectoryError, NotADirectoryError, PermissionError — terminal on both sync and async paths. A bare OSError is still retryable.
• Argument-binding errors (missing/extra parameters resolved from the schema) are terminal — they never retry regardless of policy.

Common Patterns

Per-tool override for unreliable API

from praisonaiagents import Agent, ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool(retry_policy=RetryPolicy(
    max_attempts=5,
    backoff_factor=3.0,
    jitter=True,
    retry_on={"timeout", "rate_limit", "connection_error"}
))
def external_weather_api(location: str) -> str:
    """Get weather from external API - known to be flaky."""
    return requests.get(f"https://api.weather.com/current?q={location}").text

agent = Agent(
    name="weather_bot",
    tools=[external_weather_api, local_calculation],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=2))  # Default for other tools
)

YAML configuration

agents:
  api_researcher:
    role: API Researcher
    instructions: "Research using external APIs"
    tools: [web_search, api_tool]
    tool_retry_policy:
      max_attempts: 4
      retry_on: [timeout, rate_limit]
      backoff_factor: 2.0
      jitter: true

CLI usage

praisonai \
  --tool-retry-attempts 5 \
  --tool-retry-delay 500 \
  --tool-retry-backoff 2.0 \
  --tool-retry-on "timeout,rate_limit" \
  "Research renewable energy trends"