Reliability Guarantees

Safe defaults that keep long tool-using and multi-task runs going

Long tool-using conversations and mixed async task batches used to fail in three subtle ways. Those failure classes are now handled automatically — no configuration, no new imports, no API changes.

python
import asyncio
from praisonaiagents import Agent, Task, PraisonAIAgents

researcher = Agent(name="Researcher", instructions="Research the topic")
writer = Agent(name="Writer", instructions="Write based on the research")

research = Task(
    name="research",
    description="Research quantum computing",
    agent=researcher,
    async_execution=True,
)

article = Task(
    name="article",
    description="Write an article using the research",
    agent=writer,
    async_execution=True,
    context=[research],
)

asyncio.run(
    PraisonAIAgents(agents=[researcher, writer], tasks=[research, article]).astart()
)

The article task depends on research. Even when both run async in the same batch, the dependency is awaited first — article never sees an empty result.

Diagram

Reliability Guarantees

Malformed tool JSON

Reported to model as role=tool error

Window trim

Tool-call ↔ tool-result pair preserved

Async batch dependency

Dependency flushed before dependent

Async turn failure

chat_history rolled back to pre-turn snapshot

Tool returns non-serializable value

Fed to model as str value on every path incl. streaming, turn continues

Tool returns list-of-errors

Surfaced as role=tool error on every path

Async turn failure → achat()/astart() roll back the pre-turn chat_history snapshot, matching the sync path. (Since PR #4739 — see Async Agents.)

Quick Start

  1. Tool-using agent survives a malformed tool call

    If the model emits a tool call whose arguments JSON is truncated or malformed, the parse failure is reported back to the model as a role="tool" error and the run continues. Nothing to enable.

    python
    from praisonaiagents import Agent
    
    def search_web(query: str) -> str:
        """Search the web for a query."""
        return f"Results for {query}"
    
    agent = Agent(
        name="Researcher",
        instructions="Answer using the search tool",
        tools=[search_web],
    )
    
    agent.start("Find recent papers on quantum error correction")
  2. A Session that survives long tool-using conversations

    A windowed Session never emits a transcript that starts on an orphaned role="tool" message, so strict providers keep accepting it after many tool turns.

    python
    from praisonaiagents import Session
    
    session = Session(session_id="research_chat", user_id="user_1")
    agent = session.Agent(name="Assistant", role="Research helper")
    
    agent.start("Summarise today's findings")
    session.save_state({"topic": "quantum computing"})
  3. Dependent async tasks keep their context

    Two async_execution=True tasks in the same batch, where the second depends on the first via context=[...], run in the correct order automatically. The ordering guarantee applies on both entry points; use astart() if you want the tasks to actually run in parallel.

    python
    import asyncio
    from praisonaiagents import Agent, Task, PraisonAIAgents
    
    researcher = Agent(name="Researcher", instructions="Research the topic")
    writer = Agent(name="Writer", instructions="Write based on the research")
    
    research = Task(
        name="research",
        description="Research quantum computing",
        agent=researcher,
        async_execution=True,
    )
    
    article = Task(
        name="article",
        description="Write an article using the research",
        agent=writer,
        async_execution=True,
        context=[research],
    )
    
    asyncio.run(
        PraisonAIAgents(agents=[researcher, writer], tasks=[research, article]).astart()
    )
  4. A tool returning a non-serializable value doesn't crash the turn

    A tool returning a datetime, set, bytes, or custom object is fed to the model as str(value) on every path — the run continues instead of raising TypeError.

    python
    from datetime import datetime
    from praisonaiagents import Agent
    
    def current_time() -> datetime:
        """Return the current time."""
        return datetime.now()  # not JSON-serializable
    
    agent = Agent(
        name="Clock",
        instructions="Tell the user the current time using the tool.",
        tools=[current_time],
    )
    
    agent.start("What time is it?")

How It Works

Malformed tool-call JSON

Diagram
sequenceDiagram
    participant User
    participant Agent
    participant LLM

    User->>Agent: Request
    Agent->>LLM: Tool schema
    LLM-->>Agent: Tool call with truncated / invalid JSON arguments
    Note over Agent: Sentinel returned (NOT {})
    Agent->>LLM: role=tool error — "arguments could not be parsed; tool was NOT executed"
    LLM-->>Agent: Corrected tool call with valid JSON
    Agent-->>User: Answer

All seven execution paths — five in llm.py (sync sequential, async sequential, streaming batch, non-streaming batch, and async Responses-API loop) plus two in chat_mixin.py (async OpenAI-native tool-completion, sync OpenAI-native streaming) — now surface a role="tool" error message when a tool call’s arguments JSON cannot be parsed, and skip the dispatch instead of calling the tool with wrong arguments.

The exact message the model sees:

Error: arguments for tool '<function_name>' could not be parsed (the argument string
was invalid or truncated). The tool was NOT executed. Please re-emit the tool call
with complete, valid JSON arguments.

A parse failure returns a distinct internal sentinel that is kept separate from {}. Legitimately empty arguments ({}) still dispatch the tool as normal — only genuinely unparseable JSON surfaces as an error.

Execution pathCoverage
Sync sequential (llm.py)Reports role="tool" error, skips dispatch
Async sequential (llm.py)Reports role="tool" error, skips dispatch
Streaming batch (llm.py::get_response_stream)Reports role="tool" error, skips dispatch
Non-streaming batch (llm.py::get_response)Reports role="tool" error, skips dispatch
Async Responses-API loop (llm.py::get_response_async)Reports role="tool" error, skips dispatch
Async OpenAI-native tool-completion (chat_mixin.py::_achat_completion)Reports role="tool" error, skips dispatch (PR #5121)
Sync OpenAI-native streaming (chat_mixin.py::_start_stream_impl)Reports role="tool" error, skips dispatch (PR #5121)

The sentinel _TOOL_ARGUMENTS_PARSE_FAILED and the helpers tool_arguments_parse_failed / tool_parse_error_message are defined in agent/tool_execution.py and reused by llm/llm.py via a shared import, so the sentinel is a single object identity across both dispatch layers (an is check only works against the same object).

Uniform tool-result message

Diagram
LLMFormatterAgentLLMFormatterAgentOne formatter for every non-Ollama pathTool result (error, list-error, or non-serializable value)Same error sentence / str(value) fallbackrole=tool messageAnswer

Every non-Ollama path now routes tool results through one formatter, so the same tool value produces the same message regardless of which internal branch handled it. Previously five inline copies had drifted apart:

copy in llm.pyerror sentence usedlist-of-errors handledjson.dumps guarded
sync sequential"...inform the user." (shorter)yesno
sync batch (non-Ollama)"...could not be completed" (fuller)yesno
async secondary"...inform the user." (shorter)no — dumped as datano
async batch (non-Ollama)"...could not be completed" (fuller)yesno
streaming"...could not be completed" (fuller)yesyes

The uniform behaviour is:

  • A tool returning [{"error": "..."}] is always surfaced as an error to the model.
  • The error sentence is the same everywhere: Error: <msg>. Please inform the user that the operation could not be completed.
  • A non-JSON-serializable value (set, datetime, bytes, DataFrame, custom class) falls back to a stringified result instead of raising TypeError — on every path, including streaming (since PR #5233).
Execution pathCoverage
Sync sequentialSame message; list-error surfaced; non-serializable safe
Sync batch (non-Ollama)Same message; list-error surfaced; non-serializable safe
Async sequentialSame message; list-error now surfaced (previously dumped as data); non-serializable safe
Async batch (non-Ollama)Same message; list-error surfaced; non-serializable safe
StreamingNon-serializable safe (PR #5233); real execution failures still surfaced as {"error": ...}
Ollama (OllamaAdapter)Intentional divergence: role: "user", natural-language shape

The Ollama path keeps its natural-language role: "user" shape — the one genuine divergence, unchanged.

Before PR #5233 the streaming path did not have this parity. chat_completion_with_tools_stream wrapped tool execution and JSON serialization in one try with a single generic except Exception, so a tool that succeeded but returned a datetime, set, bytes, DataFrame, or custom object was reported to the model as `”Error executing function:

“. PR #5233 splits execution and serialization into two try blocks: the serialization block uses the same except (TypeError, ValueError) fallback the sync/async loops already used and stringifies into {“result”: “…”}. A tool that genuinely raises still surfaces as {“error”: str(e)}` — the split does not swallow real failures.

Tool-pair-safe window trim

Diagram
TranscriptStoreTranscriptStoreNaive slice could start on role=toolTrim to active windowPreserve tool-call ↔ tool-result pairsRetained tail never starts on an orphaned tool message

Windowed retention in DefaultSessionStore routes its slice through a tool-pair-safe guard, and the TruncateOptimizer / SlidingWindowOptimizer context strategies skip any leading orphaned role="tool" messages after trimming. Strict providers (OpenAI, Anthropic) reject a transcript that opens on a tool result whose originating assistant tool_calls message was trimmed away — that shape is no longer emitted.

Same-batch async dependency ordering

Diagram
Async Task B (depends on A)Async Task ARunnerAsync Task B (depends on A)Async Task ARunnerTaskB depends on A and A is still pendingQueue in batchFlush batch firstResultQueue with A’s context filled inCorrect output

arun_all_tasks tracks pending (task_id, coroutine) pairs. Before queuing an async task whose dependency is still pending in the current batch, it flushes the batch so the dependency finishes first. If the just-flushed dependency ended up failed, the failure cascades to the dependent instead of running it with missing upstream context. Both dependency edges are covered: task.context and workflow previous_tasks (from next_tasks).

Sub-agent transcripts can’t clobber a user’s session

session.close() on a session with session.Agent(...) sub-agents cannot overwrite an unrelated user’s session, even when a sub-agent’s name would sanitise to the same filename. Sub-agent transcripts are stored inside the parent’s own record under metadata["agent_histories"], so the old collision (PraisonAI issue #4120) is unrepresentable (PraisonAI PR #4126). See Session Store for the storage layout.


Crash-safe persistence

File-backed state is written atomically (PR #5357, extended by PRs #5871 and #6111): the value is serialised first, written to a temp file in the same directory, fsync-ed, then swapped in with os.replace. A crash, OOM kill, or a non-serialisable value cannot truncate or corrupt the previous good file — the last-known-good state always survives.

StateGuarantee
Permission rules & approvalsAtomic write — a downgraded deny rule after restart is impossible
Learned policy rules (policy/config.py)Atomic write with a unique temp file per write — safe for concurrent processes
PermissionAllowlistAtomic write; a corrupt file on load is quarantined and the allowlist starts empty (see below)
Workflow checkpointsAtomic write — a crash mid-write cannot force a restart from step 0
Plans & todosAtomic write
Cost reportsAtomic write
Async JSON storage (BaseJSONStorage.save_async)Serialise-first, then temp file + os.replace
RAG hierarchical-summary stateAtomic write — a failed or partial hierarchy save cannot truncate the previous hierarchy
Knowledge incremental-index stateAtomic write
Fast-context file indexAtomic write
User YAML/TOML configAtomic write; a symlinked config updates its target and the file mode is preserved
Code-editing writes (code_write_file, code_search_replace, code_apply_diff, multiedit)Atomic write (PR #6205) — a cancelled or timed-out edit leaves the original file untouched. See AI Code Editing → Crash-Safe Writes

Corrupt-allowlist quarantine (fail-closed)

If PermissionAllowlist.load(filepath) reads a file that is not valid JSON (a torn write from an older build, a hand-edit, disk damage), the file is moved aside to <filepath>.<pid>.<ns>.corrupt, a warning is logged, and an empty allowlist is returned. No tool is implicitly allowed, so the agent fails closed — review the .corrupt file and re-add any rules you still want.

python
from praisonaiagents.approval import PermissionAllowlist

allowlist = PermissionAllowlist.load("allow.json")  # corrupt file → [] and allow.json.*.corrupt
print(allowlist.list_tools())

Remote calls have deadlines

A stalled remote can no longer hang an agent indefinitely (PR #5871).

CallGuarantee
Remote skill git clone / rev-parse120 s deadline and GIT_TERMINAL_PROMPT=0; on timeout the cached copy is used
GitHub API requests in ContextAgent15 s socket timeout; the error reaches the caller’s existing fallback

Local git operations (snapshots, worktrees) have no fixed deadline, so large repositories are not cut off mid-operation.


Configuration Options

No configuration required — these are default, automatic behaviours. There are no new flags, no new config options, and no import changes. Upgrading is enough.


Best Practices

Prefer role=tool error surfaces over try/except around agent.start()

You do not need to wrap agent.start(...) in try/except to guard against malformed tool-call output. The parse failure is already reported back to the model as a role="tool" error, giving it a turn to self-correct. This guarantee applies uniformly across sync, async, streaming, and Responses-API code paths.

Set a Session retention window without worrying about tool-message boundaries

Bounded retention preserves tool-call ↔ tool-result pairs automatically. The retained tail never starts on an orphaned tool message, so you can cap history freely without provider-side “invalid conversation” errors.

Give dependent async tasks their context=[…]

Declare dependencies with context=[task] (or a workflow next_tasks edge). Batch ordering is preserved for you — the dependency is flushed before the dependent runs, so the dependent always sees real upstream output.

You don't need to make tool return values JSON-serializable for reliability

A tool returning datetime, set, bytes, or a custom object is fed to the model as str(value) and the turn continues. Serialization is still recommended for readability, but it is no longer a correctness requirement.


Related