Quick Start
Stream progress from a tool
Add an
on_progress=None parameter. The executor detects it and wires updates through automatically — no other changes needed.How It Works
The executor inspects each tool’s signature, forwards progress it emits, and records a deferred handle without blocking.| Behaviour | Guarantee |
|---|---|
| Backward compatible | on_progress is only passed to tools whose signature accepts it. |
| Native async | async def tools are awaited automatically. |
| Non-blocking defer | A defer(...) return surfaces the note to the model immediately. |
| Safe channels | A failing on_progress callback is swallowed; the tool keeps running. |
| Source stamped | Each ToolProgress is tagged with its tool_call_id and function_name. |
Which return type do I choose?
Pick the simplest option that fits how long your tool runs.Configuration Options
ToolProgress describes a single incremental update a tool emits while working.
| Option | Type | Default | Description |
|---|---|---|---|
text | str | required | Human-readable progress message. |
id | Optional[str] | None | Stable id so a channel can edit a single draft message. |
replace | bool | True | If True, replaces the prior draft; else appends. |
tool_call_id | Optional[str] | None | Source tool call id — stamped by the executor. |
function_name | Optional[str] | None | Source tool name — stamped by the executor. |
DeferredToolResult is a handle a tool returns when it kicks off background work.
| Option | Type | Default | Description |
|---|---|---|---|
handle_id | str | required | Identifier for the background job, used to resolve later. |
note | str | "started; will resolve later" | Message shown to the model now. |
defer() factory builds a DeferredToolResult; handle_id defaults to a generated uuid.uuid4().hex when omitted.
| Option | Type | Default | Description |
|---|---|---|---|
note | str | "started; will resolve later" | Message surfaced to the model immediately. |
handle_id | Optional[str] | None | Job id; auto-generated when omitted. |
ToolResult carries these extra fields alongside result.
| Field / property | Type | Description |
|---|---|---|
progress | List[ToolProgress] | All progress updates the tool emitted during this call. |
deferred | Optional[DeferredToolResult] | Non-None when the tool returned a defer(...) handle. |
is_deferred | bool | True iff deferred is not None. |
structured_error | Optional[Dict] | {"error": True, "type": ..., "message": ..., "tool": ...} on failure, else None. |
Common Patterns
Async tool with progress
Anasync def tool is awaited natively — no asyncio.run wrapper needed.
Deferred job resolved by handle
Returndefer(...) now, then resolve the job later by its handle_id.
Structured error inspection
Readstructured_error to get the error type and message instead of a flattened string.
Best Practices
Backward compatible by default
Backward compatible by default
Tools without an
on_progress parameter are called the old way. No change is needed unless you want progress — the executor auto-detects the parameter via inspect.signature.Never let the UI break your tool
Never let the UI break your tool
The executor swallows exceptions raised by an
on_progress callback and keeps the tool running. A broken UI channel never kills a tool call.Stamp your own source only when proxying
Stamp your own source only when proxying
The executor stamps
tool_call_id and function_name automatically. Only set them yourself when you relay updates from another tool.Prefer defer() over blocking on external systems
Prefer defer() over blocking on external systems
Job queues, video renders, and batch pipelines belong behind a
defer() handle so the turn continues while the work runs.Related
Tool Progress Streaming
Event/sink-based progress via
emit_tool_progress()Async Tool Safety
Rules for async tools inside sync flows
Structured LLM Errors
How structured errors surface to the model
Custom Tools
Building your own tools

