Quick Start
With Configuration
Enable provider prompt caching explicitly — later turns hit the cached system block and history prefix:
How It Works
Chat history you pass into
agent.start(...) is not mutated. The SDK deep-copies the boundary message before adding a marker, so the same history dict can be reused across OpenAI, Gemini, and Anthropic in mixed pipelines.| Component | Caching behaviour |
|---|---|
| Memory search results | Deterministic order by content hash and timestamps |
| Tool schemas | Consistent ordering across invocations |
| Context structure | Stable system + memory before dynamic user input |
Cache breakpoints
On Anthropic models the SDK marks two stable regions with acache_control breakpoint: the system prompt (instructions, tools, memory prelude) and the stable conversation-history prefix. The two most recent messages stay uncached — they change every turn, and marking them would move the cache boundary and invalidate the prefix.
The history-prefix breakpoint lands on the last stable message. Each turn the prefix grows by one exchange, so older prefixes stay cache-hittable while the volatile tail (the two most recent messages) is re-sent uncached.
Anthropic allows up to 4 breakpoints per request. The SDK counts markers already present — the system block plus any markers left on caller-supplied history — and returns early before adding a new one if that count reaches 4, so the provider cap is never exceeded.
Provider behaviour
| Provider | Explicit cache_control markers | How caching happens |
|---|---|---|
Anthropic (anthropic/*, Bedrock Anthropic, Vertex Anthropic) | ✅ Yes — system block + history-prefix marker | Manual breakpoints, up to 4 per request (Anthropic cap) |
OpenAI (openai/*, Azure OpenAI) | ❌ No | Automatic prefix caching — SDK just keeps the prefix byte-stable |
Google (gemini/*, Vertex Gemini) | ❌ No | Automatic prefix caching — SDK just keeps the prefix byte-stable |
| Others (Ollama, local models) | ❌ No | No provider-side caching |
_explicit_cache_breakpoints() in llm.py — Anthropic gets explicit markers; OpenAI and Gemini rely on automatic prefix caching so no markers are emitted.
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
enabled | bool | True | Response caching via CachingConfig |
prompt_caching | Optional[bool] | None | Provider prompt-cache control |
max_items | int | 3 | Max items per memory category in build_context_for_task() |
include_in_output | Optional[bool] | None | Include memory in assembled prompt (manual assembly) |
Best Practices
Keep memory writes between turns minimal
Keep memory writes between turns minimal
Frequent memory updates change the prefix and reduce cache hits — batch updates at conversation boundaries.
Place stable content first
Place stable content first
Instructions, tools, and static memory before user messages and fresh data.
Use consistent parameters
Use consistent parameters
Varying
max_items between turns changes context and breaks cache effectiveness.Pair with cache optimisation
Pair with cache optimisation
See Prompt Cache Optimisation for tool sorting and the deterministic memory prefix.
Related
Prompt Cache Optimisation
Stable prefixes, tool sorting, and deterministic memory ordering
Advanced Memory
Memory configuration and search strategies

