Skip to main content
Reuse stable context across turns — pay full token cost once for instructions, tools, and unchanged memory.
The user continues the conversation; cached instructions and memory skip repeat token charges.

Quick Start

1

Simple Usage

Deterministic ordering is automatic with memory:
2

With Configuration

Enable provider prompt caching explicitly — later turns hit the cached system block and history prefix:
3

OpenAI — no explicit markers

OpenAI applies prefix caching automatically; the SDK keeps the prefix byte-stable so no markers are emitted:

How It Works

Chat history you pass into agent.start(...) is not mutated. The SDK deep-copies the boundary message before adding a marker, so the same history dict can be reused across OpenAI, Gemini, and Anthropic in mixed pipelines.
ComponentCaching behaviour
Memory search resultsDeterministic order by content hash and timestamps
Tool schemasConsistent ordering across invocations
Context structureStable system + memory before dynamic user input

Cache breakpoints

On Anthropic models the SDK marks two stable regions with a cache_control breakpoint: the system prompt (instructions, tools, memory prelude) and the stable conversation-history prefix. The two most recent messages stay uncached — they change every turn, and marking them would move the cache boundary and invalidate the prefix. The history-prefix breakpoint lands on the last stable message. Each turn the prefix grows by one exchange, so older prefixes stay cache-hittable while the volatile tail (the two most recent messages) is re-sent uncached. Anthropic allows up to 4 breakpoints per request. The SDK counts markers already present — the system block plus any markers left on caller-supplied history — and returns early before adding a new one if that count reaches 4, so the provider cap is never exceeded.

Provider behaviour

ProviderExplicit cache_control markersHow caching happens
Anthropic (anthropic/*, Bedrock Anthropic, Vertex Anthropic)✅ Yes — system block + history-prefix markerManual breakpoints, up to 4 per request (Anthropic cap)
OpenAI (openai/*, Azure OpenAI)❌ NoAutomatic prefix caching — SDK just keeps the prefix byte-stable
Google (gemini/*, Vertex Gemini)❌ NoAutomatic prefix caching — SDK just keeps the prefix byte-stable
Others (Ollama, local models)❌ NoNo provider-side caching
Verified against _explicit_cache_breakpoints() in llm.py — Anthropic gets explicit markers; OpenAI and Gemini rely on automatic prefix caching so no markers are emitted.

Configuration Options

OptionTypeDefaultDescription
enabledboolTrueResponse caching via CachingConfig
prompt_cachingOptional[bool]NoneProvider prompt-cache control
max_itemsint3Max items per memory category in build_context_for_task()
include_in_outputOptional[bool]NoneInclude memory in assembled prompt (manual assembly)

Best Practices

Frequent memory updates change the prefix and reduce cache hits — batch updates at conversation boundaries.
Instructions, tools, and static memory before user messages and fresh data.
Varying max_items between turns changes context and breaks cache effectiveness.
See Prompt Cache Optimisation for tool sorting and the deterministic memory prefix.

Prompt Cache Optimisation

Stable prefixes, tool sorting, and deterministic memory ordering

Advanced Memory

Memory configuration and search strategies