Prompt Caching

Reduce cost and latency with deterministic, cacheable context for Anthropic and Google models

Reuse stable context across turns — pay full token cost once for instructions, tools, and unchanged memory.

from praisonaiagents import Agent, CachingConfig

agent = Agent(
    name="Researcher",
    instructions="Answer research questions using memory.",
    llm="anthropic/claude-sonnet-4-20250514",
    memory=True,
    caching=CachingConfig(prompt_caching=True),
)

agent.start("What did we learn last week about prompt caching?")
agent.start("What are the cost savings?")  # cache hit on stable prefix

The user continues the conversation; cached instructions and memory skip repeat token charges.

Quick Start

1. Simple Usage

   Deterministic ordering is automatic with memory:

   from praisonaiagents import Agent
   
   agent = Agent(
       name="Researcher",
       instructions="Answer research questions using memory.",
       memory=True,
   )
   
   agent.start("What did we learn last week about prompt caching?")

2. With Configuration

   Enable provider prompt caching explicitly — later turns hit the cached system block and history prefix:

   from praisonaiagents import Agent, CachingConfig
   
   agent = Agent(
       name="Researcher",
       instructions="Answer research questions using memory.",
       llm="anthropic/claude-sonnet-4-20250514",
       memory=True,
       caching=CachingConfig(prompt_caching=True),
   )
   
   agent.start("What did we learn last week?")
   agent.start("Summarise the top three findings.")  # cache hit on system + history prefix
   agent.start("Turn those into action items.")      # still hitting the same prefix

3. OpenAI — no explicit markers

   OpenAI applies prefix caching automatically; the SDK keeps the prefix byte-stable so no markers are emitted:

   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful analyst.",
       llm="openai/gpt-4o",
       memory=True,
   )
   
   agent.start("Summarise the report")
   agent.start("What changed since yesterday?")  # automatic prefix cache hit

How It Works

Cache breakpoints

On Anthropic models the SDK marks two stable regions with a cache_control breakpoint: the system prompt (instructions, tools, memory prelude) and the stable conversation-history prefix. The two most recent messages stay uncached — they change every turn, and marking them would move the cache boundary and invalidate the prefix.

The history-prefix breakpoint lands on the last stable message. Each turn the prefix grows by one exchange, so older prefixes stay cache-hittable while the volatile tail (the two most recent messages) is re-sent uncached.

sequenceDiagram
    participant User
    participant Agent
    participant LLM

    User->>Agent: First question
    Agent->>LLM: [system 🔒] [history prefix (empty) 🔒] [user "First question"]
    LLM-->>Agent: Answer

    User->>Agent: Second question
    Agent->>LLM: [system 🔒] [history prefix (Q1, A1) 🔒] [user "Second question"]
    Note over Agent,LLM: prefix cached from turn 1

    User->>Agent: Third question
    Agent->>LLM: [system 🔒] [history prefix (Q1, A1, Q2, A2) 🔒] [user "Third question"]
    Note over Agent,LLM: grows one turn at a time; each older prefix stays cache-hittable

Anthropic allows up to 4 breakpoints per request. The SDK counts markers already present — the system block plus any markers left on caller-supplied history — and returns early before adding a new one if that count reaches 4, so the provider cap is never exceeded.

Provider behaviour

Verified against _explicit_cache_breakpoints() in llm.py — Anthropic gets explicit markers; OpenAI and Gemini rely on automatic prefix caching so no markers are emitted.

Configuration Options

Best Practices

Related