Prompt Cache Optimization

Stable, cache-friendly prompts across memory, tools, and tasks for lower cost and latency

Turn on memory on a supported model — the SDK sorts tools and lays out context so providers can cache the stable prefix.

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="openai/gpt-4o",
    memory=True,
)

agent.start("Summarise the latest report")
# Turn 2+ reuses cached system prompt + memory prefix
agent.start("What changed since yesterday?")

The user chats across turns; stable prefixes stay cached for lower cost.

How It Works

Quick Start

1. Simple Usage

   Automatic on supported models when memory is enabled:

   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="openai/gpt-4o",
       memory=True,
   )
   
   agent.start("Summarise the latest report")

2. With Configuration

   Make prompt caching explicit:

   from praisonaiagents import Agent, CachingConfig
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="anthropic/claude-sonnet-4-20250514",
       memory=True,
       caching=CachingConfig(prompt_caching=True),
   )
   
   agent.start("Analyse the data trends")

How It Works

On OpenAI and Gemini the SDK emits no explicit cache markers. Those providers apply prefix caching automatically as long as the request prefix is byte-identical across turns. The optimisations on this page — deterministic tool order, stable memory layout, no wall-clock in the system prompt — are what make automatic caching work.

Check model support:

from praisonaiagents.llm.model_capabilities import supports_prompt_caching

supports_prompt_caching("openai/gpt-4o")  # True
supports_prompt_caching("ollama/llama3")  # False

Detecting Cache Invalidation

When a turn’s prefix changes and the provider cache misses, the gateway can tell you — via a hook event and a metric, automatically when metrics are on.

from praisonaiagents import Agent
from praisonaiagents.agent import prompt_prefix_signature

agent = Agent(
    instructions="You are a helpful assistant",
    llm="openai/gpt-4o",
    memory=True,
)

# Fingerprint whatever the provider will cache
sig_before = prompt_prefix_signature(agent)
# ... swap a tool or edit instructions ...
sig_after = prompt_prefix_signature(agent)
assert sig_before != sig_after  # cache miss next turn

Fingerprinted: model identity (from agent.llm, or .model, or str()), the sorted set of tool schema fingerprints (name + description + JSON-serialised params for dict tools; name + docstring for callables), and the agent.instructions string. Tool reordering keeps the signature stable.

Excluded (to avoid false positives): arrival time, routing facts, memory/RAG recall, and per-turn user text.

The PROMPT_PREFIX_INVALIDATED hook listener receives:

event_name = HookEvent.PROMPT_PREFIX_INVALIDATED
session_id = "<storage_key>"
agent_name = "<agent.agent_name or agent.name or 'bot'>"
timestamp  = "<time.time() as str>"
cwd        = "<os.getcwd()>"
extra      = {
    "old_sig": "<sha256 hex of previous turn's prefix>",
    "new_sig": "<sha256 hex of this turn's prefix>",
    "reason":  "tools/model",
}

Configuration Options

Set via Agent(caching=True) or Agent(caching=CachingConfig(...)).

The include_cache_boundary flag on build_cache_optimized_context() controls whether the returned dict contains the marker; it does not affect what is sent on the wire.

Best Practices

Related