Caching

Cache LLM responses to reduce costs and latency with a single parameter

Caching stores LLM responses to avoid redundant API calls, cutting costs and speeding up repeat queries.

python

from praisonaiagents import Agent

agent = Agent(
    name="Assistant",
    instructions="Answer customer support questions.",
    caching=True,
)

agent.start("What is your return policy?")

The user asks the same support question again; a cache hit returns the prior answer without another LLM call.

Diagram

Quick Start

Level 1 — Bool (simplest)

Enable caching with a single flag — identical requests return instantly from cache.

python

from praisonaiagents import Agent

agent = Agent(
    name="Support Bot",
    instructions="You are a helpful assistant.",
    caching=True,
)

agent.start("Summarise the benefits of caching.")

Level 2 — String (preset)

Pass "prompt" to enable provider-level prompt caching alongside response caching.

python

from praisonaiagents import Agent

agent = Agent(
    name="Support Bot",
    instructions="You are a helpful assistant.",
    caching="prompt",
)

agent.start("Summarise the benefits of caching.")

Level 3 — Config class (full control)

Use CachingConfig to control response caching and provider-level prompt caching separately.

python

from praisonaiagents import Agent, CachingConfig

agent = Agent(
    name="Researcher",
    instructions="Answer research questions using memory.",
    model="anthropic/claude-3-5-sonnet",
    caching=CachingConfig(
        enabled=True,
        prompt_caching=True,
    ),
)

agent.start("What are the cost savings from prompt caching?")

How It Works

Diagram

Phase	What happens
1. Request received	Agent checks the cache using the request as key
2. Cache hit	Stored response returned instantly — no API cost
3. Cache miss	LLM processes the request, result stored for next time
4. Prompt caching	Provider caches stable prefix (system prompt, tools) across turns

Choosing Your Caching Level

Diagram

Configuration Options

Card

Full list of options, types, and defaults — CachingConfig

Option	Type	Default	Description
`enabled`	`bool`	`True`	Enable or disable response caching
`prompt_caching`	`bool \	None`	`None`	Enable provider-level prompt caching (Anthropic, Google)

Common Patterns

Pattern 1 — FAQ Bot with response caching

Identical user questions are answered from cache, saving API calls.

python

from praisonaiagents import Agent

agent = Agent(
    name="FAQ Bot",
    instructions="Answer frequently asked questions about our product.",
    caching=True,
)

response1 = agent.start("What are your business hours?")
response2 = agent.start("What are your business hours?")  # cache hit

Pattern 2 — Research agent with prompt caching

Large system prompts and tool definitions are cached at the provider level, reducing token costs on every turn.

python

from praisonaiagents import Agent, CachingConfig

agent = Agent(
    name="Research Assistant",
    instructions="You are an expert research assistant with access to a large knowledge base.",
    model="anthropic/claude-3-5-sonnet",
    memory=True,
    caching=CachingConfig(
        enabled=True,
        prompt_caching=True,
    ),
)

agent.start("Summarise last month's findings on climate data.")
agent.start("What trends stand out?")  # stable prefix cached

Pattern 3 — Disable caching for real-time data

Turn off caching when responses must always be fresh.

python

from praisonaiagents import Agent, CachingConfig

agent = Agent(
    name="Live Prices",
    instructions="Report real-time stock prices.",
    caching=CachingConfig(enabled=False),
)

agent.start("What is the current price of AAPL?")

Best Practices

Enable caching for predictable queries

Caching delivers the most value when users ask similar questions repeatedly — support bots, FAQ systems, and document Q&A are ideal candidates. Start with caching=True and measure cache-hit rates before adding configuration.

Use prompt caching for large system prompts

If your instructions string is long (policies, personas, tool lists), setting prompt_caching=True caches the stable prefix at the provider level. This reduces per-turn token cost significantly on Anthropic and Google models.

Disable caching for real-time or personalised responses

Pricing feeds, live dashboards, and highly personalised answers change on every request. Set CachingConfig(enabled=False) to bypass the cache entirely and always call the LLM.

Combine with memory for cross-session savings

Pairing caching=True with memory=True lets the agent recall past sessions while still serving cached responses for repeated questions — fewer API calls and consistent answers.

Card

Prompt Caching — provider-level caching for stable context

Card

Prompt Cache Optimization — tips to maximise cache hit rates

Caching

Quick Start

How It Works

Choosing Your Caching Level

Configuration Options

Common Patterns

Pattern 1 — FAQ Bot with response caching

Pattern 2 — Research agent with prompt caching

Pattern 3 — Disable caching for real-time data

Best Practices

Related