Persistence Overview

Database persistence for PraisonAI Agents

Persistence Overview

PraisonAI supports automatic database persistence for conversations, knowledge, and state management across 22 database backends.

Quick Start

Enable persistence in 2 lines:

from praisonaiagents import Agent

agent = Agent(
    name="Assistant",
    memory={
        "db": "postgresql://localhost/mydb",
        "session_id": "my-session"
    }
)
response = agent.start("Hello!")  # Auto-persists
print(response)

Installation

pip install "praisonaiagents[tools]"

Supported Backends

Backend Aliases

The persistence registry provides user-friendly aliases for common databases:

Conversation stores:

• neon, cockroachdb, xata → postgres
• asyncpg, postgres_async → async_postgres
• aiomysql, mysql_async → async_mysql
• sqlite_sync → sync_sqlite
• aiosqlite, sqlite_async → async_sqlite
• libsql → turso

Knowledge stores:

• chromadb → chroma
• mongodb_atlas, cosmos, azure_cosmos → Vector store variants
• llama_index, langchain_adapter → Framework adapters

State stores:

• motor, mongodb_async → async_mongodb (fully implements the StateStore contract as of PR #4287 — hash / TTL / keys operations are now available on the async backend). See Async MongoDB (motor).

Non-blocking prefix scan (Redis, Upstash)

PraisonAIDB.get_runs() and get_traces() iterate keys via a cursor-based SCAN on Redis / Upstash rather than the blocking KEYS command. Other backends implement scan_prefix() as keys(f"{prefix}"), so nothing ever falls back to an unbounded KEYS .

Architecture

┌─────────────────────────────────────────┐
│           praisonaiagents.Agent         │
│         (memory={db: "..."})            │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│     Memory/Knowledge/State Adapters     │
│         (DbAdapter Protocol)            │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│        Database Backends Layer          │
├─────────────┬─────────────┬─────────────┤
│ Conversation│  Knowledge  │    State    │
│   Store     │    Store    │    Store    │
└─────────────┴─────────────┴─────────────┘

Two Entry Points, One Owner

PraisonAIDB is the single owner of the “agent lifecycle → store” write hooks; PersistenceOrchestrator is a thin façade that delegates those hooks to it. To pass backend-specific kwargs to a single store, use the per-store option dicts — see Configuration Options.

Both entry points share the same store-write logic:

• PraisonAIDB (the live path) owns the lifecycle-hook and store-dispatch logic — on_message / aon_message, on_agent_end / aon_agent_end, and the sync/async dispatch decision. MemoryConfig(db=PraisonAIDB(...)) routes straight to it and is the recommended user-facing wiring.
• PersistenceOrchestrator delegates its store-write side to a PraisonAIDB instance built lazily from its own stores (internal PraisonAIDB._from_stores(...) plumbing), while keeping its unique surface: the resume-aware on_agent_start return contract, the LRU session cache, and the knowledge / state / context helpers.
• The orchestrator path (via wrap_agent_with_persistence, PersistentAgent, or create_persistent_session — see Orchestrator-Based Persistence) continues to work unchanged and now shares the same store-write logic under the hood.

Key Features

• Zero Config: SQLite works out of the box with memory=True
• Session Resume: praisonai session resume <id> is a first-class restore — it brings back chat history, model, and agent name, not just a transcript. Pass a follow-up prompt to continue immediately: praisonai session resume <id> "<prompt>".
• Runtime State Mirroring: lightweight per-turn artefacts for native↔plugin handoff (opt-in via SessionConfig.mirror_runtime_state)
• Lazy Loading: No performance impact until used
• CLI Support: praisonai persistence doctor/run/resume

Concurrency & Thread Safety

Persistence sync wrappers (get_session, add_message, get, set, delete, list_keys, clear, close, …) are safe to call from any context — plain sync scripts, worker threads, or code running inside a live event loop (FastAPI, Streamlit, background tasks). They route through a canonical async bridge, so you will not see RuntimeError: This event loop is already running anymore.

Async hooks offloading (PR #1829): Sync conversation stores plugged into async hooks are now automatically offloaded via asyncio.to_thread() rather than blocking the event loop. PraisonAIDB owns this dispatch (_call_store / _dispatch_async): it uses isinstance(store, AsyncConversationStore) to decide whether to await the store directly or wrap it in thread execution. As of PR #4394, PersistenceOrchestrator delegates its store writes through these same helpers rather than keeping a second copy, so the offloading behaviour is identical no matter which entry point you use.

Async state & knowledge ABCs (PR #5403): AsyncStateStore and AsyncKnowledgeStore are the async-sibling ABCs for state and knowledge persistence, mirroring AsyncConversationStore. They give third-party native-async backends a formal interface to subclass instead of implementing the sync ABC behind run_sync shims. Shipped backends are not yet rewired onto them. See Async State Store and Async Knowledge Store.

Async-store-safe session export / import (PR #5380)

PraisonAIDB.export_session / import_session now route through the same _call_store / _dispatch_async helpers used by every other lifecycle hook, so they work correctly on mode="async" conversation stores from a sync caller instead of leaking un-awaited coroutines. Two new async twins ship alongside them: aexport_session(session_id) and aimport_session(data).

Before this PR, export_session called the store’s get_session / get_messages directly, so on an AsyncConversationStore it hit AttributeError: 'coroutine' object has no attribute 'session_id' and raised a RuntimeWarning: coroutine ... was never awaited. The shipped praisonai persistence export / import CLI commands call these methods, so any user on mode="async" hit the bug on every export. Reference: PraisonAI PR #5380. See Async Conversation Store for a code example.

Non-lossy tool-call persistence (PR #3837)

Tool call results are persisted verbatim; the 1000-character truncation is gone.

PraisonAIDB.on_tool_call / aon_tool_call previously wrote str(result)[:1000], silently dropping any tool history longer than 1 KB. Both paths now delegate to a shared _serialize_tool_call(tool_name, args, result) helper that stores the full result. An agent that resumes a session sees the complete tool output it produced, so downstream reasoning that depended on the tail of a long tool result now works after resume.

No user action required — the behaviour is on by default, and existing rows written with truncation are unaffected (only new writes change). Reference: PraisonAI PR #3837.

As of PR #1763, you can also opt into the dedicated sync_sqlite backend (mode="sync") — it provides per-call connection locking (threading.RLock) for multi-agent scenarios where you’d rather not depend on the legacy sync wrappers.

The DbAdapter‘s lazy store initialisation is also race-free: the first thread to touch the adapter constructs the stores, and subsequent threads see the ready instance. Since PR #4394, PersistenceOrchestrator‘s message-write and session-end operations delegate to PraisonAIDB (through _call_store / _dispatch_async), so this race-free guarantee now covers both entry points.

The session cache inside PersistenceOrchestrator is also thread-safe as of PR #1609. Reads return a deepcopy of the cached ConversationSession, so multiple agents sharing one orchestrator can read/update sessions concurrently without races.

The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata) and PR #1724 (agent info, gateway info, clear). Two processes pointed at the same ~/.praisonai/sessions/ directory can now interleave add_message and set_agent_info / clear_session / set_gateway_info calls without losing messages. Reads reload from disk under lock on every get_chat_history / get_session call.

HierarchicalSessionStore inherits the JSON store’s reload-under-lock guarantees and additionally preserves parent_id, children_ids, and snapshots across all mutators (PR #1745). UI hosts that fork sessions or take snapshots are safe to run alongside an agent’s auto_save writer.

SqliteTranscriptStore (PraisonAI PR #3409) subclasses DefaultSessionStore but swaps the FileLock-guarded JSON files for one WAL SQLite row per session, wrapping each read-modify-write in a BEGIN IMMEDIATE transaction for cross-process append-safety. It is the default gateway transcript backend when session.persist: true — which is now the default (see Gateway Session Persistence and SQLite Transcript Store).

SQLite conversation store — clean shutdown across threads

SQLiteConversationStore opens one connection per calling thread (threading.local). As of PraisonAI #4537, store.close() drains every tracked connection, not just the caller’s — safe to call from a FastAPI lifespan hook, a shutdown supervisor, or atexit after fanning the store across a worker pool. Long-lived deployments (praisonai serve, gateway, async job pool, kanban) no longer leak an fd + SQLite memory arena per worker thread.

Under check_same_thread=True, connections owned by another thread can only be closed by that thread; those stay tracked (not silently dropped) so the owning thread’s later close() finishes the drain. No sqlite3.ProgrammingError reaches user code.

The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata), PR #1724 (agent info, gateway info, clear), and PR #1727 (LocalManagedAgent._persist_state). PRs #1759 and #1764 extended the same FileLock-guarded reload to the read path (get_chat_history, get_session, get_sessions_by_agent), so two processes pointed at the same ~/.praisonai/sessions/ directory observe each other’s writes without any stale-cache window.

Session Cache Size

PersistenceOrchestrator keeps recently-touched sessions in a bounded in-memory LRU (OrderedDict) so long-running servers/bots don’t grow forever when every request carries a fresh session_id.

# Raise the cap for a chat server that juggles many concurrent sessions
export PRAISONAI_SESSION_CACHE_MAX=8192

Shutting Down the Orchestrator

Close the orchestrator to release each store’s pooled connections when your app or script ends.

The shortest way an agent user reclaims stores is a context manager:

from praisonaiagents import Agent
from praisonai.persistence import PersistenceOrchestrator

# Sync agent / script: context manager
with PersistenceOrchestrator.from_env() as orch:
    agent = Agent(name="Assistant", memory={"orchestrator": orch})
    agent.start("Hello!")
# stores closed on exit, per-store errors isolated

Async agents use async with — this is why aclose() was added:

import asyncio
from praisonaiagents import Agent
from praisonai.persistence import PersistenceOrchestrator

async def main():
    async with PersistenceOrchestrator.from_env() as orch:
        agent = Agent(name="Assistant", memory={"orchestrator": orch})
        await agent.astart("Hello!")
    # stores closed on exit — never blocks the event loop

asyncio.run(main())

FastAPI lifespans call aclose() from the async shutdown handler. Before PR #5348, calling close() here raised "would block the running loop" from run_sync_or_offload:

from contextlib import asynccontextmanager
from fastapi import FastAPI
from praisonai.persistence import PersistenceOrchestrator

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.orch = PersistenceOrchestrator.from_env()
    yield
    await app.state.orch.aclose()  # non-blocking, isolates per-store errors

app = FastAPI(lifespan=lifespan)

Pick the shutdown path that matches how your process runs:

How it works:

• close() (sync) — closes each of conversation / knowledge / state in turn. Per-store errors are isolated so one failing store never strands the others’ pooled connections. A store’s attribute is reset to None only on a successful close, so a re-close is idempotent; a store that raised keeps its reference and can be retried by a later close() / aclose() instead of silently leaking its pooled connections.
• aclose() (async, new in PR #5348) — awaits each store’s native aclose when available, else offloads close via asyncio.to_thread. Same per-store error isolation and idempotence as close(). Safe to call from a live event loop — close() is not (it routes through run_sync_or_offload, which raises from a running loop).
• Async context manager (async with, new in PR #5348) — __aenter__ returns self, __aexit__ calls aclose(). Pair with FastAPI lifespans or async agent runners.
• Sync context manager (with) — already existed and continues to call close() on exit.

Schema Versioning (PR #1829)

New unified schema system: All SQL conversation stores now inherit from _SQLConversationStoreBase with consistent SCHEMA_VERSION = "1.0.0". This applies to PostgreSQL, MySQL (new), and all existing SQL backends uniformly.

The base class provides:

• Standardized table schemas for sessions and messages
• Dialect-specific type mappings (_id_type, _json_type, _float_type)
• Unified retry logic with max_retries and retry_delay parameters
• Automatic serverless database detection and exponential backoff
• Consistent table_prefix handling across all SQL stores

MySQL backend: The new MySQLConversationStore (from praisonai.persistence.conversation.mysql_new) includes PlanetScale auto-retry with exponential backoff for serverless cold-starts.

Schema Migration (PR #1597)

If you ran an async store before this release, either:

1. Pass table_prefix="praisonai_" explicitly to keep your old tables, or
2. Rename existing tables: ALTER TABLE praisonai_sessions RENAME TO praison_sessions; (and likewise for _messages).

New columns: Async session/message schemas now include state (sessions), and tool_calls + tool_call_id (messages). The store creates them automatically on first connect via CREATE TABLE IF NOT EXISTS. Existing tables on a pre-#1597 schema will not auto-migrate; add the columns with:

ALTER TABLE praison_sessions ADD COLUMN state TEXT;
ALTER TABLE praison_messages ADD COLUMN tool_calls TEXT;
ALTER TABLE praison_messages ADD COLUMN tool_call_id TEXT;

Custom Stores & Aliases

Register custom storage backends with the persistence registry, including optional aliases:

from praisonai.persistence import get_default_registry
from praisonaiagents import Agent

def my_custom_store(connection_string, **kwargs):
    return CustomStoreImplementation(connection_string, **kwargs)

# Register on the shared default so Agent(memory={"db": "my_store://..."}) sees it
registry = get_default_registry("conversation")
registry.register_store(
    "my_store",
    my_custom_store,
    aliases=("mine", "ms", "custom_db"),
)

# All of these now resolve to my_custom_store
agent = Agent(memory={"db": "my_store://config"})
agent = Agent(memory={"db": "mine://config"})       # alias
agent = Agent(memory={"db": "ms://config"})         # alias
agent = Agent(memory={"db": "custom_db://config"})  # alias

Next Steps

• Quickstart – Get started in 5 minutes
• Session Resume – Continue conversations
• Session Runtime State – Per-turn runtime artefacts for replay and handoff
• Async Conversation Store – AsyncConversationStore protocol for non-blocking persistence
• Async State Store – AsyncStateStore protocol for non-blocking key-value state
• Async Knowledge Store – AsyncKnowledgeStore protocol for non-blocking vector/RAG persistence
• CLI Reference – Command-line usage
• Backend Plugins – Custom storage backends