Skip to main content

Persistence Overview

PraisonAI supports automatic database persistence for conversations, knowledge, and state management across 22 database backends.

Quick Start

Enable persistence in 2 lines:

Installation

Supported Backends

GCS state store (as of PraisonAI PR #5332): GCSStateStore now implements the full StateStore contract — keys, ttl, expire, hget, hset, hgetall, hdel. Earlier releases raised TypeError: Can't instantiate abstract class GCSStateStore before any GCS call. See Google Cloud Storage.
MemoryStateStore atomic writes (PR #5332): _save() is now atomic — the payload is JSON-serialised first (so a non-serialisable value fails before touching disk), then written to a sibling tempfile with fsync + os.replace. A crash, disk-full, or Ctrl+C mid-write no longer leaves a truncated JSON file that the next _load() would silently drop. Existing file permissions are preserved across the replace.

Backend Aliases

The persistence registry provides user-friendly aliases for common databases: Conversation stores:
  • neon, cockroachdb, xata → postgres
  • asyncpg, postgres_async → async_postgres
  • aiomysql, mysql_async → async_mysql
  • sqlite_sync → sync_sqlite
  • aiosqlite, sqlite_async → async_sqlite
  • libsql → turso
Knowledge stores:
  • chromadb → chroma
  • mongodb_atlas, cosmos, azure_cosmos → Vector store variants
  • llama_index, langchain_adapter → Framework adapters
State stores:
  • motor, mongodb_async → async_mongodb (fully implements the StateStore contract as of PR #4287 — hash / TTL / keys operations are now available on the async backend). See Async MongoDB (motor).

Non-blocking prefix scan (Redis, Upstash)

PraisonAIDB.get_runs() and get_traces() iterate keys via a cursor-based SCAN on Redis / Upstash rather than the blocking KEYS command. Other backends implement scan_prefix() as keys(f"{prefix}*"), so nothing ever falls back to an unbounded KEYS *.
Reference: PraisonAI PR #3812. The StateStore.scan_prefix(prefix) method is backend plumbing — self-hosters on Redis / Upstash get a “recent runs” read that never stalls other clients on the same instance. See Run History.

Architecture

Two Entry Points, One Owner

PraisonAIDB is the single owner of the “agent lifecycle → store” write hooks; PersistenceOrchestrator is a thin façade that delegates those hooks to it. To pass backend-specific kwargs to a single store, use the per-store option dicts — see Configuration Options. Both entry points share the same store-write logic:
  • PraisonAIDB (the live path) owns the lifecycle-hook and store-dispatch logic — on_message / aon_message, on_agent_end / aon_agent_end, and the sync/async dispatch decision. MemoryConfig(db=PraisonAIDB(...)) routes straight to it and is the recommended user-facing wiring.
  • PersistenceOrchestrator delegates its store-write side to a PraisonAIDB instance built lazily from its own stores (internal PraisonAIDB._from_stores(...) plumbing), while keeping its unique surface: the resume-aware on_agent_start return contract, the LRU session cache, and the knowledge / state / context helpers.
  • The orchestrator path (via wrap_agent_with_persistence, PersistentAgent, or create_persistent_session) continues to work unchanged and now shares the same store-write logic under the hood.
Reference: PraisonAI PR #4394 (fixes #4383). The public API is unchanged — this is an internal consolidation. _from_stores is internal plumbing, not a public API.

Key Features

  • Zero Config: SQLite works out of the box with memory=True
  • Session Resume: praisonai session resume <id> is a first-class restore — it brings back chat history, model, and agent name, not just a transcript. Pass a follow-up prompt to continue immediately: praisonai session resume <id> "<prompt>".
  • Runtime State Mirroring: lightweight per-turn artefacts for native↔plugin handoff (opt-in via SessionConfig.mirror_runtime_state)
  • Lazy Loading: No performance impact until used
  • CLI Support: praisonai persistence doctor/run/resume

Concurrency & Thread Safety

Persistence sync wrappers (get_session, add_message, get, set, delete, list_keys, clear, close, …) are safe to call from any context — plain sync scripts, worker threads, or code running inside a live event loop (FastAPI, Streamlit, background tasks). They route through a canonical async bridge, so you will not see RuntimeError: This event loop is already running anymore. Async hooks offloading (PR #1829): Sync conversation stores plugged into async hooks are now automatically offloaded via asyncio.to_thread() rather than blocking the event loop. PraisonAIDB owns this dispatch (_call_store / _dispatch_async): it uses isinstance(store, AsyncConversationStore) to decide whether to await the store directly or wrap it in thread execution. As of PR #4394, PersistenceOrchestrator delegates its store writes through these same helpers rather than keeping a second copy, so the offloading behaviour is identical no matter which entry point you use.
Async state-store hook naming (PR #4287): The DB adapter’s async lifecycle hooks now call async_set / async_get on the state store rather than the non-existent aset / aget names. Previously the async path silently fell back to sync methods off-loaded to a thread that re-entered the async bridge, pinning the event loop under load. No user action — async agents backed by any state store (Redis, MongoDB, DynamoDB, …) now stay non-blocking end-to-end.
Async RAG offload (PR #3837): PersistenceOrchestrator.aretrieve_knowledge() / aadd_knowledge() await native async knowledge stores and offload sync stores via asyncio.to_thread, so RAG calls from an async agent no longer block the event loop. See Async Knowledge Retrieval.
Async state & knowledge ABCs (PR #5403): AsyncStateStore and AsyncKnowledgeStore are the async-sibling ABCs for state and knowledge persistence, mirroring AsyncConversationStore. They give third-party native-async backends a formal interface to subclass instead of implementing the sync ABC behind run_sync shims. Shipped backends are not yet rewired onto them. See Async State Store and Async Knowledge Store.

Async-store-safe session export / import (PR #5380)

PraisonAIDB.export_session / import_session now route through the same _call_store / _dispatch_async helpers used by every other lifecycle hook, so they work correctly on mode="async" conversation stores from a sync caller instead of leaking un-awaited coroutines. Two new async twins ship alongside them: aexport_session(session_id) and aimport_session(data). Before this PR, export_session called the store’s get_session / get_messages directly, so on an AsyncConversationStore it hit AttributeError: 'coroutine' object has no attribute 'session_id' and raised a RuntimeWarning: coroutine ... was never awaited. The shipped praisonai persistence export / import CLI commands call these methods, so any user on mode="async" hit the bug on every export. Reference: PraisonAI PR #5380. See Async Conversation Store for a code example.

Non-lossy tool-call persistence (PR #3837)

Tool call results are persisted verbatim; the 1000-character truncation is gone. PraisonAIDB.on_tool_call / aon_tool_call previously wrote str(result)[:1000], silently dropping any tool history longer than 1 KB. Both paths now delegate to a shared _serialize_tool_call(tool_name, args, result) helper that stores the full result. An agent that resumes a session sees the complete tool output it produced, so downstream reasoning that depended on the tail of a long tool result now works after resume. No user action required — the behaviour is on by default, and existing rows written with truncation are unaffected (only new writes change). Reference: PraisonAI PR #3837. As of PR #1763, you can also opt into the dedicated sync_sqlite backend (mode="sync") — it provides per-call connection locking (threading.RLock) for multi-agent scenarios where you’d rather not depend on the legacy sync wrappers. The DbAdapter’s lazy store initialisation is also race-free: the first thread to touch the adapter constructs the stores, and subsequent threads see the ready instance. Since PR #4394, PersistenceOrchestrator’s message-write and session-end operations delegate to PraisonAIDB (through _call_store / _dispatch_async), so this race-free guarantee now covers both entry points. The session cache inside PersistenceOrchestrator is also thread-safe as of PR #1609. Reads return a deepcopy of the cached ConversationSession, so multiple agents sharing one orchestrator can read/update sessions concurrently without races. The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata) and PR #1724 (agent info, gateway info, clear). Two processes pointed at the same ~/.praisonai/sessions/ directory can now interleave add_message and set_agent_info / clear_session / set_gateway_info calls without losing messages. Reads reload from disk under lock on every get_chat_history / get_session call. HierarchicalSessionStore inherits the JSON store’s reload-under-lock guarantees and additionally preserves parent_id, children_ids, and snapshots across all mutators (PR #1745). UI hosts that fork sessions or take snapshots are safe to run alongside an agent’s auto_save writer. SqliteTranscriptStore (PraisonAI PR #3409) subclasses DefaultSessionStore but swaps the FileLock-guarded JSON files for one WAL SQLite row per session, wrapping each read-modify-write in a BEGIN IMMEDIATE transaction for cross-process append-safety. It is the default gateway transcript backend when session.persist: true — which is now the default (see Gateway Session Persistence and SQLite Transcript Store).

SQLite conversation store — clean shutdown across threads

SQLiteConversationStore opens one connection per calling thread (threading.local). As of PraisonAI #4537, store.close() drains every tracked connection, not just the caller’s — safe to call from a FastAPI lifespan hook, a shutdown supervisor, or atexit after fanning the store across a worker pool. Long-lived deployments (praisonai serve, gateway, async job pool, kanban) no longer leak an fd + SQLite memory arena per worker thread. Under check_same_thread=True, connections owned by another thread can only be closed by that thread; those stay tracked (not silently dropped) so the owning thread’s later close() finishes the drain. No sqlite3.ProgrammingError reaches user code.
These improvements were added in PraisonAI PR #1466, #1609, #1709, #1724, and #1727 to ensure robust multi-threaded operation in production environments.
The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata), PR #1724 (agent info, gateway info, clear), and PR #1727 (LocalManagedAgent._persist_state). PRs #1759 and #1764 extended the same FileLock-guarded reload to the read path (get_chat_history, get_session, get_sessions_by_agent), so two processes pointed at the same ~/.praisonai/sessions/ directory observe each other’s writes without any stale-cache window.

Session Cache Size

PersistenceOrchestrator keeps recently-touched sessions in a bounded in-memory LRU (OrderedDict) so long-running servers/bots don’t grow forever when every request carries a fresh session_id.
Reference: PraisonAI PR #3792. Eviction is least-recently-used — reads and writes mark recency, and the oldest session is dropped once the cap is reached.

Shutting Down the Orchestrator

Close the orchestrator to release each store’s pooled connections when your app or script ends. The shortest way an agent user reclaims stores is a context manager:
Async agents use async with — this is why aclose() was added:
FastAPI lifespans call aclose() from the async shutdown handler. Before PR #5348, calling close() here raised "would block the running loop" from run_sync_or_offload:
Pick the shutdown path that matches how your process runs: How it works:
  • close() (sync) — closes each of conversation / knowledge / state in turn. Per-store errors are isolated so one failing store never strands the others’ pooled connections. A store’s attribute is reset to None only on a successful close, so a re-close is idempotent; a store that raised keeps its reference and can be retried by a later close() / aclose() instead of silently leaking its pooled connections.
  • aclose() (async, new in PR #5348) — awaits each store’s native aclose when available, else offloads close via asyncio.to_thread. Same per-store error isolation and idempotence as close(). Safe to call from a live event loop — close() is not (it routes through run_sync_or_offload, which raises from a running loop).
  • Async context manager (async with, new in PR #5348) — __aenter__ returns self, __aexit__ calls aclose(). Pair with FastAPI lifespans or async agent runners.
  • Sync context manager (with) — already existed and continues to call close() on exit.
Reference: PraisonAI PR #5348. The aclose() method and async context manager are new; both share close()’s per-store isolation. See Thread Safety for the sync-from-any-context wrappers these close paths build on.

Schema Versioning (PR #1829)

New unified schema system: All SQL conversation stores now inherit from _SQLConversationStoreBase with consistent SCHEMA_VERSION = "1.0.0". This applies to PostgreSQL, MySQL (new), and all existing SQL backends uniformly. The base class provides:
  • Standardized table schemas for sessions and messages
  • Dialect-specific type mappings (_id_type, _json_type, _float_type)
  • Unified retry logic with max_retries and retry_delay parameters
  • Automatic serverless database detection and exponential backoff
  • Consistent table_prefix handling across all SQL stores
MySQL backend: The new MySQLConversationStore (from praisonai.persistence.conversation.mysql_new) includes PlanetScale auto-retry with exponential backoff for serverless cold-starts.

Schema Migration (PR #1597)

Breaking change: All async conversation stores (async_sqlite, async_postgres, async_mysql) now default to table_prefix="praison_" (previously "praisonai_"). Sync stores were already on praison_.
If you ran an async store before this release, either:
  1. Pass table_prefix="praisonai_" explicitly to keep your old tables, or
  2. Rename existing tables: ALTER TABLE praisonai_sessions RENAME TO praison_sessions; (and likewise for _messages).
New columns: Async session/message schemas now include state (sessions), and tool_calls + tool_call_id (messages). The store creates them automatically on first connect via CREATE TABLE IF NOT EXISTS. Existing tables on a pre-#1597 schema will not auto-migrate; add the columns with:
The default (zero-config) JSON session store now carries the same tool_calls / tool_call_id fields automatically as of PraisonAI PR #3099 — no migration needed. The file format stays backward compatible: old text-only session files load unchanged, and the new fields are only written when a message actually has them. See Session Resume.
DB adapters can now persist the same structured tool transcript. PraisonAI PR #5082 adds an optional capability protocol ToolTurnDbAdapterProtocol (with on_assistant_message / on_tool_message) that adapters implement to persist assistant tool_calls and role="tool" results faithfully. The built-in praisonai.db() adapter implements it out of the box — resumed DB sessions replay the same tool-turn shape the JSON store already did. Legacy adapters without the new hooks keep the prior text-only behaviour, so nothing breaks on upgrade. See Custom DB Adapter.

Custom Stores & Aliases

Register custom storage backends with the persistence registry, including optional aliases:
The shared default registry is what agents use at runtime. Constructing a fresh StoreRegistry(...) registers backends only on that isolated instance — use get_default_registry("conversation") when you want Agent(memory=...) to see your backend immediately.
The register_store() method is the canonical API. The register() method is preserved as a backward-compatible alias.

Next Steps