Hardened SQLite Storage

WAL-with-safe-fallback SQLite connector with opt-in corruption detection and repair — keeps agent memory intact on Docker, NFS, and Kubernetes volumes

Every SQLite store PraisonAI ships uses a hardened connection factory that keeps agent memory and session data intact even on network and container filesystems, without any user configuration.

Diagram

🤖 Agent

🧠 Memory / Session

🔌 sqlite_connect

✅ WAL on normal FS

⚠️ DELETE on NFS/FUSE

✅ WAL preserved on existing WAL DB

An agent running on Docker Desktop, NFS, or a Kubernetes volume keeps its memory without a single line of configuration.

Quick Start

  1. Simple Usage

    Nothing to configure — hardening is automatic for any SQLite-backed store.

    python
    from praisonaiagents import Agent
    
    agent = Agent(
        name="Assistant",
        instructions="You are helpful.",
        memory=True,
    )
    agent.start("Remember my favourite colour is blue")
  2. Custom Store

    Build your own SQLite-backed store with the hardened factory.

    python
    from praisonaiagents.storage import sqlite_connect
    
    conn = sqlite_connect("./state.db")  # WAL where supported, DELETE where not
    conn.execute("CREATE TABLE IF NOT EXISTS t (x INTEGER)")

How It Works

The factory tries WAL first, then probes the on-disk header before ever falling back — so a database another connection wrote in WAL is never downgraded.

Diagram
.db Filesqlite_connectStore.db Filesqlite_connectStorealt[header already reports WAL][fresh or non-WAL]alt[WAL accepted][WAL rejected (NFS/FUSE)]connect(path)PRAGMA journal_mode=WAL“wal”WAL connectionread 100-byte headerWAL headerleave as WAL (protect peers)PRAGMA journal_mode=DELETEDELETE connection
FilesystemJournal mode chosenNotes
ext4/APFS/NTFS (local)WALConcurrent readers with a writer
Docker Desktop (gRPC-FUSE)DELETE (fresh DB) or WAL (existing WAL header)Silent fallback, no corruption
NFS/SMB home dirDELETEByte-range locking honoured
Fly.io / K8s network volumeDELETENo -shm sidecar issues
:memory:WALNo fallback needed
Pre-existing WAL file on hostile FSWAL (preserved)Never downgraded — peer commits protected

Integrity Guard

When guard=True, the connector runs a cheap quick_check on open and — on a malformed file — forensically backs up the bad bytes and attempts a bounded, least-destructive repair before returning a usable connection. SqliteSessionStore and SqliteTranscriptStore opt in by default so a corrupt sessions.db no longer takes the durable path down.

Diagram

ok

locked/busy

corrupt

verified intact

failed

🔌 Open

🔍 quick_check

✅ Use connection

💾 Forensic backup
.corrupt-<epoch>

🛠 Online-backup
repair

⚛ atomic os.replace

📦 Quarantine
fresh empty DB

Corruption on open

A malformed database is detected, backed up, and repaired in place — or quarantined so a fresh empty DB opens instead of crashing.

Diagram
sessions.dbsqlite_connect(guard=True)SqliteTranscriptStoresessions.dbsqlite_connect(guard=True)SqliteTranscriptStorealt[snapshot intact][snapshot still bad]connect(guard=True)PRAGMA quick_check(1)“malformed”copy → sessions.db.corrupt-1717082000000 (+sidecars)online-backup into sessions.db.repair-tmp-<pid>-<id>PRAGMA quick_check on snapshotos.replace(snapshot, sessions.db)os.remove(sessions.db) (+ stale -wal/-shm)usable connection

The guard treats a transient database is locked / database is busy on the probe as healthy-but-busy — a peer momentarily holding the file is left exactly as-is, with no backup and no repair.

What lands on disk

FileWhen it appearsPurposeRetention
<file>.corrupt-<epoch_ms> (+-wal/-shm)On every corruption event with backup capacityPreserves malformed bytes for forensicsNewest 3 per DB (older pruned)
<file>.repair-tmp-<pid>-<id>During a repair attempt onlyIn-flight snapshot for online-backup rebuildDeleted on success or failure
<file>.repair (sidecar ledger)After a repair attemptCounts attempts so a permanently-bad file is not re-repaired on every bootCleared when a repair succeeds

The forensic backup fails closed on low disk — it needs 2 × file_size + 1 MiB free, so a corruption event never itself fills the disk. Backups are retention-capped at _MAX_FORENSIC_BACKUPS = 3 per file, and repairs are bounded at _MAX_REPAIR_ATTEMPTS = 3 per file (tracked in the .repair sidecar) so a boot-loop cannot churn a permanently-bad file. These are internal capacity constants — reason about them, but don’t override them.

Public helpers

sqlite_quick_check is a cheap PRAGMA quick_check(1) wrapper that returns False on any malformed-image / I/O error and re-raises transient lock/busy errors so callers can tell “locked” (leave alone) apart from “corrupt” (back up + repair).

python
from praisonaiagents.storage import sqlite_quick_check
import sqlite3

conn = sqlite3.connect("./state.db")
if not sqlite_quick_check(conn):
    print("This file failed the structural probe — consider guard=True on connect.")

Should I turn guard on?

Diagram

Yes — durable path,
can’t afford a crash

Building a scratch/tmp store
where a bad file is OK to lose

Low-disk environment where
forensic copies could hurt

Am I building a durable store
whose corruption would crash boot?

guard=True
+repair=True +backup=True

guard=False
default

guard=True, backup=True,
repair=False (still fails closed
on low disk)


Configuration Options

The sqlite_connect factory forwards unknown keyword arguments to sqlite3.connect.

OptionTypeDefaultDescription
path`str \os.PathLike`requiredDB path or ":memory:"
check_same_threadboolFalseForwarded to sqlite3.connect; core stores share connections under their own lock
isolation_levelOptional[str]"" (sqlite3 default)None for autocommit
busy_timeout_msint5000PRAGMA busy_timeout in milliseconds
synchronousstr"FULL"PRAGMA synchronous; pass "NORMAL" for throughput under WAL
guardboolFalseRun quick_check on open and, on a malformed file, forensically back it up and attempt a bounded least-destructive repair before connecting. Off by default so existing callers are unchanged.
repairboolTrueAttempt an online-backup repair on corruption. Only takes effect when guard=True. Bounded by a sidecar attempt-ledger (<file>.repair).
backupboolTrueTake a retention-capped, disk-space-aware forensic backup of the malformed file before repair. Only takes effect when guard=True.

Common Patterns

Docker / Kubernetes / NFS deployments need no special handling — the default Agent(memory=True) already survives them.

python
from praisonaiagents import Agent

agent = Agent(name="Assistant", instructions="You are helpful.", memory=True)

Build a custom SQLite-backed store on top of the same hardening.

python
from praisonaiagents.storage import sqlite_connect

conn = sqlite_connect("./custom.db", busy_timeout_ms=10000)

Favour throughput over the last-transaction durability window with synchronous="NORMAL" (safe under WAL against process crashes).

python
from praisonaiagents.storage import sqlite_connect

conn = sqlite_connect("./fast.db", synchronous="NORMAL")

Choosing a synchronous Level

Diagram

Yes

Throughput matters,
process-crash safe is enough

Need survival
through OS/power crash?

synchronous=FULL
default

synchronous=NORMAL
WAL only


Scope


Best Practices

Let the factory choose the journal mode

Do not run PRAGMA journal_mode=WAL yourself — sqlite_connect picks WAL where it works and DELETE where it does not.

Keep synchronous=FULL unless you measured a bottleneck

FULL survives an OS or power crash. Switch to NORMAL only after profiling shows a durability-vs-throughput trade-off worth making.

Use sqlite_connect for custom stores

For any SQLite-backed store you build, call sqlite_connect instead of sqlite3.connect to inherit the fallback and durability defaults.

Expect DELETE on hostile filesystems

On Docker Desktop, NFS, or network volumes the on-disk journal mode may be DELETE. This is correct behaviour, not a regression, and does not change any API.

Let the guard classify a locked file — it won't back it up

A transient database is locked / database is busy on the quick_check probe is treated as healthy-but-busy, and the file is left untouched. Only a genuinely malformed image triggers a backup or repair.

Turn guard=True on for any custom durable store you build with sqlite_connect

This is what the shipped session/transcript stores do; anything else that must open reliably should follow.


Related