Cipher’s MultiBackendHistoryProvider delivers high-availability, fault-tolerant, and durable conversation history storage for coding agents and AI assistants. By combining a fast primary database, a durable backup, and a Write-Ahead Log (WAL), Cipher ensures your chat history is never lost—even in the face of backend failures or network interruptions.

​
Why MultiBackend?

  • Resilience: If the primary database is unavailable, Cipher automatically falls back to the backup for reads.
  • Durability: All writes are first recorded in a WAL, guaranteeing no message loss even if the backup is temporarily down.
  • Consistency: The WAL ensures all messages are eventually persisted to the backup, providing strong durability guarantees.

​
Architecture

Cipher’s multi-backend storage is designed for real-world reliability:
  • Primary: Fast, main database for reads/writes (e.g., Redis, Postgres).
  • Backup: Durable, secondary database for redundancy (e.g., S3, SQLite, cloud DB).
  • WAL: In-memory or persistent log for write durability and async backup.

​
How It Works

  • saveMessage:
    1. Write to primary DB.
    2. Write to WAL (must succeed for durability).
    3. WAL is periodically flushed to backup DB asynchronously.
  • getHistory:
    • Read from primary DB.
    • If primary fails, fallback to backup DB.
  • clearHistory:
    • Clears all three: primary, backup, and WAL.
  • WAL Flush Worker:
    • Runs on an interval (default: 5s), flushing pending WAL entries to backup DB.

​
Enabling MultiBackend in Cipher

​
Environment Variables

Configure MultiBackend storage using these environment variables in your .env file: 
# Enable MultiBackend storage
CIPHER_MULTI_BACKEND=1

# Primary database (PostgreSQL example)
CIPHER_PG_URL=postgres://cipheruser:cipherpass@localhost:5432/cipherdb

# WAL flush interval in milliseconds (default: 5000)
CIPHER_WAL_FLUSH_INTERVAL=5000

# Storage configuration for backup
STORAGE_DATABASE_TYPE=sqlite
STORAGE_DATABASE_NAME=cipher.db
STORAGE_DATABASE_PATH=./data

​
Quick Setup

  1. Set CIPHER_MULTI_BACKEND=1 to enable
  2. Configure primary database (PostgreSQL recommended)
  3. Set backup storage (SQLite for local, PostgreSQL for production)

​
Supported Backends

  • Primary: PostgreSQL, SQLite, Redis, In-Memory
  • Backup: PostgreSQL, SQLite, Redis, In-Memory

​
Key Behavior

  • Writes: Primary must succeed, backup via async WAL
  • Reads: Primary first, auto-fallback to backup
  • WAL flush: Every 5 seconds (configurable)

​
API

The MultiBackendHistoryProvider implements:
  • getHistory(sessionId, limit?)
  • saveMessage(sessionId, message)
  • clearHistory(sessionId)
  • disconnect()
All methods are async.

​
WAL Implementation

  • The default WAL is in-memory (WALHistoryProvider), but you can extend it for persistent logging.
  • WAL entries are marked as flushed only after successful backup write.
  • If backup fails, WAL retains entries and retries on the next flush.

​
Best Practices

  • Use a fast, low-latency DB for primary (e.g., Redis, Postgres).
  • Use a highly durable, possibly slower DB for backup (e.g., S3, SQLite, cloud DB).
  • For production, consider a persistent WAL (not just in-memory) for maximum durability.

​
Extending

  • Implement your own IConversationHistoryProvider for custom storage.
  • Extend WALHistoryProvider for persistent WAL (e.g., file, database).
Cipher’s multi-backend storage ensures your conversation history is always safe, no matter what happens to your infrastructure.