Knowledge memory

Knowledge memory is the primary storage system for factual information, technical concepts, and domain-specific knowledge in Cipher. It automatically extracts and stores programming knowledge, code patterns, and implementation details from conversations, making them available for future reference through semantic search.

Configuration

Required Environment Variables

# OpenAI API Key (Required for embeddings)
OPENAI_API_KEY=sk-proj-your-openai-key-here

# Vector Store Type (Required)
VECTOR_STORE_TYPE=qdrant  # qdrant, milvus, in-memory

Vector Storage Configuration

Qdrant Configuration

# Connection Options (choose one)
VECTOR_STORE_URL=http://localhost:6333           # Complete URL
# OR
VECTOR_STORE_HOST=localhost                      # Host + Port
VECTOR_STORE_PORT=6333

# Optional Settings
VECTOR_STORE_API_KEY=your-qdrant-api-key        # For Qdrant Cloud
VECTOR_STORE_COLLECTION=knowledge_memory         # Collection name
VECTOR_STORE_DIMENSION=1536                     # Embedding dimensions
VECTOR_STORE_DISTANCE=Cosine                    # Distance metric
VECTOR_STORE_ON_DISK=false                      # Store vectors on disk

Milvus Configuration

# Connection Options (choose one)
VECTOR_STORE_URL=http://localhost:19530          # Complete URL
# OR
VECTOR_STORE_HOST=localhost                      # Host + Port
VECTOR_STORE_PORT=19530

# Authentication (Zilliz Cloud)
VECTOR_STORE_USERNAME=your-username              # Optional
VECTOR_STORE_PASSWORD=your-password              # Optional
VECTOR_STORE_API_KEY=your-api-token             # Optional

# Collection Settings
VECTOR_STORE_COLLECTION=knowledge_memory         # Collection name
VECTOR_STORE_DIMENSION=1536                     # Embedding dimensions

Development Configuration

# In-Memory (Development/Testing)
VECTOR_STORE_TYPE=in-memory
VECTOR_STORE_COLLECTION=knowledge_memory
VECTOR_STORE_DIMENSION=1536
VECTOR_STORE_MAX_VECTORS=10000                  # Memory limit

Knowledge Memory Tools

Cipher provides two main tools for knowledge memory management:

Extract & Operate Memory Tool

cipher_extract_and_operate_memory This tool automatically processes conversations and determines what knowledge should be stored, updated, or removed. It uses intelligent content analysis and LLM-powered decision making to extract meaningful programming knowledge while avoiding redundant or low-quality information. Key Features:

Automatic knowledge extraction from conversations
Intelligent similarity detection to prevent duplicates
LLM-powered decision making for memory operations (ADD/UPDATE/DELETE)
Rich metadata tagging and categorization
Code pattern recognition and preservation

Input Parameters

{
  interaction: string | string[],           // Conversation text(s)
  existingMemories?: Array<{               // Optional existing memories
    id: string,
    text: string,
    metadata?: Record<string, any>
  }>,
  context?: {                              // Optional context
    sessionId?: string,
    userId?: string,
    projectId?: string,
    conversationTopic?: string
  },
  options?: {                              // Configuration options
    similarityThreshold?: number,           // 0.0-1.0 (default: 0.8)
    maxSimilarResults?: number,            // 1-20 (default: 5)
    useLLMDecisions?: boolean,             // Enable LLM decisions
    enableDeleteOperations?: boolean       // Allow DELETE operations
  },
  memoryMetadata?: {                       // Custom metadata
    projectId?: string,
    userId?: string,
    environment?: string
  }
}

Output Structure

{
  success: boolean,
  extraction: {
    extracted: number,                     // Number of facts extracted
    skipped: number,                       // Non-significant content skipped
    facts: Array<{
      id: string,
      preview: string,
      metadata: {
        hasCodeBlock: boolean,
        hasCommand: boolean,
        length: number
      }
    }>
  },
  memory: Array<{
    id: number,
    text: string,
    event: 'ADD' | 'UPDATE' | 'DELETE' | 'NONE',
    tags: string[],
    confidence: number,
    reasoning: string,
    code_pattern?: string,
    old_memory?: string
  }>,
  summary: Array<{
    factPreview: string,
    action: string,
    confidence: number,
    reason: string,
    targetId: number
  }>,
  timestamp: string
}

Storage Payload

Knowledge memories are stored with rich metadata:

interface KnowledgePayload {
  id: number,
  text: string,                           // The knowledge content
  tags: string[],                         // Technical tags (e.g., 'react', 'api')
  confidence: number,                     // Quality confidence (0.0-1.0)
  reasoning: string,                      // Why this operation was chosen
  event: 'ADD' | 'UPDATE' | 'DELETE' | 'NONE',
  timestamp: string,
  version: 2,
  
  // Enhanced metadata
  domain?: string,                        // Inferred domain (e.g., 'frontend')
  sourceSessionId?: string,              // Source session tracking
  qualitySource: 'similarity' | 'llm' | 'heuristic',
  code_pattern?: string,                 // Extracted code patterns
  old_memory?: string                    // Previous version (UPDATE only)
}

Memory Search Tool

cipher_search_memory This tool performs semantic search over the stored knowledge base to retrieve relevant information based on natural language queries. It uses vector similarity search to find the most contextually relevant knowledge for any given question or topic. Key Features:

Natural language query processing
Semantic similarity search using embeddings
Configurable result filtering and ranking
Detailed metadata and confidence scoring
Fast retrieval with similarity thresholds

Input Parameters

{
  query: string,                          // Natural language search query
  top_k?: number,                        // Max results (1-50, default: 5)
  similarity_threshold?: number,         // Min similarity (0.0-1.0, default: 0.3)
  include_metadata?: boolean             // Include detailed metadata
}

Output Structure

{
  success: boolean,
  query: string,
  results: Array<{
    id: string,
    text: string,                         // Knowledge content
    tags: string[],                       // Technical tags
    timestamp: string,
    similarity: number,                   // Similarity score (0.0-1.0)
    source: 'knowledge',
    memoryType: 'knowledge',
    
    // Knowledge-specific fields
    confidence?: number,
    reasoning?: string,
    event?: string,
    domain?: string,
    qualitySource?: string,
    code_pattern?: string,
    sourceSessionId?: string
  }>,
  metadata: {
    totalResults: number,
    searchTime: number,
    embeddingTime: number,
    maxSimilarity: number,
    minSimilarity: number,
    averageSimilarity: number,
    searchMode: 'knowledge'
  },
  timestamp: string
}

Advanced Features

Intelligent Content Filtering

Knowledge memory automatically filters content to focus on significant programming knowledge:

Technical Patterns: Identifies code blocks, commands, and implementation details
Programming Concepts: Extracts algorithms, design patterns, and best practices
Domain Knowledge: Captures project-specific information and configurations
Quality Assessment: Evaluates content relevance and completeness

LLM-Powered Decision Making

When enabled, the system uses LLM analysis to make intelligent memory operation decisions:

Similarity Analysis: Compares new content with existing memories
Content Enhancement: Identifies improved or corrected versions
Contradiction Detection: Finds conflicting information requiring updates
Deduplication: Prevents redundant memory entries

Automatic Tagging

Extracts technical tags from content for improved organization:

// Example extracted tags
["react", "component", "hooks", "typescript", "frontend", "api", "async"]

Code Pattern Extraction

Automatically identifies and preserves code patterns:

code_pattern: "const useAsync = (asyncFunction) => { /* implementation */ }"

Examples

Basic Knowledge Extraction

# Input conversation
User: "How do I create a React hook for async operations?"
Assistant: "You can create a custom hook like this:

const useAsync = (asyncFunction) => {
  const [data, setData] = useState(null);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);
  
  useEffect(() => {
    setLoading(true);
    asyncFunction()
      .then(setData)
      .catch(setError)
      .finally(() => setLoading(false));
  }, [asyncFunction]);
  
  return { data, loading, error };
};

This pattern is useful for API calls and other async operations."

# Extracted knowledge
{
  "text": "React custom hook useAsync for handling async operations with loading states, error handling, and data management using useState and useEffect hooks",
  "tags": ["react", "hooks", "async", "usestate", "useeffect", "javascript"],
  "code_pattern": "const useAsync = (asyncFunction) => { /* implementation */ }",
  "event": "ADD",
  "confidence": 0.95
}

Memory Search Example

# Search query
"How to handle async operations in React?"

# Search results
[
  {
    "text": "React custom hook useAsync for handling async operations...",
    "tags": ["react", "hooks", "async"],
    "similarity": 0.94,
    "code_pattern": "const useAsync = (asyncFunction) => { ... }",
    "domain": "frontend"
  },
  {
    "text": "React Query library provides powerful async state management...",
    "tags": ["react", "reactquery", "async", "cache"],
    "similarity": 0.87
  }
]

Update Operation Example

// Original memory
{
  "text": "React useEffect runs after every render",
  "tags": ["react", "useeffect"]
}

// New improved information
"React useEffect runs after every render by default, but you can control when it runs using the dependency array"

// Result: UPDATE operation
{
  "event": "UPDATE",
  "text": "React useEffect runs after every render by default, but you can control when it runs using the dependency array",
  "old_memory": "React useEffect runs after every render",
  "reasoning": "Enhanced with dependency array information"
}

Best Practices

Configuration

Set similarity threshold to 0.8+ for high-quality knowledge extraction
Use maxSimilarResults: 3-5 to prevent information overload
Enable useLLMDecisions: true for better ADD/UPDATE/DELETE decisions

Search Strategies

Query with context: “React async patterns” vs just “async”
Set top_k: 3-7 for focused results, top_k: 10+ for exploration
Use similarity_threshold: 0.4+ to filter noise

Memory Management

Review UPDATE operations - cipher may merge similar knowledge incorrectly
Monitor confidence scores - items below 0.6 may need manual review
Use enableDeleteOperations: false initially to prevent data loss
Enable debug logging - set CIPHER_LOG_LEVEL=debug to see detailed logs for cipher_extract_and_operate_memory (not visible at info level)

Knowledge memory transforms conversations into a persistent, searchable knowledge base that continuously improves the agent’s understanding and capabilities.

Getting Started

Configuration

Memory System

MCP Integration

Configuration

Required Environment Variables

Vector Storage Configuration

Qdrant Configuration

Milvus Configuration

Development Configuration

Knowledge Memory Tools

Extract & Operate Memory Tool

Memory Search Tool

Advanced Features

Intelligent Content Filtering

LLM-Powered Decision Making

Automatic Tagging

Code Pattern Extraction

Examples

Best Practices

Configuration

Search Strategies

Memory Management

Getting Started

Configuration

Memory System

MCP Integration

​Configuration

​Required Environment Variables

​Vector Storage Configuration

​Qdrant Configuration

​Milvus Configuration

​Development Configuration

​Knowledge Memory Tools

​Extract & Operate Memory Tool

​Memory Search Tool

​Advanced Features

​Intelligent Content Filtering

​LLM-Powered Decision Making

​Automatic Tagging

​Code Pattern Extraction

​Examples

​Best Practices

​Configuration

​Search Strategies

​Memory Management

Configuration

Required Environment Variables

Vector Storage Configuration

Qdrant Configuration

Milvus Configuration

Development Configuration

Knowledge Memory Tools

Extract & Operate Memory Tool

Memory Search Tool

Advanced Features

Intelligent Content Filtering

LLM-Powered Decision Making

Automatic Tagging

Code Pattern Extraction

Examples

Best Practices

Configuration

Search Strategies

Memory Management