Skip to main content

Configuration

Required Environment Variables

# OpenAI API Key (Required for embeddings)
OPENAI_API_KEY=sk-proj-your-openai-key-here

# Vector Store Type (Required)
VECTOR_STORE_TYPE=qdrant  # qdrant, milvus, in-memory

Vector Storage Configuration

Qdrant Configuration

# Connection Options (choose one)
VECTOR_STORE_URL=http://localhost:6333           # Complete URL
# OR
VECTOR_STORE_HOST=localhost                      # Host + Port
VECTOR_STORE_PORT=6333

# Optional Settings
VECTOR_STORE_API_KEY=your-qdrant-api-key        # For Qdrant Cloud
VECTOR_STORE_COLLECTION=knowledge_memory         # Collection name
VECTOR_STORE_DIMENSION=1536                     # Embedding dimensions
VECTOR_STORE_DISTANCE=Cosine                    # Distance metric
VECTOR_STORE_ON_DISK=false                      # Store vectors on disk

Milvus Configuration

# Connection Options (choose one)
VECTOR_STORE_URL=http://localhost:19530          # Complete URL
# OR
VECTOR_STORE_HOST=localhost                      # Host + Port
VECTOR_STORE_PORT=19530

# Authentication (Zilliz Cloud)
VECTOR_STORE_USERNAME=your-username              # Optional
VECTOR_STORE_PASSWORD=your-password              # Optional
VECTOR_STORE_API_KEY=your-api-token             # Optional

# Collection Settings
VECTOR_STORE_COLLECTION=knowledge_memory         # Collection name
VECTOR_STORE_DIMENSION=1536                     # Embedding dimensions

Development Configuration

# In-Memory (Development/Testing)
VECTOR_STORE_TYPE=in-memory
VECTOR_STORE_COLLECTION=knowledge_memory
VECTOR_STORE_DIMENSION=1536
VECTOR_STORE_MAX_VECTORS=10000                  # Memory limit

Knowledge Memory Tools

Cipher provides two main tools for knowledge memory management:

Extract & Operate Memory Tool

cipher_extract_and_operate_memory This tool automatically processes conversations and determines what knowledge should be stored, updated, or removed. It uses intelligent content analysis and LLM-powered decision making to extract meaningful programming knowledge while avoiding redundant or low-quality information. Key Features:
  • Automatic knowledge extraction from conversations
  • Intelligent similarity detection to prevent duplicates
  • LLM-powered decision making for memory operations (ADD/UPDATE/DELETE)
  • Rich metadata tagging and categorization
  • Code pattern recognition and preservation
{
  interaction: string | string[],           // Conversation text(s)
  existingMemories?: Array<{               // Optional existing memories
    id: string,
    text: string,
    metadata?: Record<string, any>
  }>,
  context?: {                              // Optional context
    sessionId?: string,
    userId?: string,
    projectId?: string,
    conversationTopic?: string
  },
  options?: {                              // Configuration options
    similarityThreshold?: number,           // 0.0-1.0 (default: 0.8)
    maxSimilarResults?: number,            // 1-20 (default: 5)
    useLLMDecisions?: boolean,             // Enable LLM decisions
    enableDeleteOperations?: boolean       // Allow DELETE operations
  },
  memoryMetadata?: {                       // Custom metadata
    projectId?: string,
    userId?: string,
    environment?: string
  }
}
{
  success: boolean,
  extraction: {
    extracted: number,                     // Number of facts extracted
    skipped: number,                       // Non-significant content skipped
    facts: Array<{
      id: string,
      preview: string,
      metadata: {
        hasCodeBlock: boolean,
        hasCommand: boolean,
        length: number
      }
    }>
  },
  memory: Array<{
    id: number,
    text: string,
    event: 'ADD' | 'UPDATE' | 'DELETE' | 'NONE',
    tags: string[],
    confidence: number,
    reasoning: string,
    code_pattern?: string,
    old_memory?: string
  }>,
  summary: Array<{
    factPreview: string,
    action: string,
    confidence: number,
    reason: string,
    targetId: number
  }>,
  timestamp: string
}
Knowledge memories are stored with rich metadata:
interface KnowledgePayload {
  id: number,
  text: string,                           // The knowledge content
  tags: string[],                         // Technical tags (e.g., 'react', 'api')
  confidence: number,                     // Quality confidence (0.0-1.0)
  reasoning: string,                      // Why this operation was chosen
  event: 'ADD' | 'UPDATE' | 'DELETE' | 'NONE',
  timestamp: string,
  version: 2,
  
  // Enhanced metadata
  domain?: string,                        // Inferred domain (e.g., 'frontend')
  sourceSessionId?: string,              // Source session tracking
  qualitySource: 'similarity' | 'llm' | 'heuristic',
  code_pattern?: string,                 // Extracted code patterns
  old_memory?: string                    // Previous version (UPDATE only)
}

Memory Search Tool

cipher_search_memory This tool performs semantic search over the stored knowledge base to retrieve relevant information based on natural language queries. It uses vector similarity search to find the most contextually relevant knowledge for any given question or topic. Key Features:
  • Natural language query processing
  • Semantic similarity search using embeddings
  • Configurable result filtering and ranking
  • Detailed metadata and confidence scoring
  • Fast retrieval with similarity thresholds
{
  query: string,                          // Natural language search query
  top_k?: number,                        // Max results (1-50, default: 5)
  similarity_threshold?: number,         // Min similarity (0.0-1.0, default: 0.3)
  include_metadata?: boolean             // Include detailed metadata
}
{
  success: boolean,
  query: string,
  results: Array<{
    id: string,
    text: string,                         // Knowledge content
    tags: string[],                       // Technical tags
    timestamp: string,
    similarity: number,                   // Similarity score (0.0-1.0)
    source: 'knowledge',
    memoryType: 'knowledge',
    
    // Knowledge-specific fields
    confidence?: number,
    reasoning?: string,
    event?: string,
    domain?: string,
    qualitySource?: string,
    code_pattern?: string,
    sourceSessionId?: string
  }>,
  metadata: {
    totalResults: number,
    searchTime: number,
    embeddingTime: number,
    maxSimilarity: number,
    minSimilarity: number,
    averageSimilarity: number,
    searchMode: 'knowledge'
  },
  timestamp: string
}

Advanced Features

Intelligent Content Filtering

Knowledge memory automatically filters content to focus on significant programming knowledge:
  • Technical Patterns: Identifies code blocks, commands, and implementation details
  • Programming Concepts: Extracts algorithms, design patterns, and best practices
  • Domain Knowledge: Captures project-specific information and configurations
  • Quality Assessment: Evaluates content relevance and completeness

LLM-Powered Decision Making

When enabled, the system uses LLM analysis to make intelligent memory operation decisions:
  • Similarity Analysis: Compares new content with existing memories
  • Content Enhancement: Identifies improved or corrected versions
  • Contradiction Detection: Finds conflicting information requiring updates
  • Deduplication: Prevents redundant memory entries

Automatic Tagging

Extracts technical tags from content for improved organization:
// Example extracted tags
["react", "component", "hooks", "typescript", "frontend", "api", "async"]

Code Pattern Extraction

Automatically identifies and preserves code patterns:
code_pattern: "const useAsync = (asyncFunction) => { /* implementation */ }"

Examples

# Input conversation
User: "How do I create a React hook for async operations?"
Assistant: "You can create a custom hook like this:

const useAsync = (asyncFunction) => {
  const [data, setData] = useState(null);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);
  
  useEffect(() => {
    setLoading(true);
    asyncFunction()
      .then(setData)
      .catch(setError)
      .finally(() => setLoading(false));
  }, [asyncFunction]);
  
  return { data, loading, error };
};

This pattern is useful for API calls and other async operations."

# Extracted knowledge
{
  "text": "React custom hook useAsync for handling async operations with loading states, error handling, and data management using useState and useEffect hooks",
  "tags": ["react", "hooks", "async", "usestate", "useeffect", "javascript"],
  "code_pattern": "const useAsync = (asyncFunction) => { /* implementation */ }",
  "event": "ADD",
  "confidence": 0.95
}
# Search query
"How to handle async operations in React?"

# Search results
[
  {
    "text": "React custom hook useAsync for handling async operations...",
    "tags": ["react", "hooks", "async"],
    "similarity": 0.94,
    "code_pattern": "const useAsync = (asyncFunction) => { ... }",
    "domain": "frontend"
  },
  {
    "text": "React Query library provides powerful async state management...",
    "tags": ["react", "reactquery", "async", "cache"],
    "similarity": 0.87
  }
]
// Original memory
{
  "text": "React useEffect runs after every render",
  "tags": ["react", "useeffect"]
}

// New improved information
"React useEffect runs after every render by default, but you can control when it runs using the dependency array"

// Result: UPDATE operation
{
  "event": "UPDATE",
  "text": "React useEffect runs after every render by default, but you can control when it runs using the dependency array",
  "old_memory": "React useEffect runs after every render",
  "reasoning": "Enhanced with dependency array information"
}

Best Practices

Configuration

  • Set similarity threshold to 0.8+ for high-quality knowledge extraction
  • Use maxSimilarResults: 3-5 to prevent information overload
  • Enable useLLMDecisions: true for better ADD/UPDATE/DELETE decisions

Search Strategies

  • Query with context: “React async patterns” vs just “async”
  • Set top_k: 3-7 for focused results, top_k: 10+ for exploration
  • Use similarity_threshold: 0.4+ to filter noise

Memory Management

  • Review UPDATE operations - cipher may merge similar knowledge incorrectly
  • Monitor confidence scores - items below 0.6 may need manual review
  • Use enableDeleteOperations: false initially to prevent data loss
  • Enable debug logging - set CIPHER_LOG_LEVEL=debug to see detailed logs for cipher_extract_and_operate_memory (not visible at info level)
Knowledge memory transforms conversations into a persistent, searchable knowledge base that continuously improves the agent’s understanding and capabilities.
I