cortexdb

README

CortexDB

An embedded cognitive memory and graph database for AI Agents.

CortexDB is a 100% pure Go library that transforms a single SQLite file into a powerful AI storage engine. It blends Hybrid Vector Search, GraphRAG, MCP Tool Calling, and a Hindsight-inspired Agent Memory System, giving AI applications a structured, persistent, and intelligent brain without complex external infrastructure.

✨ Why CortexDB?

• 🧠 Agent Memory (Hindsight) – Full retain → recall → reflect lifecycle with multi-channel TEMPR retrieval.
• 🕸️ Dual-Mode GraphRAG – Use embedder-backed GraphRAG when vectors are available, or lexical/tool-calling GraphRAG when only an LLM is available.
• 🔍 Hybrid Search – Combines Vector similarity (HNSW) and precise Keyword matching (FTS5) using RRF fusion.
• 🔌 MCP Stdio Server – Expose CortexDB tools to external LLMs through the official Model Context Protocol Go SDK.
• 🏗️ Structured Data Friendly – Easily map SQL/CSV rows to natural language + metadata for advanced PreFilter querying.
• 🪶 Ultra Lightweight – Single SQLite file, zero external dependencies. Pure Go.
• 🛡️ Secure – Row-Level Security via ACL fields to isolate multi-tenant data.

🚀 Quick Start

go get github.com/liliang-cn/cortexdb/v2

Run the built-in MCP stdio server:

go run ./cmd/cortexdb-mcp-stdio

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/liliang-cn/cortexdb/v2/pkg/cortexdb"
)

func main() {
	// Initialize CortexDB (auto-creates tables for vectors, docs, memory, graph)
	db, err := cortexdb.Open(cortexdb.DefaultConfig("brain.db"))
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	ctx := context.Background()

	// 1. Store a memory/fact
	db.Quick().Add(ctx, []float32{0.1, 0.2, 0.9}, "Go is a statically typed, compiled language.")

	// 2. Recall similar concepts
	results, _ := db.Quick().Search(ctx, []float32{0.1, 0.2, 0.8}, 1)
	if len(results) > 0 {
		fmt.Println("Recalled:", results[0].Content)
	}
}

🏗 Core Capabilities

1. The Agent Memory System (Hindsight)

CortexDB includes hindsight, a dedicated bionic memory module for Agents. It doesn't just store logs; it categorizes memories (Facts, Beliefs, Observations) and recalls them dynamically.

import "github.com/liliang-cn/cortexdb/v2/pkg/hindsight"

sys, _ := hindsight.New(&hindsight.Config{DBPath: "agent_memory.db"})
defer sys.Close()

// Create an Agent profile with personality traits
bank := hindsight.NewBank("travel-agent-1", "Travel Assistant")
bank.Empathy = 4      // 1-5 scale
bank.Skepticism = 2   // 1-5 scale
sys.CreateBank(ctx, bank)

// RETAIN: Store structured observations about the user
sys.Retain(ctx, &hindsight.Memory{
	BankID:   "travel-agent-1",
	Type:     hindsight.WorldMemory,
	Content:  "Alice prefers window seats and vegetarian meals.",
	Vector:   embed("Alice prefers window seats..."),
	Entities: []string{"user:alice", "preference:flight", "preference:food"},
})

// RECALL: Multi-channel TEMPR retrieval (Temporal, Entity, Memory, Priming, Recall)
results, _ := sys.Recall(ctx, &hindsight.RecallRequest{
	BankID:      "travel-agent-1",
	QueryVector: embed("What food should I order for Alice?"),
	Strategy:    hindsight.DefaultStrategy(), // Uses all channels + RRF Fusion
})

2. High-Level Text & Structured Data APIs

Stop dealing with raw []float32 arrays manually. Hook up your Embedder and let CortexDB handle the rest.

// 1. Inject your embedding model (OpenAI, Ollama, etc.)
db, _ := cortexdb.Open(config, cortexdb.WithEmbedder(myOpenAIEmbedder))

// 2. Insert raw text directly
db.InsertText(ctx, "doc_1", "The new iPhone 15 Pro features a titanium body.", map[string]string{
	"category": "electronics",
	"price":    "999",
})

// 3. Search purely by text
results, _ := db.SearchText(ctx, "latest Apple phones", 5)

// 4. Hybrid Search (Semantic + Exact Keyword)
hybridRes, _ := db.HybridSearchText(ctx, "titanium body", 5)

// 5. FTS5 Only (No vectors needed, super fast!)
ftsRes, _ := db.SearchTextOnly(ctx, "iPhone", cortexdb.TextSearchOptions{TopK: 5})

See examples/text_api for the full code.

3. GraphRAG (Knowledge Graph)

Transform relational data (like SQL tables) into a Knowledge Graph for multi-hop reasoning.

// 1. Insert Nodes
db.Graph().UpsertNode(ctx, &graph.GraphNode{
	ID: "dept_eng", NodeType: "department", Content: "Engineering Department", Vector: vec1,
})
db.Graph().UpsertNode(ctx, &graph.GraphNode{
	ID: "emp_alice", NodeType: "employee", Content: "Alice (Senior Go Dev)", Vector: vec2,
})

// 2. Create Relationships (Edges)
db.Graph().UpsertEdge(ctx, &graph.GraphEdge{
	FromNodeID: "emp_alice",
	ToNodeID:   "dept_eng",
	EdgeType:   "BELONGS_TO",
	Weight:     1.0,
})

// 3. Traverse the Graph automatically during RAG
neighbors, _ := db.Graph().Neighbors(ctx, "emp_alice", graph.TraversalOptions{
	EdgeTypes: []string{"BELONGS_TO"},
	MaxDepth:  1,
})
// Finds that Alice belongs to the Engineering Department without complex SQL JOINs!

See examples/structured_data for the full code.

There is also first-class RDF knowledge graph support when you want real triples/quads instead of ad-hoc property graphs:

db.UpsertKnowledgeNamespace(ctx, cortexdb.KnowledgeGraphNamespaceUpsertRequest{
	Prefix: "ex",
	URI:    "https://example.com/",
})

db.UpsertKnowledgeGraph(ctx, cortexdb.KnowledgeGraphUpsertRequest{
	Triples: []cortexdb.KnowledgeGraphTriple{
		{
			Subject:   graph.NewIRI("ex:alice"),
			Predicate: graph.NewIRI("rdf:type"),
			Object:    graph.NewIRI("schema:Person"),
		},
		{
			Subject:   graph.NewIRI("ex:alice"),
			Predicate: graph.NewIRI("schema:name"),
			Object:    graph.NewLiteral("Alice"),
		},
	},
})

rdfDump, _ := db.ExportKnowledgeGraph(ctx, cortexdb.KnowledgeGraphExportRequest{
	Format: cortexdb.KnowledgeGraphFormatNQuads,
})

sparql, _ := db.QueryKnowledgeGraph(ctx, cortexdb.KnowledgeGraphQueryRequest{
	Query: `
PREFIX ex: <https://example.com/>
PREFIX schema: <https://schema.org/>

SELECT ?name WHERE {
	ex:alice schema:name ?name .
}
`,
})

inferred, _ := db.RefreshKnowledgeGraphInference(ctx, cortexdb.KnowledgeGraphInferenceRefreshRequest{})
_ = inferred

Current SPARQL support is an embedded subset: PREFIX, SELECT, ASK, CONSTRUCT, DESCRIBE, INSERT DATA, INSERT ... WHERE, DELETE DATA, DELETE WHERE, DELETE ... INSERT ... WHERE, WITH, USING, GRAPH, OPTIONAL, UNION, VALUES, BIND, FILTER, REGEX, LANG, DATATYPE, COALESCE, IF, arithmetic expressions, GROUP BY, HAVING, COUNT, SUM, AVG, MIN, MAX, SAMPLE, GROUP_CONCAT, ORDER BY, LIMIT, and OFFSET. RDFS-lite inference is also built in, including rdfs:subClassOf, rdfs:subPropertyOf, rdfs:domain, and rdfs:range.

Higher-level GraphRAG retrieval is also available when an embedder is configured:

result, _ := db.SearchGraphRAG(ctx, "Where does Alice work?", cortexdb.GraphRAGQueryOptions{
	TopK:          4,
	RetrievalMode: cortexdb.RetrievalModeAuto, // auto | lexical | graph
})

RetrievalMode is useful because graph expansion can be expensive on large graphs:

• lexical keeps retrieval fast by skipping graph expansion.
• graph always expands the graph.
• auto uses lightweight entity heuristics to decide whether graph expansion is worth the cost.

If you already have external LLM orchestration, the no-embedder tool surface exposes the same idea through retrieval_mode and keywords / alternate_queries.

Schema validation and inference can also be layered onto this workflow:

_, _ = db.SaveOntologySchema(ctx, cortexdb.OntologySaveRequest{
	SchemaID: "company-graph",
	Activate: true,
	EntityTypes: []cortexdb.OntologyEntityType{
		{Name: "entity"},
		{Name: "person"},
		{Name: "organization"},
		{Name: "city"},
	},
	RelationTypes: []cortexdb.OntologyRelationType{
		{Name: "works_at", AllowedFromTypes: []string{"person"}, AllowedToTypes: []string{"organization"}},
		{Name: "located_in", AllowedFromTypes: []string{"organization"}, AllowedToTypes: []string{"city"}},
		{Name: "works_in_city", AllowedFromTypes: []string{"person"}, AllowedToTypes: []string{"city"}},
	},
})

_, _ = db.ApplyInferenceRules(ctx, cortexdb.ApplyInferenceRequest{
	DocumentID: "alice-berlin-proof",
	Rules: []cortexdb.InferenceRule{
		{
			RuleID:             "employment_city",
			LeftRelationType:   "works_at",
			RightRelationType:  "located_in",
			ResultRelationType: "works_in_city",
		},
	},
})

Use the new end-to-end examples for this flow:

• examples/graphrag_embedder
• examples/llm_tool_calling

4. MCP Tool Calling / No-Embedder Mode

If you do not have an embedding model but you do have an LLM, CortexDB can still be used as an MCP tool server. In this mode:

• the LLM expands the user goal into many keywords, aliases, synonyms, abbreviations, and multilingual variants
• CortexDB uses FTS5/BM25 for seed retrieval
• graph expansion is optional through retrieval_mode=lexical|graph|auto

Run the stdio MCP server:

go run ./cmd/cortexdb-mcp-stdio

Or embed it directly in your own Go process:

if err := db.RunMCPStdio(context.Background(), cortexdb.MCPServerOptions{}); err != nil {
	log.Fatal(err)
}

High-level Go APIs:

knowledge, _ := db.SaveKnowledge(ctx, cortexdb.KnowledgeSaveRequest{
	KnowledgeID: "doc-1",
	Title:       "Alice at Acme",
	Content:     "Alice works at Acme on GraphRAG.",
})

memory, _ := db.SaveMemory(ctx, cortexdb.MemorySaveRequest{
	MemoryID:  "mem-1",
	UserID:    "user-1",
	Scope:     cortexdb.MemoryScopeUser,
	Namespace: "assistant",
	Content:   "Alice prefers concise answers.",
})

_, _ = knowledge, memory

Main MCP tools:

• knowledge_save
• knowledge_update
• knowledge_get
• knowledge_search
• knowledge_delete
• memory_save
• memory_update
• memory_get
• memory_search
• memory_delete
• ingest_document
• upsert_entities
• upsert_relations
• search_text
• search_chunks_by_entities
• expand_graph
• get_nodes
• get_chunks
• build_context
• search_graphrag_lexical

The higher-level ontology and inference tools are also available:

• ontology_save
• ontology_get
• ontology_list
• ontology_delete
• apply_inference

Recommended docs:

• docs/retrieval_planning.md
• docs/evaluation.md
• docs/release_notes_llm_first.md

5. Advanced Metadata Filtering

Filter large datasets before performing vector search using a SQL-like expression builder.

// Find laptops under $2000 that are currently in stock
filter := core.NewMetadataFilter().
	And(core.NewMetadataFilter().Equal("category", "laptop")).
	And(core.NewMetadataFilter().LessThan("price", 2000.0)).
	And(core.NewMetadataFilter().GreaterThan("stock", 0)).
	Build()

opts := core.AdvancedSearchOptions{
	SearchOptions: core.SearchOptions{TopK: 5},
	PreFilter:     filter, 
}
results, _ := db.Vector().SearchWithAdvancedFilter(ctx, queryVec, opts)

📚 Database Schema

CortexDB manages the following tables automatically inside your single .db file:

Table	Description
embeddings	Vectors, content, JSON metadata, ACLs.
documents	Parent records for embeddings (Title, URL, Version).
sessions	Chat sessions / threads.
messages	Chat logs (Role, Content, Vector, Timestamp).
messages_fts	FTS5 virtual table for BM25 keyword search over messages.
collections	Logical namespaces for multi-tenancy.
chunks_fts	FTS5 virtual table for keyword search over embeddings.
graph_nodes	Knowledge graph nodes with vector embeddings.
graph_edges	Directed relationships between graph nodes.

📊 Performance (128-dim, Apple M2 Pro)

Index Type	Insert Speed	Search QPS	Memory (1 M vecs)
HNSW	~580 ops/s	~720 QPS	~1.2 GB (SQ8)
IVF	~14,500 ops/s	~1,230 QPS	~1.0 GB (SQ8)

⚖️ License

MIT License. See LICENSE file.

Documentation

Overview

Package cortexdb provides a lightweight, embeddable vector database for Go AI projects.

cortexdb is a 100% pure Go library designed to be the storage kernel for RAG (Retrieval-Augmented Generation) systems. Built on SQLite using modernc.org/sqlite (NO CGO REQUIRED!), it provides vector storage, full-text search (FTS5), knowledge graphs, and chat memory management in a single database file.

Key Features

• 🚀 RAG-Ready - Built-in support for Documents, Chat Sessions, and Messages.
• 🔍 Hybrid Search - Combine Vector Search (HNSW/IVF) with Keyword Search (FTS5) using RRF fusion.
• 🧠 Memory Efficient - Built-in SQ8 quantization reduces RAM usage by 75%.
• 🛡️ Secure - Row-Level Security via ACL fields and SQL-level push-down filtering.
• 🔧 100% Pure Go - Easy cross-compilation and zero-dependency deployment.
• 🕸️ GraphRAG - Advanced graph operations for complex relationship-based retrieval.
• 🔌 MCP Stdio Server - Expose CortexDB tools to external LLMs through the official Model Context Protocol Go SDK.
• 🧠 High-level Knowledge & Memory APIs - Simplified SaveKnowledge and SaveMemory operations for building agentic workflows.
• 🧭 Semantic Router - Intent routing with configurable similarity and thresholds.
• 🧠 Hindsight - Biomimetic agent memory with TEMPR retrieval.

Quick Start

import (
    "context"
    "github.com/liliang-cn/cortexdb/v2/pkg/cortexdb"
)

func main() {
    // 1. Open database with default configuration
    config := cortexdb.DefaultConfig("vectors.db")
    db, _ := cortexdb.Open(config)
    defer db.Close()

    // 2. Use the Quick interface for simple operations
    ctx := context.Background()
    quick := db.Quick()

    // Add vector
    quick.Add(ctx, []float32{0.1, 0.2, 0.3}, "Go is awesome")

    // Search
    results, _ := quick.Search(ctx, []float32{0.1, 0.2, 0.28}, 5)
}

RAG and Hybrid Search

cortexdb provides high-level APIs for building RAG applications:

import "github.com/liliang-cn/cortexdb/v2/pkg/core"

// Perform hybrid search (Vector + Full-Text Search)
results, err := db.Vector().HybridSearch(ctx, queryVec, "search term", core.HybridSearchOptions{
    TopK: 5,
})

Knowledge and Memory

Build agentic systems with higher-level persistence:

// Save a document and its knowledge artifacts (chunks, entities)
db.SaveKnowledge(ctx, cortexdb.KnowledgeSaveRequest{
    KnowledgeID: "doc_1",
    Title:       "CortexDB Overview",
    Content:     "CortexDB is a lightweight vector database...",
})

// Store a memory record with scope and importance
db.SaveMemory(ctx, cortexdb.MemorySaveRequest{
    MemoryID:  "mem_1",
    Content:   "Alice prefers window seats.",
    Importance: 4.5,
})

Chat Memory

Built-in conversation history management:

// Add a message to a session
db.Vector().AddMessage(ctx, &core.Message{
    SessionID: "session_123",
    Role:      "user",
    Content:   "How do I use cortexdb?",
})

Advanced Configuration

Configure indexing and quantization for production:

config := cortexdb.DefaultConfig("data.db")
config.IndexType = core.IndexTypeHNSW // or core.IndexTypeIVF
config.SimilarityFn = core.CosineSimilarity
config.Dimensions = 1536

db, err := cortexdb.Open(config)

For deeper control (quantization, logger, text similarity), use core.Config with core.NewWithConfig.

Observability

cortexdb supports structured logging via core.Config.Logger when using core.NewWithConfig.

MCP Tool Calling

Run as an MCP stdio server to expose tools to LLM clients:

if err := db.RunMCPStdio(ctx, cortexdb.MCPServerOptions{}); err != nil {
    log.Fatal(err)
}

Semantic Router

Route user intent before expensive LLM calls:

import (
    "context"
    "github.com/liliang-cn/cortexdb/v2/pkg/core"
    semanticrouter "github.com/liliang-cn/cortexdb/v2/pkg/semantic-router"
)

router, _ := semanticrouter.NewRouter(
    semanticrouter.NewMockEmbedder(1536),
    semanticrouter.WithThreshold(0.82),
    semanticrouter.WithSimilarityFunc(core.CosineSimilarity),
)

_ = router.Add(&semanticrouter.Route{
    Name: "refund",
    Utterances: []string{"我要退款", "申请退款"},
})

result, _ := router.Route(context.Background(), "我要退款")
_ = result

Hindsight Memory

Long-term agent memory with TEMPR retrieval:

import "github.com/liliang-cn/cortexdb/v2/pkg/hindsight"

sys, _ := hindsight.New(&hindsight.Config{DBPath: "agent_memory.db"})
_ = sys

For more detailed examples, see the examples/ directory.

Index

• Constants

Constants

const Version = "2.16.0"

Version represents the current version of the cortexdb library.