vault

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotSupported = fmt.Errorf("operation not supported by obsidian backend")

ErrNotSupported is returned for Logseq-specific operations (DataScript queries).

View Source 
var ErrPathEscape = fmt.Errorf("path escapes vault boundary")

ErrPathEscape is returned when a resolved path escapes the vault boundary.

Functions

func ExtractHeadingFromContent added in v1.0.0 

func ExtractHeadingFromContent(content string) string

ExtractHeadingFromContent returns the heading text (without # prefix) from a block's content, or empty string if not a heading. Examines only the first line. This is the single implementation of heading extraction — used by both VaultStore.generateEmbeddings() and the CLI indexing pipeline.

func GenerateEmbeddings added in v1.4.0 

func GenerateEmbeddings(s *store.Store, embedder embed.Embedder, pageName string, blocks []types.BlockEntity, headingPath []string) int

GenerateEmbeddings creates vector embeddings for blocks and persists them to the store. This is the shared implementation used by both VaultStore (serve-time persistence) and the CLI indexing pipeline, eliminating duplication (same pattern as PersistBlocks/PersistLinks).

Returns the number of embeddings generated. Embedding failures are logged but don't block indexing (graceful degradation).

func HeadingLevelFromContent added in v1.0.0 

func HeadingLevelFromContent(content string) int

HeadingLevelFromContent returns the markdown heading level (1-6) for a block's content, or 0 if the content does not start with a heading. Examines only the first line of multi-line content.

func ParseDocument added in v1.0.0 

func ParseDocument(docID, content string) (props map[string]any, blocks []types.BlockEntity)

ParseDocument parses a markdown document's content into frontmatter properties and a hierarchical block tree. This is the exported entry point for external callers (e.g., `dewey index`) that need to parse document content without requiring a vault.Client instance or filesystem access.

The docID parameter is used as a seed for deterministic block UUID generation.

Design decision: Wraps the unexported parseFrontmatter() and parseMarkdownBlocks() rather than exporting them directly, to provide a clean API boundary and avoid coupling callers to internal parsing details (per research R2).

func PersistBlocks added in v1.0.0 

func PersistBlocks(s *store.Store, pageName string, blocks []types.BlockEntity, parentUUID sql.NullString, startPos int) error

PersistBlocks recursively inserts blocks into the store. This is the shared implementation used by both VaultStore.persistBlocks() and the CLI indexing pipeline, eliminating duplication (Architect DRY finding). 

func PersistLinks(s *store.Store, pageName string, blocks []types.BlockEntity) error

PersistLinks extracts wikilinks from blocks and persists them to the store. This is the shared implementation used by both VaultStore.persistLinks() and the CLI indexing pipeline, eliminating duplication (Architect DRY finding).

func SetLogLevel added in v1.2.0 

func SetLogLevel(level log.Level)

SetLogLevel sets the logging level for the vault package. Use log.DebugLevel for verbose output during diagnostics.

func SetLogOutput added in v1.3.1 

func SetLogOutput(w io.Writer, level log.Level)

SetLogOutput replaces the vault package logger with one that writes to the given writer at the given level. Used to enable file logging.

func SortByRelevance 

func SortByRelevance(results []SearchResult, query string)

SortByRelevance sorts search results by term frequency. Not currently used but available for future ranking improvements.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client implements backend.Backend for an Obsidian vault on disk. It reads all .md files on initialization and serves queries from memory.

func New 

func New(vaultPath string, opts ...Option) *Client

New creates a new Obsidian vault client. Call Load() to index the vault. The ignore matcher is built eagerly so it is available to all startup paths (Load, IncrementalIndex, FullIndex) without requiring Load() to be called first.

func (*Client) AppendBlockInPage 

func (c *Client) AppendBlockInPage(_ context.Context, page string, content string) (*types.BlockEntity, error)
func (c *Client) BuildBacklinks()

BuildBacklinks must be called after Load() to build the reverse link index.

func (*Client) Close 

func (c *Client) Close() error

Close stops the file watcher.

func (*Client) CreatePage 

func (c *Client) CreatePage(_ context.Context, name string, properties map[string]any, opts map[string]any) (*types.PageEntity, error)

func (*Client) DatascriptQuery 

func (c *Client) DatascriptQuery(_ context.Context, query string, inputs ...any) (json.RawMessage, error)

func (*Client) DeletePage 

func (c *Client) DeletePage(_ context.Context, name string) error

func (*Client) FindBlocksByTag 

func (c *Client) FindBlocksByTag(_ context.Context, tag string, includeChildren bool) ([]backend.TagResult, error)

FindBlocksByTag scans all pages for blocks containing the given #tag. Implements backend.TagSearcher.

func (*Client) FindByProperty 

func (c *Client) FindByProperty(_ context.Context, key, value, operator string) ([]backend.PropertyResult, error)

FindByProperty scans all pages for matching frontmatter properties. Implements backend.PropertySearcher.

func (*Client) FullTextSearch 

func (c *Client) FullTextSearch(_ context.Context, query string, limit int) ([]backend.SearchHit, error)

FullTextSearch uses the inverted index for efficient full-text search. Implements backend.FullTextSearcher.

func (*Client) GetAllPages 

func (c *Client) GetAllPages(_ context.Context) ([]types.PageEntity, error)

func (*Client) GetBlock 

func (c *Client) GetBlock(_ context.Context, uuid string, opts ...map[string]any) (*types.BlockEntity, error)

func (*Client) GetPage 

func (c *Client) GetPage(_ context.Context, nameOrID any) (*types.PageEntity, error)

func (*Client) GetPageBlocksTree 

func (c *Client) GetPageBlocksTree(_ context.Context, nameOrID any) ([]types.BlockEntity, error)

func (*Client) GetPageLinkedReferences 

func (c *Client) GetPageLinkedReferences(_ context.Context, nameOrID any) (json.RawMessage, error)

func (*Client) InsertBlock 

func (c *Client) InsertBlock(_ context.Context, srcBlock any, content string, opts map[string]any) (*types.BlockEntity, error)

func (*Client) Load 

func (c *Client) Load() error

Load reads all .md files in the vault and builds the in-memory index.

func (*Client) MoveBlock 

func (c *Client) MoveBlock(_ context.Context, uuid string, targetUUID string, opts map[string]any) error

func (*Client) Ping 

func (c *Client) Ping(_ context.Context) error

func (*Client) PrependBlockInPage 

func (c *Client) PrependBlockInPage(_ context.Context, page string, content string) (*types.BlockEntity, error)

func (*Client) Reload 

func (c *Client) Reload() error

Reload forces a full vault re-index.

func (*Client) RemoveBlock 

func (c *Client) RemoveBlock(_ context.Context, uuid string) error

func (*Client) RenamePage 

func (c *Client) RenamePage(_ context.Context, oldName, newName string) error

func (*Client) SearchJournals 

func (c *Client) SearchJournals(_ context.Context, query, from, to string) ([]backend.JournalResult, error)

SearchJournals scans daily notes for matching content. Implements backend.JournalSearcher.

func (*Client) Store 

func (c *Client) Store() *VaultStore

Store returns the VaultStore adapter, or nil if persistence is not configured.

func (*Client) UpdateBlock 

func (c *Client) UpdateBlock(_ context.Context, uuid string, content string, opts ...map[string]any) error

func (*Client) Watch 

func (c *Client) Watch() error

Watch starts watching the vault directory for file changes and automatically re-indexes modified files. Runs in a goroutine and logs errors to stderr.

type IndexStats 

type IndexStats struct {
	New       int
	Changed   int
	Deleted   int
	Unchanged int
}

IndexStats reports the results of an incremental index operation.

func (IndexStats) Total 

func (s IndexStats) Total() int

Total returns the total number of files processed.

type Option 

type Option func(*Client)

Option configures a vault Client.

func WithDailyFolder 

func WithDailyFolder(folder string) Option

WithDailyFolder sets the daily notes subfolder (default: "daily notes").

func WithIgnorePatterns added in v1.5.0 

func WithIgnorePatterns(patterns []string) Option

WithIgnorePatterns configures additional ignore patterns (from sources.yaml) that are merged with .gitignore rules when the vault Client is constructed. These patterns use gitignore syntax and are evaluated alongside the .gitignore file found at the vault root.

func WithStore 

func WithStore(s *store.Store) Option

WithStore enables persistent indexing via the given store.Store. When set, the vault client persists index changes to SQLite alongside in-memory updates. Pass nil to disable persistence (default behavior).

type SearchIndex 

type SearchIndex struct {
	// contains filtered or unexported fields
}

SearchIndex is a simple inverted index for full-text search. Maps lowercase terms to the blocks containing them.

func NewSearchIndex 

func NewSearchIndex() *SearchIndex

NewSearchIndex creates an empty search index.

func (*SearchIndex) BuildFrom 

func (si *SearchIndex) BuildFrom(pages map[string]*cachedPage)

BuildFrom indexes all pages in the vault.

func (*SearchIndex) ReindexPage 

func (si *SearchIndex) ReindexPage(page *cachedPage)

ReindexPage removes and re-adds a single page to the index.

func (*SearchIndex) RemovePage 

func (si *SearchIndex) RemovePage(lowerName string)

RemovePage removes a page from the index.

func (*SearchIndex) Search 

func (si *SearchIndex) Search(query string, limit int) []SearchResult

Search finds blocks matching all terms in the query (AND semantics). Returns results sorted by relevance (number of term hits).

type SearchResult 

type SearchResult struct {
	PageName string `json:"page"`
	UUID     string `json:"uuid"`
	Content  string `json:"content"`
}

SearchResult is a block that matched a search query.

type VaultStore 

type VaultStore struct {
	// contains filtered or unexported fields
}

VaultStore bridges the vault.Client (in-memory index) with the store.Store (SQLite persistence). It converts between types.PageEntity/types.BlockEntity and store operations, enabling persistent indexing across sessions.

Design decision: The adapter pattern (VaultStore) was chosen over embedding store.Store directly in vault.Client to preserve separation of concerns (SOLID Single Responsibility). The vault.Client owns in-memory indexing; VaultStore owns the persistence bridge.

func NewVaultStore 

func NewVaultStore(s *store.Store, vaultPath, sourceID string) *VaultStore

NewVaultStore creates a VaultStore adapter for bridging vault and store. The sourceID identifies this vault in the store (typically "disk-local").

func (*VaultStore) FullIndex 

func (vs *VaultStore) FullIndex(c *Client) error

FullIndex performs a complete index of all vault files and persists to store. Used on first run when no .dewey/graph.db exists, or after corruption recovery.

func (*VaultStore) IncrementalIndex 

func (vs *VaultStore) IncrementalIndex(c *Client) (stats IndexStats, err error)

IncrementalIndex performs incremental indexing by comparing file content hashes against stored hashes. It returns counts of new, changed, deleted, and unchanged files.

Algorithm:

  1. Load stored content hashes from the store
  2. Walk the vault directory, computing content hashes for each .md file
  3. Compare: new files (not in store), changed files (hash differs), unchanged files (hash matches), deleted files (in store but not on disk)
  4. Re-index only new and changed files
  5. Remove deleted files from store

func (*VaultStore) LoadExternalPages added in v1.0.0 

func (vs *VaultStore) LoadExternalPages(c *Client) (int, error)

LoadExternalPages loads all non-local pages from the persistent store into the vault client's in-memory index. This makes external-source content (GitHub, web crawl) queryable via all MCP tools alongside local vault pages.

For each external page: converts store.Page → types.PageEntity, loads blocks via store.GetBlocksByPage(), reconstructs the block tree via reconstructBlockTree(), builds a cachedPage with sourceID and readOnly=true, and registers it via applyPageIndex(). Returns the number of pages loaded.

Design decision: External pages are marked readOnly=true to prevent write operations from modifying content that doesn't exist on disk (per research R7).

func (*VaultStore) LoadPages 

func (vs *VaultStore) LoadPages() (map[string]string, error)

LoadPages loads all persisted pages from the store and returns them as a map of content hashes keyed by page name. This is used for incremental indexing — comparing stored hashes against current file hashes.

IMPORTANT: Only loads pages from this vault's own source (vs.sourceID, typically "disk-local"). External-source pages (from dewey index) are excluded so that IncrementalIndex() does not treat them as deleted (they don't have corresponding .md files on disk).

func (*VaultStore) PersistPage 

func (vs *VaultStore) PersistPage(page *cachedPage) error

PersistPage writes a single page and its blocks/links to the store. It replaces any existing data for the page (delete + re-insert). Uses parameterized queries throughout (FR-028).

func (*VaultStore) RemovePage 

func (vs *VaultStore) RemovePage(pageName string) error

RemovePage deletes a page and its associated blocks/links from the store.

func (*VaultStore) SetEmbedder 

func (vs *VaultStore) SetEmbedder(e embed.Embedder)

SetEmbedder configures the embedding provider for the vault store. When set, embedding generation is integrated into the indexing pipeline. Pass nil to disable embedding generation (graceful degradation).

func (*VaultStore) SyncToStore 

func (vs *VaultStore) SyncToStore(pages map[string]*cachedPage) error

SyncToStore persists the entire in-memory index to the store. Used for first-run full-index and corruption recovery.

func (*VaultStore) ValidateStore 

func (vs *VaultStore) ValidateStore() error

ValidateStore checks the store's schema version and integrity. Returns nil if the store is valid, or an error describing the issue. Used for corruption detection (T018).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.