bluge

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

README

Bluge Bluge

PkgGoDev Tests Lint

modern text indexing in go - blugelabs.com

Installation

The module is published directly from this repository and does not require a replace directive:

go get github.com/fy0/bluge@master

import "github.com/fy0/bluge"

About This Branch

This branch is a performance- and correctness-focused fork of the official Bluge v0.2.2 baseline (5741419). It keeps the public document, writer, and query APIs familiar while replacing the segment implementation and optimizing the indexing and search hot paths.

Performance Comparison
Metric Current v2 Bluge Official Bleve v2.5.7
Writer configuration ordinary, batch 1,000 ordinary, batch 1,000 scorch, batch 1,000
Indexing time 2m2.235s average 2m39.744s 2m25.974s
Throughput 12,674 docs/s 9,688 docs/s 10,602 docs/s
Fresh LOCATION lifecycle 8.203 ms 39 ms 35 ms
Resident LOCATION median 1.630 ms 30.1 ms 16.6 ms
Peak Go Alloc 78.39 MiB 71.71 MiB 78.53 MiB
Peak Go Sys 118.05 MiB 101.14 MiB 108.46 MiB
Final index size 303.48 MiB average 475.40 MiB 536.47 MiB
Release .exe size 4.53 MiB / 4,749,824 bytes 4.60 MiB / 4,819,456 bytes 10.67 MiB / 11,192,320 bytes
Improvement Ratios

Official Bluge is normalized to 100%. Values below 100% are better for query time, index size, binary size, indexing time, and memory; values above 100% are better only for throughput.

Engine Query time Index size Binary size Indexing time Throughput Peak Alloc Peak Sys
Bluge Official 100% 100% 100% 100% 100% 100% 100%
Current v2 21.03% 63.84% 98.56% 76.52% 130.82% 109.32% 116.72%
Bleve v2.5.7 89.74% 112.85% 232.23% 91.38% 109.43% 109.51% 107.24%

Current v2 lowers the fresh query lifecycle by 78.97%, index size by 36.16%, and indexing time by 23.48% relative to official Bluge, while increasing throughput by 30.82%.

LanceDB And zvec
Metric Current v2 LanceDB positions LanceDB Match-only zvec v0.5.1 adapter
Build time 2m2.235s average 2m37.440s 1m32.239s 13m6.275s
Throughput 12,674 docs/s 9,830 docs/s 16,778 docs/s 1,968 docs/s
Fresh LOCATION lifecycle 8.203 ms 34 ms 32.5 ms 227 ms
Final directory 303.48 MiB average 669.00 MiB 242.95 MiB 1,549.43 MiB

LanceDB Match-only omits token positions and phrase-query support, so it is not feature-equivalent to the position-preserving configurations. The zvec adapter uses duplicate pre-tokenized fields and a compatibility vector. External engine builds are historical single runs on the same machine and workload; native RSS and Go heap samples are not directly comparable.

Exact Block-Max BM25 WAND
Query Total candidates Scored candidates Reduction Resident median p95
LOCATION 287,959 1,979 99.31% 1.630 ms 2.249 ms
THE 1,042,353 3,136 99.70% 1.684 ms 2.254 ms
LOCATION OR THE 1,330,312 146,084 89.02% not profiled not profiled

The build and directory figures are four-run averages. Query values are medians from ten fresh processes, with 20 resident searches per process and no result cache. The current v2 memory row is from the validation build; the other memory rows retain their original single-run measurements. WAND changes candidate traversal but preserves scoring semantics: full indexes matched the ordinary collector for Top-1/5/20/100 by document ID, tie order, and every bit of the float64 score.

See the detailed v1/v2 benchmark report for per-run data, root-cause evidence, methodology, validation commands, and remaining scheduling risks. The older standalone measurements remain in the historical benchmark report.

Branch Highlights
  • zapx-bluge v2 segment format - the default backend is a pure-Go, text-focused adaptation of zapx. It does not require FAISS or cgo.
  • Exact Block-Max BM25 WAND - high-cardinality terms persist 64-posting blocks with non-dominated frequency/raw-norm impacts. Score-descending Top-N term and OR queries use exact block upper bounds and a raw collector; unsupported similarities, sorts, aggregations, explanations, locations, and low-cardinality-only terms retain the ordinary collection path.
  • Compact text storage - stored fields are packed into Snappy-compressed blocks, integer posting chunks select raw or compressed encoding by size, and native numeric and document-value metadata use compact canonical forms.
  • Exact BM25 inputs - the standard BM25 IDF formula is corrected and guarded against invalid statistics. Segments persist exact per-field document counts and total term frequencies, including multi-valued fields.
  • Configured norms are preserved - custom DefaultSimilarity and PerFieldSimilarity norm calculations survive segment creation, persistence, and merges.
  • Native Bluge document build path - analyzed fields and token frequencies are exported directly into the segment builder. Custom segment.Document implementations continue to use the compatibility path.
  • Faster ordinary Writer persistence - a bounded set of newly persisted, read-only segments is reused instead of being immediately reopened and mmap'ed. Reuse is capped at four segments and 8 MiB; persistence, fsync, snapshot atomicity, and crash recovery are unchanged.
  • Parallel OfflineWriter - segment construction uses bounded parallelism, followed by bounded, multi-round merges that honor the configured merge fan-in.
  • Lower query overhead - collection statistics are read directly from the segment, BM25 can score raw stored norms, single-term queries avoid redundant work, Block-Max skips non-competitive postings without scoring them, and the common score-descending Top-N path avoids generic sort-value construction.
  • Stable _id behavior - _id remains stored, indexed, and available as a doc value by default.
  • 32-bit coverage - atomic counters used by the segment code are aligned and the core packages are tested with GOARCH=386.
Index Compatibility

The on-disk format is zapx-bluge v2. It is intentionally incompatible with zapx-bluge v1, the official ice-backed release, and the earlier experimental zap v17 format; those indexes must be rebuilt or migrated offline. The vector API and zapx vector section extension points remain available. The default build does not require FAISS or cgo and returns ErrVectorUnsupported unless an application supplies a vector backend.

The repository also includes opt-in vector backends. FlatVectorBackend is a cgo-free exact baseline with sidecar persistence, updates, deletes, and Bluge query filters. USearchVectorBackend is the first ANN integration. On 64-bit Windows, Linux, and macOS it loads a separately-built USearch 2.26 native library through purego, so the Go package remains buildable and runnable with CGO_ENABLED=0. See the vector search selection and implementation notes and the USearch native adapter instructions before deploying the ANN backend.

EmbeddedUSearchVectorBackend is the format-integrated variant for the read/write knowledge-base path. It stores one serialized USearch payload and its local-doc mapping in the zapx vector section of each segment; it does not create a per-field .usearch sidecar. Segment merges rebuild that payload after applying deletes and doc-number remapping. If the native library is missing while opening an existing index, text search remains available and vector search returns ErrVectorUnsupported; writing new vector documents still requires the native artifact.

For RAG ingestion, one Bluge Document should represent one chunk, matching the row model used by LanceDB and zvec. Give the chunk its own _id, index the chunk text for BM25, attach its embedding as a vector field, and store source_id, chunk position, and other metadata on that same document. Text and ANN results then fuse on the same chunk ID; the full source normally remains in an external source store instead of being indexed a second time. Bluge does not split source material or run an embedding model.

Writer.InsertMany and Writer.UpdateMany submit a slice of documents as one text-index batch and one vector mutation batch. OfflineWriter.InsertMany groups a large initial corpus by its configured batch size and is preferred for bulk construction. Native USearch additions cross the FFI boundary in bounded row-major batches rather than one call per vector. The default text-only path for Insert and Batch does not allocate vector changes or inspect document IDs, while still rejecting an accidental vector field before the text batch is accepted.

Vector requests can be composed without changing the text search API:

vector := bluge.NewVectorSearchRequest("embedding", query).SetK(10)
hits, err := reader.SearchVectorRequest(ctx, vector)

hybrid := bluge.NewHybridSearchRequest(
    bluge.NewMatchQuery("wireless headphones").SetField("title"),
    vector,
).SetK(10).SetFusion(bluge.HybridFusionRRF)
hybridHits, err := reader.HybridSearch(ctx, hybrid)

HybridSearch supports a Bluge filter, weighted score fusion, and reciprocal rank fusion. Bluge evaluates filters against the text snapshot and pushes the resulting native key set into USearch's filtered HNSW traversal without native callbacks into Go. The USearch native library is built and published independently by the usearch-ffi GitHub Actions workflow; it is not compiled by go build.

BM25 Scoring Modes

NewBM25Similarity() is the default and implements the canonical BM25 term score, including the (k1 + 1) numerator factor. It uses the number of documents containing a field when calculating that field's average length.

Two explicit compatibility modes are also available:

import "github.com/fy0/bluge/search/similarity"

config := bluge.DefaultConfig(path)

// Preserve scores produced by earlier versions of this Bluge fork.
config.DefaultSimilarity = similarity.NewLegacyBM25Similarity()

// Approximate Bleve v2.5.7 BM25 ranking for migration or comparison.
config.DefaultSimilarity = similarity.NewBleveBM25Similarity()

The Bleve mode reproduces its square-root term frequency, query normalization, global document-count statistics, and per-segment dictionary cardinality used as the average-length numerator. Its compatibility contract is result ordering for Term, Match AND/OR, Phrase, and Boolean text queries; raw scores are not portable between engines. All three built-in modes share the same on-disk norm encoding, so switching among them does not require rebuilding a zapx-bluge v2 index.

Features

  • Supported field types:
    • Text, Numeric, Date, Geo Point
  • Supported query types:
    • Term, Phrase, Match, Match Phrase, Prefix
    • Conjunction, Disjunction, Boolean
    • Numeric Range, Date Range
  • BM25 Similarity/Scoring with pluggable interfaces
  • Search result match highlighting
  • Extendable Aggregations:
    • Bucketing
      • Terms
      • Numeric Range
      • Date Range
    • Metrics
      • Min/Max/Count/Sum
      • Avg/Weighted Avg
      • Cardinality Estimation (HyperLogLog++)
      • Quantile Approximation (T-Digest)

Indexing

    config := bluge.DefaultConfig(path)
    writer, err := bluge.OpenWriter(config)
    if err != nil {
        log.Fatalf("error opening writer: %v", err)
    }
    defer writer.Close()

    doc := bluge.NewDocument("example").
        AddField(bluge.NewTextField("name", "bluge"))

    err = writer.Update(doc.ID(), doc)
    if err != nil {
        log.Fatalf("error updating document: %v", err)
    }

Querying

    reader, err := writer.Reader()
    if err != nil {
        log.Fatalf("error getting index reader: %v", err)
    }
    defer reader.Close()

    query := bluge.NewMatchQuery("bluge").SetField("name")
    request := bluge.NewTopNSearch(10, query).
        WithStandardAggregations()
    documentMatchIterator, err := reader.Search(context.Background(), request)
    if err != nil {
        log.Fatalf("error executing search: %v", err)
    }
    match, err := documentMatchIterator.Next()
    for err == nil && match != nil {
        err = match.VisitStoredFields(func(field string, value []byte) bool {
            if field == "_id" {
                fmt.Printf("match: %s\n", string(value))
            }
            return true
        })
        if err != nil {
            log.Fatalf("error loading stored fields: %v", err)
        }
        match, err = documentMatchIterator.Next()
    }
    if err != nil {
        log.Fatalf("error iterator document matches: %v", err)
    }

Repobeats

Alt

License

Apache License Version 2.0

Documentation

Overview

Package bluge is a library for indexing and searching text.

Example Opening New Index, Indexing Data 

config := bluge.DefaultConfig(path)
writer, err := bluge.OpenWriter(config)
if err != nil {
	log.Fatalf("error opening writer: %v", err)
}
defer writer.Close()

doc := bluge.NewDocument("example").
	AddField(bluge.NewTextField("name", "bluge"))

err = writer.Update(doc.ID(), doc)
if err != nil {
	log.Fatalf("error updating document: %v", err)
}

Example Getting Index Reader, Searching Data 

    reader, err := writer.Reader()
	if err != nil {
		log.Fatalf("error getting index reader: %v", err)
	}
	defer reader.Close()

	query := bluge.NewMatchQuery("bluge").SetField("name")
	request := bluge.NewTopNSearch(10, query).
		WithStandardAggregations()
	documentMatchIterator, err := reader.Search(context.Background(), request)
	if err != nil {
		log.Fatalf("error executing search: %v", err)
	}
	match, err := documentMatchIterator.Next()
	for err == nil && match != nil {

		// load the identifier for this match
		err = match.VisitStoredFields(func(field string, value []byte) bool {
			if field == "_id" {
				fmt.Printf("match: %s\n", string(value))
			}
			return true
		})
		if err != nil {
			log.Fatalf("error loading stored fields: %v", err)
		}
		match, err = documentMatchIterator.Next()
	}
	if err != nil {
		log.Fatalf("error iterator document matches: %v", err)
	}

Index

Constants

View Source 
const (
	// Document must satisfy AT LEAST ONE of term searches.
	MatchQueryOperatorOr = 0
	// Document must satisfy ALL of term searches.
	MatchQueryOperatorAnd = 1
)

Variables

View Source 
var (
	ErrVectorBackendReadOnly   = errors.New("vector backend is read-only")
	ErrVectorFieldNotFound     = errors.New("vector field was not found")
	ErrVectorInvalidDimension  = errors.New("vector dimension is invalid")
	ErrVectorInvalidK          = errors.New("vector k must be greater than zero")
	ErrVectorInvalidValue      = errors.New("vector contains NaN or infinity")
	ErrVectorInvalidField      = errors.New("vector field is invalid")
	ErrVectorFilterUnsupported = errors.New("vector backend does not support query filters")
)
View Source 
var (
	ErrVectorInvalidRequest = errors.New("vector search request is invalid")
	ErrHybridInvalidRequest = errors.New("hybrid search request is invalid")
)
View Source 
var ErrVectorUnsupported = errors.New("vector search is not enabled")
View Source 
var MaxNumeric = math.Inf(1)
View Source 
var MinNumeric = math.Inf(-1)

Functions

func DecodeDateTime 

func DecodeDateTime(value []byte) (time.Time, error)

func DecodeGeoLonLat 

func DecodeGeoLonLat(value []byte) (lon, lat float64, err error)

func DecodeNumericFloat64 

func DecodeNumericFloat64(value []byte) (float64, error)

func MultiSearch 

func MultiSearch(ctx context.Context, req SearchRequest, readers ...*Reader) (search.DocumentMatchIterator, error)

func NewBatch 

func NewBatch() *index.Batch

NewBatch creates a new empty batch.

Types

type AllMatches 

type AllMatches struct {
	BaseSearch
}

func NewAllMatches 

func NewAllMatches(q Query) *AllMatches

func (*AllMatches) AddAggregation 

func (s *AllMatches) AddAggregation(name string, aggregation search.Aggregation)

func (*AllMatches) Collector 

func (s *AllMatches) Collector() search.Collector

func (*AllMatches) ExplainScores 

func (s *AllMatches) ExplainScores() *AllMatches

func (*AllMatches) IncludeLocations 

func (s *AllMatches) IncludeLocations() *AllMatches

func (*AllMatches) WithStandardAggregations 

func (s *AllMatches) WithStandardAggregations() *AllMatches

type Analyzer 

type Analyzer interface {
	Analyze(input []byte) analysis.TokenStream
}

type BaseSearch 

type BaseSearch struct {
	// contains filtered or unexported fields
}

func (BaseSearch) Aggregations 

func (b BaseSearch) Aggregations() search.Aggregations

func (BaseSearch) Options 

func (b BaseSearch) Options() SearchOptions

func (BaseSearch) Query 

func (b BaseSearch) Query() Query

func (BaseSearch) Searcher 

func (b BaseSearch) Searcher(i search.Reader, config Config) (search.Searcher, error)

type BooleanQuery 

type BooleanQuery struct {
	// contains filtered or unexported fields
}

func NewBooleanQuery 

func NewBooleanQuery() *BooleanQuery

NewBooleanQuery creates a compound Query composed of several other Query objects. These other query objects are added using the AddMust() AddShould() and AddMustNot() methods. Result documents must satisfy ALL of the must Queries. Result documents must satisfy NONE of the must not Queries. Result documents that ALSO satisfy any of the should Queries will score higher.

func (*BooleanQuery) AddMust 

func (q *BooleanQuery) AddMust(m ...Query) *BooleanQuery

func (*BooleanQuery) AddMustNot 

func (q *BooleanQuery) AddMustNot(m ...Query) *BooleanQuery

func (*BooleanQuery) AddShould 

func (q *BooleanQuery) AddShould(m ...Query) *BooleanQuery

func (*BooleanQuery) Boost 

func (q *BooleanQuery) Boost() float64

func (*BooleanQuery) MinShould 

func (q *BooleanQuery) MinShould() int

MinShould returns the minimum number of should queries that need to match

func (*BooleanQuery) MustNots 

func (q *BooleanQuery) MustNots() []Query

MustNots returns queries that the documents must not match

func (*BooleanQuery) Musts 

func (q *BooleanQuery) Musts() []Query

Musts returns the queries that the documents must match

func (*BooleanQuery) Searcher 

func (q *BooleanQuery) Searcher(i search.Reader, options search.SearcherOptions) (rv search.Searcher, err error)

func (*BooleanQuery) SetBoost 

func (q *BooleanQuery) SetBoost(b float64) *BooleanQuery

func (*BooleanQuery) SetMinShould 

func (q *BooleanQuery) SetMinShould(minShould int) *BooleanQuery

SetMinShould requires that at least minShould of the should Queries must be satisfied.

func (*BooleanQuery) Shoulds 

func (q *BooleanQuery) Shoulds() []Query

Shoulds returns queries that the documents may match

func (*BooleanQuery) Validate 

func (q *BooleanQuery) Validate() error

type CompositeField 

type CompositeField struct {
	*TermField
	// contains filtered or unexported fields
}

func NewCompositeField 

func NewCompositeField(name string, defaultInclude bool, include, exclude []string) *CompositeField

func NewCompositeFieldExcluding 

func NewCompositeFieldExcluding(name string, excluding []string) *CompositeField

func NewCompositeFieldIncluding 

func NewCompositeFieldIncluding(name string, including []string) *CompositeField

func (*CompositeField) Analyze 

func (c *CompositeField) Analyze(int) int

func (*CompositeField) Consume 

func (c *CompositeField) Consume(field Field)

func (*CompositeField) EachTerm 

func (c *CompositeField) EachTerm(vt segment.VisitTerm)

func (*CompositeField) Length 

func (c *CompositeField) Length() int

func (*CompositeField) PositionIncrementGap 

func (c *CompositeField) PositionIncrementGap() int

func (*CompositeField) Size 

func (c *CompositeField) Size() int

type Config 

type Config struct {
	Logger *log.Logger

	DefaultSearchField    string
	DefaultSearchAnalyzer *analysis.Analyzer
	DefaultSimilarity     search.Similarity
	PerFieldSimilarity    map[string]search.Similarity
	VectorBackend         VectorBackend

	SearchStartFunc func(size uint64) error
	SearchEndFunc   func(size uint64)
	// contains filtered or unexported fields
}

func DefaultConfig 

func DefaultConfig(path string) Config

func DefaultConfigWithDirectory 

func DefaultConfigWithDirectory(df func() index.Directory) Config

func InMemoryOnlyConfig 

func InMemoryOnlyConfig() Config

func (Config) DisableOptimizeConjunction 

func (config Config) DisableOptimizeConjunction() Config

func (Config) DisableOptimizeConjunctionUnadorned 

func (config Config) DisableOptimizeConjunctionUnadorned() Config

func (Config) DisableOptimizeDisjunctionUnadorned 

func (config Config) DisableOptimizeDisjunctionUnadorned() Config

func (Config) WithOfflineWriterConcurrency 

func (config Config) WithOfflineWriterConcurrency(concurrency int) Config

WithOfflineWriterConcurrency sets the maximum number of concurrent segment builds and merge tasks used by OfflineWriter.

func (Config) WithSearchStartFunc 

func (config Config) WithSearchStartFunc(f func(size uint64) error) Config

func (Config) WithSegmentType 

func (config Config) WithSegmentType(typ string) Config

func (Config) WithSegmentVersion 

func (config Config) WithSegmentVersion(ver uint32) Config

func (Config) WithVectorBackend 

func (config Config) WithVectorBackend(backend VectorBackend) Config

func (Config) WithVirtualField 

func (config Config) WithVirtualField(field Field) Config

WithVirtualField allows you to describe a field that the index will behave as if all documents in this index were indexed with these field/terms, even though nothing is physically persisted about them in the index.

type DateRangeQuery 

type DateRangeQuery struct {
	// contains filtered or unexported fields
}

func NewDateRangeInclusiveQuery 

func NewDateRangeInclusiveQuery(start, end time.Time, startInclusive, endInclusive bool) *DateRangeQuery

NewDateRangeInclusiveQuery creates a new Query for ranges of date values. Date strings are parsed using the DateTimeParser configured in the 

top-level config.QueryDateTimeParser

Either, but not both endpoints can be nil. startInclusive and endInclusive control inclusion of the endpoints.

func NewDateRangeQuery 

func NewDateRangeQuery(start, end time.Time) *DateRangeQuery

NewDateRangeQuery creates a new Query for ranges of date values. Date strings are parsed using the DateTimeParser configured in the 

top-level config.QueryDateTimeParser

Either, but not both endpoints can be nil.

func (*DateRangeQuery) Boost 

func (q *DateRangeQuery) Boost() float64

func (*DateRangeQuery) End 

func (q *DateRangeQuery) End() (time.Time, bool)

End returns the date range end and if the end is included in the query

func (*DateRangeQuery) Field 

func (q *DateRangeQuery) Field() string

func (*DateRangeQuery) Searcher 

func (q *DateRangeQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*DateRangeQuery) SetBoost 

func (q *DateRangeQuery) SetBoost(b float64) *DateRangeQuery

func (*DateRangeQuery) SetField 

func (q *DateRangeQuery) SetField(f string) *DateRangeQuery

func (*DateRangeQuery) Start 

func (q *DateRangeQuery) Start() (time.Time, bool)

Start returns the date range start and if the start is included in the query

func (*DateRangeQuery) Validate 

func (q *DateRangeQuery) Validate() error

type Document 

type Document []Field

func NewDocument 

func NewDocument(id string) *Document

func NewDocumentWithIdentifier 

func NewDocumentWithIdentifier(id Identifier) *Document

func (*Document) AddField 

func (d *Document) AddField(f Field) *Document

func (Document) Analyze 

func (d Document) Analyze()

func (Document) EachField 

func (d Document) EachField(vf segment.VisitField)

func (Document) ID 

func (d Document) ID() segment.Term

ID is an experimental helper method to simplify common use cases

func (Document) Size 

func (d Document) Size() int

func (Document) ToBlugeIndexDocument 

func (d Document) ToBlugeIndexDocument() (*blugeidx.Document, error)

ToBlugeIndexDocument exports an analyzed document to the native segment build representation.

type EmbeddedUSearchVectorBackend added in v0.6.0 

type EmbeddedUSearchVectorBackend struct {
	// contains filtered or unexported fields
}

EmbeddedUSearchVectorBackend stores one serialized USearch index in every zapx segment. It is deliberately separate from USearchVectorBackend, whose global sidecar remains useful as a performance baseline.

func NewEmbeddedUSearchVectorBackend added in v0.6.0 

func NewEmbeddedUSearchVectorBackend() *EmbeddedUSearchVectorBackend

func (*EmbeddedUSearchVectorBackend) BuildVectorPayload added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) BuildVectorPayload(field string,
	records []zapxtext.VectorRecord) (zapxtext.VectorPayload, error)

func (*EmbeddedUSearchVectorBackend) MergeVectorPayload added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) MergeVectorPayload(field string,
	inputs []zapxtext.VectorMergeInput) (zapxtext.VectorPayload, error)

func (*EmbeddedUSearchVectorBackend) Name added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) Name() string

func (*EmbeddedUSearchVectorBackend) Open added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) Open(Config) (VectorIndex, error)

func (*EmbeddedUSearchVectorBackend) OpenSnapshot added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) OpenSnapshot(snapshot *index.Snapshot) (VectorIndex, error)

func (*EmbeddedUSearchVectorBackend) SegmentVectorBackend added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) SegmentVectorBackend()

func (*EmbeddedUSearchVectorBackend) ValidateVectorChanges added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) ValidateVectorChanges(changes []VectorChange) error

func (*EmbeddedUSearchVectorBackend) WithLibraryPath added in v0.6.0 

func (b *EmbeddedUSearchVectorBackend) WithLibraryPath(path string) *EmbeddedUSearchVectorBackend

type Field 

type Field interface {
	segment.Field

	Analyze(int) int
	AnalyzedTokenFrequencies() analysis.TokenFrequencies

	PositionIncrementGap() int

	Size() int
}

type FieldConsumer 

type FieldConsumer interface {
	Consume(Field)
}

FieldConsumer is anything which can consume a field Fields can implement this interface to consume the content of another field.

type FieldOptions 

type FieldOptions int
const (
	Index FieldOptions = 1 << iota
	Store
	SearchTermPositions
	HighlightMatches
	Sortable
	Aggregatable
)

func (FieldOptions) IncludeLocations 

func (o FieldOptions) IncludeLocations() bool

func (FieldOptions) Index 

func (o FieldOptions) Index() bool

func (FieldOptions) IndexDocValues 

func (o FieldOptions) IndexDocValues() bool

func (FieldOptions) Store 

func (o FieldOptions) Store() bool

type FlatVectorBackend added in v0.6.0 

type FlatVectorBackend struct {
	// contains filtered or unexported fields
}

FlatVectorBackend is the cgo-free exact-search backend. A non-empty path persists the vector sidecar; an empty path keeps the index in memory.

It is intentionally exact. That makes it useful as the correctness and recall baseline for a future FAISS, Rust, or WASM backend.

func NewFlatVectorBackend added in v0.6.0 

func NewFlatVectorBackend(path string) *FlatVectorBackend

NewFlatVectorBackend creates an exact vector backend. The sidecar is independent from the Bluge text snapshot and should normally live next to the configured index directory.

func (*FlatVectorBackend) Name added in v0.6.0 

func (b *FlatVectorBackend) Name() string

func (*FlatVectorBackend) Open added in v0.6.0 

func (b *FlatVectorBackend) Open(_ Config) (VectorIndex, error)

type FuzzyQuery 

type FuzzyQuery struct {
	// contains filtered or unexported fields
}

func NewFuzzyQuery 

func NewFuzzyQuery(term string) *FuzzyQuery

NewFuzzyQuery creates a new Query which finds documents containing terms within a specific fuzziness of the specified term. The default fuzziness is 1.

The current implementation uses Levenshtein edit distance as the fuzziness metric.

func (*FuzzyQuery) Boost 

func (q *FuzzyQuery) Boost() float64

func (*FuzzyQuery) Field 

func (q *FuzzyQuery) Field() string

func (*FuzzyQuery) Fuzziness 

func (q *FuzzyQuery) Fuzziness() int

Fuzziness returns the fuzziness of the query

func (*FuzzyQuery) Prefix 

func (q *FuzzyQuery) Prefix() int

PrefixLen returns the prefix match value

func (*FuzzyQuery) Searcher 

func (q *FuzzyQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*FuzzyQuery) SetBoost 

func (q *FuzzyQuery) SetBoost(b float64) *FuzzyQuery

func (*FuzzyQuery) SetField 

func (q *FuzzyQuery) SetField(f string) *FuzzyQuery

func (*FuzzyQuery) SetFuzziness 

func (q *FuzzyQuery) SetFuzziness(f int) *FuzzyQuery

func (*FuzzyQuery) SetPrefix 

func (q *FuzzyQuery) SetPrefix(p int) *FuzzyQuery

func (*FuzzyQuery) Term 

func (q *FuzzyQuery) Term() string

Term returns the term being queried

type GeoBoundingBoxQuery 

type GeoBoundingBoxQuery struct {
	// contains filtered or unexported fields
}

func NewGeoBoundingBoxQuery 

func NewGeoBoundingBoxQuery(topLeftLon, topLeftLat, bottomRightLon, bottomRightLat float64) *GeoBoundingBoxQuery

NewGeoBoundingBoxQuery creates a new Query for performing geo bounding box searches. The arguments describe the position of the box and documents which have an indexed geo point inside the box will be returned.

func (*GeoBoundingBoxQuery) Boost 

func (q *GeoBoundingBoxQuery) Boost() float64

func (*GeoBoundingBoxQuery) BottomRight 

func (q *GeoBoundingBoxQuery) BottomRight() []float64

BottomRight returns the end cornder of the bounding box

func (*GeoBoundingBoxQuery) Field 

func (q *GeoBoundingBoxQuery) Field() string

func (*GeoBoundingBoxQuery) Searcher 

func (q *GeoBoundingBoxQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*GeoBoundingBoxQuery) SetBoost 

func (q *GeoBoundingBoxQuery) SetBoost(b float64) *GeoBoundingBoxQuery

func (*GeoBoundingBoxQuery) SetField 

func (q *GeoBoundingBoxQuery) SetField(f string) *GeoBoundingBoxQuery

func (*GeoBoundingBoxQuery) TopLeft 

func (q *GeoBoundingBoxQuery) TopLeft() []float64

TopLeft returns the start corner of the bounding box

func (*GeoBoundingBoxQuery) Validate 

func (q *GeoBoundingBoxQuery) Validate() error

type GeoBoundingPolygonQuery 

type GeoBoundingPolygonQuery struct {
	// contains filtered or unexported fields
}

func NewGeoBoundingPolygonQuery 

func NewGeoBoundingPolygonQuery(points []geo.Point) *GeoBoundingPolygonQuery

FIXME document like the others

func (*GeoBoundingPolygonQuery) Boost 

func (q *GeoBoundingPolygonQuery) Boost() float64

func (*GeoBoundingPolygonQuery) Field 

func (q *GeoBoundingPolygonQuery) Field() string

func (*GeoBoundingPolygonQuery) Points 

func (q *GeoBoundingPolygonQuery) Points() []geo.Point

Points returns all the points being queried inside the bounding box

func (*GeoBoundingPolygonQuery) Searcher 

func (q *GeoBoundingPolygonQuery) Searcher(i search.Reader,
	options search.SearcherOptions) (search.Searcher, error)

func (*GeoBoundingPolygonQuery) SetBoost 

func (q *GeoBoundingPolygonQuery) SetBoost(b float64) *GeoBoundingPolygonQuery

func (*GeoBoundingPolygonQuery) SetField 

func (q *GeoBoundingPolygonQuery) SetField(f string) *GeoBoundingPolygonQuery

func (*GeoBoundingPolygonQuery) Validate 

func (q *GeoBoundingPolygonQuery) Validate() error

type GeoDistanceQuery 

type GeoDistanceQuery struct {
	// contains filtered or unexported fields
}

func NewGeoDistanceQuery 

func NewGeoDistanceQuery(lon, lat float64, distance string) *GeoDistanceQuery

NewGeoDistanceQuery creates a new Query for performing geo distance searches. The arguments describe a position and a distance. Documents which have an indexed geo point which is less than or equal to the provided distance from the given position will be returned.

func (*GeoDistanceQuery) Boost 

func (q *GeoDistanceQuery) Boost() float64

func (*GeoDistanceQuery) Distance 

func (q *GeoDistanceQuery) Distance() string

Distance returns the distance being queried

func (*GeoDistanceQuery) Field 

func (q *GeoDistanceQuery) Field() string

func (*GeoDistanceQuery) Location 

func (q *GeoDistanceQuery) Location() []float64

Location returns the location being queried

func (*GeoDistanceQuery) Searcher 

func (q *GeoDistanceQuery) Searcher(i search.Reader,
	options search.SearcherOptions) (search.Searcher, error)

func (*GeoDistanceQuery) SetBoost 

func (q *GeoDistanceQuery) SetBoost(b float64) *GeoDistanceQuery

func (*GeoDistanceQuery) SetField 

func (q *GeoDistanceQuery) SetField(f string) *GeoDistanceQuery

func (*GeoDistanceQuery) Validate 

func (q *GeoDistanceQuery) Validate() error

type HybridFusionMode added in v0.6.0 

type HybridFusionMode string

HybridFusionMode controls how text and vector candidates are combined. 

const (
	// HybridFusionWeighted normalizes each candidate list to [0, 1] and
	// combines the lists using TextWeight and VectorWeight.
	HybridFusionWeighted HybridFusionMode = "weighted"
	// HybridFusionRRF uses reciprocal rank fusion and is robust when text and
	// vector scores have different distributions.
	HybridFusionRRF HybridFusionMode = "rrf"
)

type HybridHit added in v0.6.0 

type HybridHit struct {
	ID          Identifier
	Score       float64
	TextScore   float64
	VectorScore float64
	TextRank    int
	VectorRank  int
}

HybridHit contains the raw scores from both branches and the fused score. A rank of zero means that the document was absent from that branch's candidate set.

type HybridSearchRequest added in v0.6.0 

type HybridSearchRequest struct {
	TextQuery Query
	Vector    *VectorSearchRequest
	Filter    Query

	K                int
	TextCandidates   int
	VectorCandidates int

	Fusion       HybridFusionMode
	TextWeight   float64
	VectorWeight float64
	RRFK         int
}

HybridSearchRequest combines a Bluge Query with a vector request. Either side may be omitted, which makes this type useful for gradual migration from text-only or vector-only search as well as true hybrid search.

func NewHybridSearchRequest added in v0.6.0 

func NewHybridSearchRequest(text Query, vector *VectorSearchRequest) *HybridSearchRequest

NewHybridSearchRequest creates a hybrid request from the two independent query objects. Both objects are optional, but at least one must be present when the request is executed.

func (*HybridSearchRequest) SetFilter added in v0.6.0 

func (r *HybridSearchRequest) SetFilter(filter Query) *HybridSearchRequest

func (*HybridSearchRequest) SetFusion added in v0.6.0 

func (r *HybridSearchRequest) SetFusion(mode HybridFusionMode) *HybridSearchRequest

func (*HybridSearchRequest) SetK added in v0.6.0 

func (r *HybridSearchRequest) SetK(k int) *HybridSearchRequest

func (*HybridSearchRequest) SetRRFK added in v0.6.0 

func (r *HybridSearchRequest) SetRRFK(k int) *HybridSearchRequest

func (*HybridSearchRequest) SetTextCandidates added in v0.6.0 

func (r *HybridSearchRequest) SetTextCandidates(candidates int) *HybridSearchRequest

func (*HybridSearchRequest) SetVectorCandidates added in v0.6.0 

func (r *HybridSearchRequest) SetVectorCandidates(candidates int) *HybridSearchRequest

func (*HybridSearchRequest) SetWeights added in v0.6.0 

func (r *HybridSearchRequest) SetWeights(text, vector float64) *HybridSearchRequest

type Identifier 

type Identifier string

func (Identifier) Field 

func (i Identifier) Field() string

func (Identifier) Term 

func (i Identifier) Term() []byte

type MatchAllQuery 

type MatchAllQuery struct {
	// contains filtered or unexported fields
}

func NewMatchAllQuery 

func NewMatchAllQuery() *MatchAllQuery

NewMatchAllQuery creates a Query which will match all documents in the index.

func (*MatchAllQuery) Boost 

func (q *MatchAllQuery) Boost() float64

func (*MatchAllQuery) Searcher 

func (q *MatchAllQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*MatchAllQuery) SetBoost 

func (q *MatchAllQuery) SetBoost(b float64) *MatchAllQuery

type MatchNoneQuery 

type MatchNoneQuery struct {
	// contains filtered or unexported fields
}

func NewMatchNoneQuery 

func NewMatchNoneQuery() *MatchNoneQuery

NewMatchNoneQuery creates a Query which will not match any documents in the index.

func (*MatchNoneQuery) Boost 

func (q *MatchNoneQuery) Boost() float64

func (*MatchNoneQuery) Searcher 

func (q *MatchNoneQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*MatchNoneQuery) SetBoost 

func (q *MatchNoneQuery) SetBoost(b float64) *MatchNoneQuery

type MatchPhraseQuery 

type MatchPhraseQuery struct {
	// contains filtered or unexported fields
}

func NewMatchPhraseQuery 

func NewMatchPhraseQuery(matchPhrase string) *MatchPhraseQuery

NewMatchPhraseQuery creates a new Query object for matching phrases in the index. An Analyzer is chosen based on the field. Input text is analyzed using this analyzer. Token terms resulting from this analysis are used to build a search phrase. Result documents must match this phrase. Queried field must have been indexed with IncludeTermVectors set to true.

func (*MatchPhraseQuery) Analyzer 

func (q *MatchPhraseQuery) Analyzer() *analysis.Analyzer

func (*MatchPhraseQuery) Boost 

func (q *MatchPhraseQuery) Boost() float64

func (*MatchPhraseQuery) Field 

func (q *MatchPhraseQuery) Field() string

func (*MatchPhraseQuery) Phrase 

func (q *MatchPhraseQuery) Phrase() string

Phrase returns the phrase being queried

func (*MatchPhraseQuery) Searcher 

func (q *MatchPhraseQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*MatchPhraseQuery) SetAnalyzer 

func (q *MatchPhraseQuery) SetAnalyzer(a *analysis.Analyzer) *MatchPhraseQuery

func (*MatchPhraseQuery) SetBoost 

func (q *MatchPhraseQuery) SetBoost(b float64) *MatchPhraseQuery

func (*MatchPhraseQuery) SetField 

func (q *MatchPhraseQuery) SetField(f string) *MatchPhraseQuery

func (*MatchPhraseQuery) SetSlop 

func (q *MatchPhraseQuery) SetSlop(dist int) *MatchPhraseQuery

SetSlop updates the sloppyness of the query the phrase terms can be as "dist" terms away from each other

func (*MatchPhraseQuery) Slop 

func (q *MatchPhraseQuery) Slop() int

Slop returns the acceptable distance between tokens

type MatchQuery 

type MatchQuery struct {
	// contains filtered or unexported fields
}

func NewMatchQuery 

func NewMatchQuery(match string) *MatchQuery

NewMatchQuery creates a Query for matching text. An Analyzer is chosen based on the field. Input text is analyzed using this analyzer. Token terms resulting from this analysis are used to perform term searches. Result documents must satisfy at least one of these term searches.

func (*MatchQuery) Analyzer 

func (q *MatchQuery) Analyzer() *analysis.Analyzer

func (*MatchQuery) Boost 

func (q *MatchQuery) Boost() float64

func (*MatchQuery) Field 

func (q *MatchQuery) Field() string

func (*MatchQuery) Fuzziness 

func (q *MatchQuery) Fuzziness() int

func (*MatchQuery) Match 

func (q *MatchQuery) Match() string

Match returns the term being queried

func (*MatchQuery) Operator 

func (q *MatchQuery) Operator() MatchQueryOperator

func (*MatchQuery) Prefix 

func (q *MatchQuery) Prefix() int

func (*MatchQuery) Searcher 

func (q *MatchQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*MatchQuery) SetAnalyzer 

func (q *MatchQuery) SetAnalyzer(a *analysis.Analyzer) *MatchQuery

func (*MatchQuery) SetBoost 

func (q *MatchQuery) SetBoost(b float64) *MatchQuery

func (*MatchQuery) SetField 

func (q *MatchQuery) SetField(f string) *MatchQuery

func (*MatchQuery) SetFuzziness 

func (q *MatchQuery) SetFuzziness(f int) *MatchQuery

func (*MatchQuery) SetOperator 

func (q *MatchQuery) SetOperator(operator MatchQueryOperator) *MatchQuery

func (*MatchQuery) SetPrefix 

func (q *MatchQuery) SetPrefix(p int) *MatchQuery

type MatchQueryOperator 

type MatchQueryOperator int

type MultiPhraseQuery 

type MultiPhraseQuery struct {
	// contains filtered or unexported fields
}

func NewMultiPhraseQuery 

func NewMultiPhraseQuery(terms [][]string) *MultiPhraseQuery

NewMultiPhraseQuery creates a new Query for finding term phrases in the index. It is like PhraseQuery, but each position in the phrase may be satisfied by a list of terms as opposed to just one. At least one of the terms must exist in the correct order, at the correct index offsets, in the specified field. Queried field must have been indexed with IncludeTermVectors set to true.

func (*MultiPhraseQuery) Boost 

func (q *MultiPhraseQuery) Boost() float64

func (*MultiPhraseQuery) Field 

func (q *MultiPhraseQuery) Field() string

func (*MultiPhraseQuery) Searcher 

func (q *MultiPhraseQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*MultiPhraseQuery) SetBoost 

func (q *MultiPhraseQuery) SetBoost(b float64) *MultiPhraseQuery

func (*MultiPhraseQuery) SetField 

func (q *MultiPhraseQuery) SetField(f string) *MultiPhraseQuery

func (*MultiPhraseQuery) SetSlop 

func (q *MultiPhraseQuery) SetSlop(dist int) *MultiPhraseQuery

SetSlop updates the sloppyness of the query the phrase terms can be as "dist" terms away from each other

func (*MultiPhraseQuery) Slop 

func (q *MultiPhraseQuery) Slop() int

Slop returns the acceptable distance between terms

func (*MultiPhraseQuery) Terms 

func (q *MultiPhraseQuery) Terms() [][]string

Terms returns the term phrases being queried

func (*MultiPhraseQuery) Validate 

func (q *MultiPhraseQuery) Validate() error

type MultiSearcherList 

type MultiSearcherList struct {
	// contains filtered or unexported fields
}

func NewMultiSearcherList 

func NewMultiSearcherList(searchers []search.Searcher) *MultiSearcherList

func (*MultiSearcherList) Close 

func (m *MultiSearcherList) Close() (err error)

func (*MultiSearcherList) DocumentMatchPoolSize 

func (m *MultiSearcherList) DocumentMatchPoolSize() int

func (*MultiSearcherList) Next 

func (m *MultiSearcherList) Next(ctx *search.Context) (*search.DocumentMatch, error)

type NumericRangeQuery 

type NumericRangeQuery struct {
	// contains filtered or unexported fields
}

func NewNumericRangeInclusiveQuery 

func NewNumericRangeInclusiveQuery(min, max float64, minInclusive, maxInclusive bool) *NumericRangeQuery

NewNumericRangeInclusiveQuery creates a new Query for ranges of numeric values. Either, but not both endpoints can be nil. Control endpoint inclusion with inclusiveMin, inclusiveMax.

func NewNumericRangeQuery 

func NewNumericRangeQuery(min, max float64) *NumericRangeQuery

NewNumericRangeQuery creates a new Query for ranges of numeric values. Either, but not both endpoints can be nil. The minimum value is inclusive. The maximum value is exclusive.

func (*NumericRangeQuery) Boost 

func (q *NumericRangeQuery) Boost() float64

func (*NumericRangeQuery) Field 

func (q *NumericRangeQuery) Field() string

func (*NumericRangeQuery) Max 

func (q *NumericRangeQuery) Max() (float64, bool)

Max returns the numeric range upperbound and if the upperbound is included

func (*NumericRangeQuery) Min 

func (q *NumericRangeQuery) Min() (float64, bool)

Min returns the numeric range lower bound and if the lowerbound is included

func (*NumericRangeQuery) Searcher 

func (q *NumericRangeQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*NumericRangeQuery) SetBoost 

func (q *NumericRangeQuery) SetBoost(b float64) *NumericRangeQuery

func (*NumericRangeQuery) SetField 

func (q *NumericRangeQuery) SetField(f string) *NumericRangeQuery

func (*NumericRangeQuery) Validate 

func (q *NumericRangeQuery) Validate() error

type OfflineWriter 

type OfflineWriter struct {
	// contains filtered or unexported fields
}

func OpenOfflineWriter 

func OpenOfflineWriter(config Config, batchSize, maxSegmentsToMerge int) (*OfflineWriter, error)

func (*OfflineWriter) Close 

func (w *OfflineWriter) Close() error

func (*OfflineWriter) Insert 

func (w *OfflineWriter) Insert(doc segment.Document) error

Insert transfers the document to the offline writer. Full batches are built asynchronously; a background build error is returned by a later Insert or Close.

func (*OfflineWriter) InsertMany added in v0.6.0 

func (w *OfflineWriter) InsertMany(documents []*Document) error

InsertMany transfers documents to the offline writer. Documents are grouped into the configured batch size so segment construction remains bounded and can run concurrently.

type PrefixQuery 

type PrefixQuery struct {
	// contains filtered or unexported fields
}

func NewPrefixQuery 

func NewPrefixQuery(prefix string) *PrefixQuery

NewPrefixQuery creates a new Query which finds documents containing terms that start with the specified prefix.

func (*PrefixQuery) Boost 

func (q *PrefixQuery) Boost() float64

func (*PrefixQuery) Field 

func (q *PrefixQuery) Field() string

func (*PrefixQuery) Prefix 

func (q *PrefixQuery) Prefix() string

Prefix return the prefix being queried

func (*PrefixQuery) Searcher 

func (q *PrefixQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*PrefixQuery) SetBoost 

func (q *PrefixQuery) SetBoost(b float64) *PrefixQuery

func (*PrefixQuery) SetField 

func (q *PrefixQuery) SetField(f string) *PrefixQuery

type Query 

type Query interface {
	Searcher(i search.Reader,
		options search.SearcherOptions) (search.Searcher, error)
}

A Query represents a description of the type and parameters for a query into the index.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

func OpenReader 

func OpenReader(config Config) (*Reader, error)

func (*Reader) Backup 

func (r *Reader) Backup(path string, cancel chan struct{}) error

func (*Reader) Close 

func (r *Reader) Close() error

func (*Reader) Count 

func (r *Reader) Count() (count uint64, err error)

func (*Reader) DictionaryIterator 

func (r *Reader) DictionaryIterator(field string, automaton segment.Automaton, start, end []byte) (segment.DictionaryIterator, error)

func (*Reader) Fields 

func (r *Reader) Fields() (fields []string, err error)

func (*Reader) HybridSearch added in v0.6.0 

func (r *Reader) HybridSearch(ctx context.Context,
	request *HybridSearchRequest) ([]HybridHit, error)

HybridSearch executes text and vector retrieval independently, then fuses their candidate sets in Go. The native backend is deliberately unaware of Bluge queries and filters.

func (*Reader) Search 

func (r *Reader) Search(ctx context.Context, req SearchRequest) (search.DocumentMatchIterator, error)

func (*Reader) SearchVector added in v0.6.0 

func (r *Reader) SearchVector(ctx context.Context, field string, query []float32,
	k int, filter Query) ([]VectorHit, error)

SearchVector is kept as a discoverable alias for callers that use verb-first naming alongside Reader.Search.

func (*Reader) SearchVectorRequest added in v0.6.0 

func (r *Reader) SearchVectorRequest(ctx context.Context,
	request *VectorSearchRequest) ([]VectorHit, error)

SearchVectorRequest executes a vector request and applies the final K after candidate retrieval. Candidates is useful for hybrid search and for ANN recall tuning; zero means exactly K candidates.

func (*Reader) VectorSearch added in v0.6.0 

func (r *Reader) VectorSearch(ctx context.Context, field string, query []float32,
	k int, filter Query) ([]VectorHit, error)

VectorSearch returns the nearest vectors for a field. A non-nil filter is evaluated by Bluge first, then applied to the vector candidates.

func (*Reader) VisitStoredFields 

func (r *Reader) VisitStoredFields(number uint64, visitor StoredFieldVisitor) error

type RegexpQuery 

type RegexpQuery struct {
	// contains filtered or unexported fields
}

func NewRegexpQuery 

func NewRegexpQuery(regexp string) *RegexpQuery

NewRegexpQuery creates a new Query which finds documents containing terms that match the specified regular expression.

func (*RegexpQuery) Boost 

func (q *RegexpQuery) Boost() float64

func (*RegexpQuery) Field 

func (q *RegexpQuery) Field() string

func (*RegexpQuery) Regexp 

func (q *RegexpQuery) Regexp() string

Regexp returns the regular expression being queried

func (*RegexpQuery) Searcher 

func (q *RegexpQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*RegexpQuery) SetBoost 

func (q *RegexpQuery) SetBoost(b float64) *RegexpQuery

func (*RegexpQuery) SetField 

func (q *RegexpQuery) SetField(f string) *RegexpQuery

func (*RegexpQuery) Validate 

func (q *RegexpQuery) Validate() error

type SearchOptions 

type SearchOptions struct {
	ExplainScores    bool
	IncludeLocations bool
	Score            string // FIXME go away
}

type SearchRequest 

type SearchRequest interface {
	Collector() search.Collector
	Searcher(i search.Reader, config Config) (search.Searcher, error)
	AddAggregation(name string, aggregation search.Aggregation)
	Aggregations() search.Aggregations
}

type StoredFieldVisitor 

type StoredFieldVisitor func(field string, value []byte) bool

type TermField 

type TermField struct {
	FieldOptions
	// contains filtered or unexported fields
}

func NewDateTimeField 

func NewDateTimeField(name string, dt time.Time) *TermField

func NewGeoPointField 

func NewGeoPointField(name string, lon, lat float64) *TermField

func NewKeywordField 

func NewKeywordField(name, value string) *TermField

func NewKeywordFieldBytes 

func NewKeywordFieldBytes(name string, value []byte) *TermField

func NewNumericField 

func NewNumericField(name string, number float64) *TermField

func NewStoredOnlyField 

func NewStoredOnlyField(name string, value []byte) *TermField

func NewTextField 

func NewTextField(name, value string) *TermField

func NewTextFieldBytes 

func NewTextFieldBytes(name string, value []byte) *TermField

func (*TermField) Aggregatable 

func (b *TermField) Aggregatable() *TermField

func (*TermField) Analyze 

func (b *TermField) Analyze(startOffset int) (lastPos int)

func (*TermField) AnalyzedLength 

func (b *TermField) AnalyzedLength() int

func (*TermField) AnalyzedTokenFrequencies 

func (b *TermField) AnalyzedTokenFrequencies() analysis.TokenFrequencies

func (*TermField) EachTerm 

func (b *TermField) EachTerm(vt segment.VisitTerm)

func (*TermField) HighlightMatches 

func (b *TermField) HighlightMatches() *TermField

func (*TermField) Length 

func (b *TermField) Length() int

func (*TermField) Name 

func (b *TermField) Name() string

func (*TermField) NumPlainTextBytes 

func (b *TermField) NumPlainTextBytes() int

func (*TermField) PositionIncrementGap 

func (b *TermField) PositionIncrementGap() int

func (*TermField) SearchTermPositions 

func (b *TermField) SearchTermPositions() *TermField

func (*TermField) SetPositionIncrementGap 

func (b *TermField) SetPositionIncrementGap(positionIncrementGap int) *TermField

func (*TermField) Size 

func (b *TermField) Size() int

func (*TermField) Sortable 

func (b *TermField) Sortable() *TermField

func (*TermField) StoreValue 

func (b *TermField) StoreValue() *TermField

func (*TermField) Value 

func (b *TermField) Value() []byte

func (*TermField) WithAnalyzer 

func (b *TermField) WithAnalyzer(fieldAnalyzer Analyzer) *TermField

type TermQuery 

type TermQuery struct {
	// contains filtered or unexported fields
}

func NewTermQuery 

func NewTermQuery(term string) *TermQuery

NewTermQuery creates a new Query for finding an exact term match in the index.

func (*TermQuery) Boost 

func (q *TermQuery) Boost() float64

func (*TermQuery) Field 

func (q *TermQuery) Field() string

func (*TermQuery) Searcher 

func (q *TermQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*TermQuery) SetBoost 

func (q *TermQuery) SetBoost(b float64) *TermQuery

func (*TermQuery) SetField 

func (q *TermQuery) SetField(f string) *TermQuery

func (*TermQuery) Term 

func (q *TermQuery) Term() string

Term returns the exact term being queried

type TermRangeQuery 

type TermRangeQuery struct {
	// contains filtered or unexported fields
}

func NewTermRangeInclusiveQuery 

func NewTermRangeInclusiveQuery(min, max string, minInclusive, maxInclusive bool) *TermRangeQuery

NewTermRangeInclusiveQuery creates a new Query for ranges of text terms. Either, but not both endpoints can be "". Control endpoint inclusion with inclusiveMin, inclusiveMax.

func NewTermRangeQuery 

func NewTermRangeQuery(min, max string) *TermRangeQuery

NewTermRangeQuery creates a new Query for ranges of text terms. Either, but not both endpoints can be "". The minimum value is inclusive. The maximum value is exclusive.

func (*TermRangeQuery) Boost 

func (q *TermRangeQuery) Boost() float64

func (*TermRangeQuery) Field 

func (q *TermRangeQuery) Field() string

func (*TermRangeQuery) Max 

func (q *TermRangeQuery) Max() (string, bool)

Max returns the query upperbound and if the upper bound is included in the query

func (*TermRangeQuery) Min 

func (q *TermRangeQuery) Min() (string, bool)

Min returns the query lower bound and if the lower bound is included in query

func (*TermRangeQuery) Searcher 

func (q *TermRangeQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*TermRangeQuery) SetBoost 

func (q *TermRangeQuery) SetBoost(b float64) *TermRangeQuery

func (*TermRangeQuery) SetField 

func (q *TermRangeQuery) SetField(f string) *TermRangeQuery

func (*TermRangeQuery) Validate 

func (q *TermRangeQuery) Validate() error

type TopNSearch 

type TopNSearch struct {
	BaseSearch
	// contains filtered or unexported fields
}

TopNSearch is used to search for a fixed number of matches which can be sorted by a custom sort order. It also allows for skipping a specified number of matches which can be used to enable pagination.

func NewTopNSearch 

func NewTopNSearch(n int, q Query) *TopNSearch

NewTopNSearch creates a search which will find the matches and return the first N when ordered by the specified sort order (default: score descending)

func (*TopNSearch) AddAggregation 

func (s *TopNSearch) AddAggregation(name string, aggregation search.Aggregation)

func (*TopNSearch) After 

func (s *TopNSearch) After(after [][]byte) *TopNSearch

After can be used to specify a sort key, any match with a sort key less than this will be skipped

func (*TopNSearch) AllMatches 

func (s *TopNSearch) AllMatches(i search.Reader, config Config) (search.Searcher, error)

func (*TopNSearch) Before 

func (s *TopNSearch) Before(before [][]byte) *TopNSearch

Before can be used to specify a sort key, any match with a sort key greather than this will be skipped

func (*TopNSearch) Collector 

func (s *TopNSearch) Collector() search.Collector

func (*TopNSearch) ExplainScores 

func (s *TopNSearch) ExplainScores() *TopNSearch

ExplainScores enables the addition of scoring explanation to each match

func (*TopNSearch) From 

func (s *TopNSearch) From() int

From returns the number of matches that will be skipped

func (*TopNSearch) IncludeLocations 

func (s *TopNSearch) IncludeLocations() *TopNSearch

IncludeLocations enables the addition of match location in the original field

func (*TopNSearch) SetFrom 

func (s *TopNSearch) SetFrom(from int) *TopNSearch

SetFrom sets the number of results to skip

func (*TopNSearch) SetScore 

func (s *TopNSearch) SetScore(mode string) *TopNSearch

func (*TopNSearch) Size 

func (s *TopNSearch) Size() int

Size returns the number of matches this search request will return

func (*TopNSearch) SortBy 

func (s *TopNSearch) SortBy(order []string) *TopNSearch

SortBy is a convenience method to specify search result sort order using a simple string slice. Strings in the slice are interpreted as the name of a field to sort ascending. The following special cases are handled.

  • the prefix '-' will sort in descending order
  • the special field '_score' can be used sort by score

func (*TopNSearch) SortByCustom 

func (s *TopNSearch) SortByCustom(order search.SortOrder) *TopNSearch

SortByCustom sets a custom sort order used to sort the matches of the search

func (*TopNSearch) SortOrder 

func (s *TopNSearch) SortOrder() search.SortOrder

SortOrder returns the sort order of the current search

func (*TopNSearch) WithStandardAggregations 

func (s *TopNSearch) WithStandardAggregations() *TopNSearch

WithStandardAggregations adds the standard aggregations in the search query The standard aggregations are:

  • count (total number of documents that matched the query)
  • max_score (the highest score of all the matched documents)
  • duration (time taken performing the search)

type USearchHardwareInfo added in v0.6.0 

type USearchHardwareInfo struct {
	Compiled  string
	Available string
}

USearchHardwareInfo reports the ISA families compiled into the native library and the subset available on the current CPU. It is useful for diagnosing a serial fallback or a mismatched release artifact.

type USearchVectorBackend added in v0.6.0 

type USearchVectorBackend struct {
	// contains filtered or unexported fields
}

USearchVectorBackend stores one USearch HNSW index per vector field. path is a directory containing the manifest and native index files. The native library is loaded through a small C ABI and can be supplied explicitly with NewUSearchVectorBackendWithLibrary or BLUGE_USEARCH_LIBRARY_PATH.

The algorithm, graph construction, distance kernels, deletion and native serialization are provided by USearch. This type owns only the Bluge ID mapping, field metadata, and the sidecar lifecycle.

func NewUSearchVectorBackend added in v0.6.0 

func NewUSearchVectorBackend(path string) *USearchVectorBackend

NewUSearchVectorBackend creates a persistent USearch backend. The native library is resolved from BLUGE_USEARCH_LIBRARY_PATH, the executable directory, or the platform library search path.

func NewUSearchVectorBackendWithLibrary added in v0.6.0 

func NewUSearchVectorBackendWithLibrary(path, libraryPath string) *USearchVectorBackend

NewUSearchVectorBackendWithLibrary creates a persistent backend with an explicit path to the separately-built native library.

func NewUSearchVectorBackendWithOptions added in v0.6.0 

func NewUSearchVectorBackendWithOptions(path string, options USearchVectorOptions) *USearchVectorBackend

NewUSearchVectorBackendWithOptions creates a backend with explicit HNSW construction and search parameters.

func (*USearchVectorBackend) HardwareAcceleration added in v0.6.0 

func (b *USearchVectorBackend) HardwareAcceleration() (USearchHardwareInfo, error)

HardwareAcceleration inspects the separately-built native library without opening a vector sidecar. Older ABI-compatible libraries may not expose the optional probe symbols; those return empty strings.

func (*USearchVectorBackend) Name added in v0.6.0 

func (b *USearchVectorBackend) Name() string

func (*USearchVectorBackend) Open added in v0.6.0 

func (b *USearchVectorBackend) Open(_ Config) (VectorIndex, error)

func (*USearchVectorBackend) WithLibraryPath added in v0.6.0 

func (b *USearchVectorBackend) WithLibraryPath(libraryPath string) *USearchVectorBackend

WithLibraryPath returns a copy configured to load libraryPath.

type USearchVectorOptions added in v0.6.0 

type USearchVectorOptions struct {
	Connectivity    int
	ExpansionAdd    int
	ExpansionSearch int
}

USearchVectorOptions controls the HNSW index created for new vector fields. Zero values use the USearch defaults selected by this adapter.

type VectorBackend 

type VectorBackend interface {
	Name() string
	Open(config Config) (VectorIndex, error)
}

type VectorBatchValidator added in v0.6.0 

type VectorBatchValidator interface {
	ValidateVectorChanges(changes []VectorChange) error
}

VectorBatchValidator can reject a batch before the text index is mutated. Backends should implement it when dimension or metric validation depends on the current index state.

type VectorBatcher added in v0.6.0 

type VectorBatcher interface {
	ApplyVectorChanges(changes []VectorChange) error
}

VectorBatcher is implemented by backends that can receive document changes. A delete with an empty Field removes every vector belonging to the ID.

type VectorCandidateSearcher added in v0.6.0 

type VectorCandidateSearcher interface {
	SearchCandidates(field string, query []float32, k int,
		allowed map[Identifier]struct{}) ([]VectorHit, error)
}

VectorCandidateSearcher lets Reader apply a Bluge Query filter before the backend ranks candidates. It avoids asking an ANN/native backend to know about Bluge's query language.

type VectorChange added in v0.6.0 

type VectorChange struct {
	ID         Identifier
	Field      string
	Vector     []float32
	Similarity VectorSimilarity
	Delete     bool
}

VectorChange is the mutation sent to a writable vector backend after the corresponding text batch has been accepted.

type VectorChangeValidator added in v0.6.0 

type VectorChangeValidator = VectorBatchValidator

VectorChangeValidator is the segment-integrated spelling of VectorBatchValidator. It is an alias because both validators run before the text batch is submitted and share the same contract.

type VectorField added in v0.6.0 

type VectorField struct {
	// contains filtered or unexported fields
}

VectorField stores an embedding on a document without adding it to the text segment. A configured VectorBackend receives it from Writer.

func NewVectorField added in v0.6.0 

func NewVectorField(name string, vector []float32) *VectorField

NewVectorField creates a cosine-similarity vector field.

func NewVectorFieldWithSimilarity added in v0.6.0 

func NewVectorFieldWithSimilarity(name string, vector []float32,
	similarity VectorSimilarity) *VectorField

NewVectorFieldWithSimilarity creates a vector field using the requested scoring function. The vector is copied so later caller mutations cannot change a pending document.

func (*VectorField) Analyze added in v0.6.0 

func (f *VectorField) Analyze(int) int

func (*VectorField) AnalyzedTokenFrequencies added in v0.6.0 

func (f *VectorField) AnalyzedTokenFrequencies() analysis.TokenFrequencies

func (*VectorField) EachTerm added in v0.6.0 

func (f *VectorField) EachTerm(segment.VisitTerm)

func (*VectorField) Index added in v0.6.0 

func (f *VectorField) Index() bool

func (*VectorField) IndexDocValues added in v0.6.0 

func (f *VectorField) IndexDocValues() bool

func (*VectorField) Length added in v0.6.0 

func (f *VectorField) Length() int

func (*VectorField) Name added in v0.6.0 

func (f *VectorField) Name() string

func (*VectorField) PositionIncrementGap added in v0.6.0 

func (f *VectorField) PositionIncrementGap() int

func (*VectorField) Size added in v0.6.0 

func (f *VectorField) Size() int

func (*VectorField) Store added in v0.6.0 

func (f *VectorField) Store() bool

func (*VectorField) Value added in v0.6.0 

func (f *VectorField) Value() []byte

func (*VectorField) VectorSimilarity added in v0.6.0 

func (f *VectorField) VectorSimilarity() VectorSimilarity

VectorSimilarity returns the score function associated with the field.

func (*VectorField) VectorValue added in v0.6.0 

func (f *VectorField) VectorValue() []float32

VectorValue returns a copy of the embedding associated with the field.

type VectorFieldSpec 

type VectorFieldSpec struct {
	Name       string
	Dims       int
	Similarity VectorSimilarity
}

type VectorHit 

type VectorHit struct {
	ID    Identifier
	Score float64
}

type VectorIndex 

type VectorIndex interface {
	Search(field string, query []float32, k int, filter Query) ([]VectorHit, error)
	Close() error
}

type VectorSearchRequest added in v0.6.0 

type VectorSearchRequest struct {
	Field      string
	Vector     []float32
	K          int
	Candidates int
	Filter     Query
}

VectorSearchRequest is the fluent form of a vector search. The existing Reader.VectorSearch method remains available for callers that do not need candidate tuning.

func NewVectorSearchRequest added in v0.6.0 

func NewVectorSearchRequest(field string, vector []float32) *VectorSearchRequest

NewVectorSearchRequest creates a vector request with a default result size of ten. The vector is copied so callers can reuse their input buffer.

func (*VectorSearchRequest) SetCandidates added in v0.6.0 

func (r *VectorSearchRequest) SetCandidates(candidates int) *VectorSearchRequest

func (*VectorSearchRequest) SetFilter added in v0.6.0 

func (r *VectorSearchRequest) SetFilter(filter Query) *VectorSearchRequest

func (*VectorSearchRequest) SetK added in v0.6.0 

func (r *VectorSearchRequest) SetK(k int) *VectorSearchRequest

type VectorSimilarity 

type VectorSimilarity string
const (
	VectorL2     VectorSimilarity = "l2_norm"
	VectorDot    VectorSimilarity = "dot_product"
	VectorCosine VectorSimilarity = "cosine"
)

type VectorSnapshotter added in v0.6.0 

type VectorSnapshotter interface {
	SnapshotVectorIndex() VectorIndex
}

VectorSnapshotter creates a read-only point-in-time view for Reader values obtained from a live Writer. It is useful for in-memory backends whose Open method intentionally returns a shared writer instance.

type WildcardQuery 

type WildcardQuery struct {
	// contains filtered or unexported fields
}

func NewWildcardQuery 

func NewWildcardQuery(wildcard string) *WildcardQuery

NewWildcardQuery creates a new Query which finds documents containing terms that match the specified wildcard. In the wildcard pattern '*' will match any sequence of 0 or more characters, and '?' will match any single character.

func (*WildcardQuery) Boost 

func (q *WildcardQuery) Boost() float64

func (*WildcardQuery) Field 

func (q *WildcardQuery) Field() string

func (*WildcardQuery) Searcher 

func (q *WildcardQuery) Searcher(i search.Reader, options search.SearcherOptions) (search.Searcher, error)

func (*WildcardQuery) SetBoost 

func (q *WildcardQuery) SetBoost(b float64) *WildcardQuery

func (*WildcardQuery) SetField 

func (q *WildcardQuery) SetField(f string) *WildcardQuery

func (*WildcardQuery) Validate 

func (q *WildcardQuery) Validate() error

func (*WildcardQuery) Wildcard 

func (q *WildcardQuery) Wildcard() string

Wildcard returns the wildcard being queried

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

func OpenWriter 

func OpenWriter(config Config) (*Writer, error)

func (*Writer) Batch 

func (w *Writer) Batch(batch *index.Batch) error

func (*Writer) Close 

func (w *Writer) Close() error

func (*Writer) Delete 

func (w *Writer) Delete(id segment.Term) error

func (*Writer) Insert 

func (w *Writer) Insert(doc segment.Document) error

func (*Writer) InsertMany added in v0.6.0 

func (w *Writer) InsertMany(documents []*Document) error

InsertMany inserts documents in one atomic text-index batch. Vector-enabled backends receive the corresponding mutations as one batch as well.

func (*Writer) Reader 

func (w *Writer) Reader() (*Reader, error)

func (*Writer) Update 

func (w *Writer) Update(id segment.Term, doc segment.Document) error

func (*Writer) UpdateMany added in v0.6.0 

func (w *Writer) UpdateMany(documents []*Document) error

UpdateMany replaces documents by their _id fields in one atomic text-index batch. Every document must contain an _id field.

Source Files

View all Source files

Directories

Path Synopsis
analyzer
char
lang/ar
lang/bg
lang/ca
lang/cjk
lang/ckb
lang/cs
lang/da
lang/de
lang/el
lang/en
Package en implements an analyzer with reasonable defaults for processing English text.
 Package en implements an analyzer with reasonable defaults for processing English text.
lang/es
lang/eu
lang/fa
lang/fi
lang/fr
lang/ga
lang/gl
lang/hi
lang/hu
lang/hy
lang/id
lang/in
lang/it
lang/nl
lang/no
lang/pt
lang/ro
lang/ru
lang/sv
lang/tr
token
Package lowercase implements a TokenFilter which converts tokens to lower case according to unicode rules.
 Package lowercase implements a TokenFilter which converts tokens to lower case according to unicode rules.
tokenizer
cmd
bluge command
bluge/cmd
lock
mergeplan
Package mergeplan provides a segment merge planning approach that's inspired by Lucene's TieredMergePolicy.java and descriptions like http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
 Package mergeplan provides a segment merge planning approach that's inspired by Lucene's TieredMergePolicy.java and descriptions like http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
internal
blugeidx
zapxbluge
zapxtext
geo
aggregations
collector
highlight
searcher
similarity

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.