v2

package
v0.1.14 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 20, 2025 License: MIT Imports: 13 Imported by: 0

README

Milvus Vector Store v2

This package provides a Milvus vector store implementation using the new Milvus SDK v2 (github.com/milvus-io/milvus/client/v2).

Background

The original vectorstores/milvus package uses the archived github.com/milvus-io/milvus-sdk-go/v2 SDK, which was deprecated by the Milvus maintainers on March 21, 2025. This new v2 package uses the actively maintained SDK at github.com/milvus-io/milvus/client/v2.

Migration from v1 Package

Quick Migration Guide

  1. Update imports:

    // Old
import "github.com/tmc/langchaingo/vectorstores/milvus"

// New
import "github.com/tmc/langchaingo/vectorstores/milvus/v2"

  2. Update configuration (optional - v1 configs are automatically converted):

    // Old
config := client.Config{Address: "localhost:19530"}

// New (recommended)
config := milvusclient.ClientConfig{Address: "localhost:19530"}

// Or keep v1 config (automatically converted)
config := client.Config{Address: "localhost:19530"} // Still works!

  3. Update function calls:

    // Old
store, err := milvus.New(ctx, config, opts...)

// New
store, err := milvusv2.New(ctx, config, opts...)
Compatibility Features

The v2 package includes compatibility adapters that allow gradual migration:

  • Config Compatibility: v1 client.Config is automatically converted to v2 milvusclient.ClientConfig
  • Index Compatibility: Use WithIndexV1() for v1 index types
  • Search Parameter Compatibility: Use WithSearchParametersV1() for v1 search parameters
  • Metric Type Compatibility: Use WithMetricTypeV1() for v1 metric types
Example: Gradual Migration
// During migration - mix v1 and v2 configurations
store, err := milvusv2.New(ctx,
    client.Config{Address: "localhost:19530"}, // v1 config
    milvusv2.WithEmbedder(embedder),
    milvusv2.WithIndexV1(oldIndex),           // v1 index
    milvusv2.WithCollectionName("my_collection"),
)

Usage

Basic Usage
package main

import (
    "context"

    "github.com/milvus-io/milvus/client/v2/entity"
    "github.com/milvus-io/milvus/client/v2/index"
    "github.com/milvus-io/milvus/client/v2/milvusclient"
    "github.com/tmc/langchaingo/embeddings"
    "github.com/tmc/langchaingo/llms/openai"
    "github.com/tmc/langchaingo/schema"
    "github.com/tmc/langchaingo/vectorstores/milvus/v2"
)

func main() {
    ctx := context.Background()

    // Create embedder
    llm, err := openai.New()
    if err != nil {
        panic(err)
    }
    embedder, err := embeddings.NewEmbedder(llm)
    if err != nil {
        panic(err)
    }

    // Configure Milvus connection
    config := milvusclient.ClientConfig{
        Address: "localhost:19530",
    }

    // Create vector store
    store, err := v2.New(ctx, config,
        v2.WithEmbedder(embedder),
        v2.WithCollectionName("my_documents"),
        v2.WithIndex(index.NewAutoIndex(entity.L2)),
    )
    if err != nil {
        panic(err)
    }

    // Add documents
    docs := []schema.Document{
        {
            PageContent: "This is a document about AI",
            Metadata: map[string]any{"topic": "AI", "source": "example"},
        },
        {
            PageContent: "This document discusses machine learning",
            Metadata: map[string]any{"topic": "ML", "source": "example"},
        },
    }

    ids, err := store.AddDocuments(ctx, docs)
    if err != nil {
        panic(err)
    }

    // Search for similar documents
    results, err := store.SimilaritySearch(ctx, "artificial intelligence", 5)
    if err != nil {
        panic(err)
    }

    for _, doc := range results {
        fmt.Printf("Score: %.3f, Content: %s\n", doc.Score, doc.PageContent)
    }
}
Configuration Options
store, err := v2.New(ctx, config,
    v2.WithEmbedder(embedder),                    // Required: embedder for vector generation
    v2.WithCollectionName("my_collection"),       // Collection name
    v2.WithPartitionName("my_partition"),         // Partition name (optional)
    v2.WithTextField("content"),                  // Text field name (default: "text")
    v2.WithMetaField("metadata"),                 // Metadata field name (default: "meta")
    v2.WithVectorField("embedding"),              // Vector field name (default: "vector")
    v2.WithPrimaryField("id"),                    // Primary field name (default: "pk")
    v2.WithMaxTextLength(1000),                   // Max text length (default: 65535)
    v2.WithShards(2),                             // Number of shards (default: 1)
    v2.WithIndex(index.NewAutoIndex(entity.L2)),  // Vector index configuration
    v2.WithMetricType(entity.L2),                 // Distance metric
    v2.WithDropOld(),                             // Drop existing collection
    v2.WithSkipFlushOnWrite(),                    // Skip flushing after writes
)

API Reference

Core Methods
  • New(ctx, config, opts...) - Create new Milvus vector store
  • AddDocuments(ctx, docs, opts...) - Add documents to the store
  • SimilaritySearch(ctx, query, numDocs, opts...) - Search for similar documents
Configuration Options
Basic Options
  • WithEmbedder(embedder) - Set the embedder (required)
  • WithCollectionName(name) - Set collection name
  • WithPartitionName(name) - Set partition name
Field Configuration
  • WithTextField(name) - Set text field name
  • WithMetaField(name) - Set metadata field name
  • WithVectorField(name) - Set vector field name
  • WithPrimaryField(name) - Set primary key field name
Performance Options
  • WithMaxTextLength(length) - Set maximum text length
  • WithShards(num) - Set number of shards
  • WithSkipFlushOnWrite() - Skip flushing after writes
Index and Metrics
  • WithIndex(index) - Set vector index
  • WithMetricType(metric) - Set distance metric
  • WithEF(ef) - Set EF parameter for HNSW
Compatibility Options (for migration)
  • WithIndexV1(index) - Use v1 index type
  • WithSearchParametersV1(params) - Use v1 search parameters
  • WithMetricTypeV1(metric) - Use v1 metric type
  • WithConsistencyLevelV1(level) - Use v1 consistency level

Index Types

The v2 package supports the new index types from the Milvus SDK v2:

// Auto index (recommended for most use cases)
index.NewAutoIndex(entity.L2)

// Flat index
index.NewFlatIndex(entity.L2)

// IVF Flat index
index.NewIvfFlatIndex(entity.L2, 1024) // metric, nlist

// HNSW index
index.NewHNSWIndex(entity.L2, 16, 200) // metric, M, efConstruction

Metric Types

Supported distance metrics:

  • entity.L2 - L2 (Euclidean) distance
  • entity.IP - Inner product
  • entity.COSINE - Cosine similarity
  • entity.HAMMING - Hamming distance (for binary vectors)
  • entity.JACCARD - Jaccard distance (for binary vectors)

Error Handling

The package defines specific error types:

  • ErrEmbedderWrongNumberVectors - Vector count mismatch
  • ErrColumnNotFound - Missing required column
  • ErrInvalidFilters - Invalid filter format
  • ErrInvalidOptions - Invalid configuration options

Testing

The package includes comprehensive tests that cover:

  • Configuration compatibility (v1/v2)
  • Index compatibility
  • Option functions
  • Document operations
  • Search operations

Run tests with:

go test ./vectorstores/milvus/v2/...

Migration Timeline

  • Current: Both packages are available
  • Recommended: Use v2 package for new projects
  • Migration: Use compatibility options for gradual migration
  • Future: v1 package will be fully deprecated when Milvus 3.0 is released

See Also

Documentation

Overview

Package v2 provides a vectorstore implementation for Milvus using the new SDK. This package uses github.com/milvus-io/milvus/client/v2 which is actively maintained and replaces the archived milvus-sdk-go/v2 package.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrEmbedderWrongNumberVectors = errors.New(
		"number of vectors from embedder does not match number of documents",
	)
	ErrColumnNotFound = errors.New("invalid field")
	ErrInvalidFilters = errors.New("invalid filters")
)
View Source 
var ErrInvalidOptions = errors.New("invalid options")

ErrInvalidOptions is returned when the options given are invalid.

Functions

This section is empty.

Types

type ConfigAdapter 

type ConfigAdapter struct{}

ConfigAdapter handles conversion between v1 and v2 configurations

func (ConfigAdapter) ToV2Config 

func (ca ConfigAdapter) ToV2Config(config interface{}) (milvusclient.ClientConfig, error)

ToV2Config converts various config types to milvusclient.ClientConfig

type Option 

type Option func(p *Store)

Option is a function type that can be used to modify the client.

func WithCollectionName 

func WithCollectionName(name string) Option

WithCollectionName sets the collection for the milvus store.

func WithConsistencyLevel 

func WithConsistencyLevel(level entity.ConsistencyLevel) Option

WithConsistencyLevel sets the consistency level for the collection.

func WithConsistencyLevelV1 

func WithConsistencyLevelV1(level oldentity.ConsistencyLevel) Option

WithConsistencyLevelV1 sets the consistency level from v1 type (compatibility).

func WithDropOld 

func WithDropOld() Option

WithDropOld sets the drop old collection flag.

func WithEF 

func WithEF(ef int) Option

WithEF sets the ef parameter for HNSW index.

func WithEmbedder 

func WithEmbedder(embedder embeddings.Embedder) Option

WithEmbedder sets the embedder to use.

func WithIndex 

func WithIndex(idx index.Index) Option

WithIndex sets the index to use for the vector field.

func WithIndexV1 

func WithIndexV1(idx oldentity.Index) Option

WithIndexV1 sets the index from v1 type (compatibility).

func WithMaxTextLength 

func WithMaxTextLength(num int) Option

WithMaxTextLength sets the maximum length of the text field in the collection.

func WithMetaField 

func WithMetaField(str string) Option

WithMetaField sets the name of the meta field in the collection schema. default is 'meta'.

func WithMetricType 

func WithMetricType(metricType entity.MetricType) Option

WithMetricType sets the metric type for the vector field.

func WithMetricTypeV1 

func WithMetricTypeV1(metricType oldentity.MetricType) Option

WithMetricTypeV1 sets metric type from v1 type (compatibility).

func WithPartitionName 

func WithPartitionName(name string) Option

WithPartitionName sets the milvus partition for the collection.

func WithPrimaryField 

func WithPrimaryField(str string) Option

WithPrimaryField sets the name of the primary field in the collection schema.

func WithSearchParameters 

func WithSearchParameters(sp map[string]interface{}) Option

WithSearchParameters sets the search parameters.

func WithSearchParametersV1 

func WithSearchParametersV1(sp oldentity.SearchParam) Option

WithSearchParametersV1 sets search parameters from v1 type (compatibility).

func WithShards 

func WithShards(num int32) Option

WithShards sets the number of shards for the collection.

func WithSkipFlushOnWrite 

func WithSkipFlushOnWrite() Option

WithSkipFlushOnWrite sets the skip flush on write flag.

func WithTextField 

func WithTextField(str string) Option

WithTextField sets the name of the text field in the collection schema.

func WithVectorField 

func WithVectorField(str string) Option

WithVectorField sets the name of the vector field in the collection schema.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is a wrapper around the milvus client using the new SDK.

func New 

func New(ctx context.Context, config interface{}, opts ...Option) (Store, error)

New creates an active client connection to the (specified, or default) collection in the Milvus server and returns the `Store` object needed by the other accessors. Supports both v1 (client.Config) and v2 (milvusclient.ClientConfig) configurations for compatibility.

func (Store) AddDocuments 

func (s Store) AddDocuments(ctx context.Context, docs []schema.Document,
	_ ...vectorstores.Option,
) ([]string, error)

AddDocuments adds the text and metadata from the documents to the Milvus collection associated with 'Store'. and returns the ids of the added documents.

func (Store) SimilaritySearch 

func (s Store) SimilaritySearch(ctx context.Context, query string, numDocuments int,
	options ...vectorstores.Option,
) ([]schema.Document, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.