embeddings

package module
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 3, 2026 License: BSD-3-Clause Imports: 21 Imported by: 3

README

go-embeddings

Go package defining a common interface for generating text and image embeddings.

Documentation

godoc is currently incomplete.

Motivation

This is a simple abstraction library, written in Go, around a variety of services which produce vector embeddings. There are many such libraries and this one is ours. It tries to be the "simplest dumbest" thing for the most common operations and data needs. These ideas are encapsulated in the EmbeddingsRequest and EmbeddingsResponse types.

type EmbeddingsRequest struct {
	Id    string `json:"id,omitempty"`
	Model string `json:"model"`
	Body  []byte `json:"body"`
}

type EmbeddingsResponse[T Float] interface {
	Id() string
	Model() string
	Embeddings() []T
	Dimensions() int32
	Precision() string
	Created() int64
}

The default implementation of the EmbeddingsResponse interface is the CommonEmbeddingsResponse type:

type CommonEmbeddingsResponse[T Float] struct {
	EmbeddingsResponse[T] `json:",omitempty"`
	CommonId              string `json:"id,omitempty"`
	CommonEmbeddings      []T    `json:"embeddings"`
	CommonModel           string `json:"model"`
	CommonCreated         int64  `json:"created"`
	CommonPrecision       string `json:"precision"`
}

While not specific to SFO Museum this package is targeted at the kinds of things SFO Museum needs to today meaning it may be lacking features you need or want.

Design

To account for the fact that most embeddings models still return float32 vector data but an increasing number of models return float64 vectors this package wraps both options in a Float interface.

type Float interface{ ~float32 | ~float64 }

That Float is then used as a generic value (for embeddings) in a common EmbeddingsResponse interface:

type EmbeddingsResponse[T Float] interface {
	Id() string
	Model() string
	Embeddings() []T
	Dimensions() int32
	Precision() string
	Created() int64
}

That interface is then used as the return value for an Embedder interface:

type Embedder[T Float] interface {
	TextEmbeddings(context.Context, *EmbeddingsRequest) (EmbeddingsResponse[T], error)
	ImageEmbeddings(context.Context, *EmbeddingsRequest) (EmbeddingsResponse[T], error)
}

This means that you need to specify the float type you want the interface to return when you instantiate that interface. For example:

ctx := context.Backgroud()

uri32 := "ollama://?model=embeddinggemma"
uri64 := "encoderfile://"

cl, _ := embeddings.NewEmbedder[float32](ctx, uri32)
cl, _ := embeddings.NewEmbedder[float64](ctx, uri64)

There are also handy NewEmbedder32 and NewEmbedder64 methods which are little more than syntactic sugar. For example:

ctx := context.Backgroud()

uri32 := "ollama://?model=embeddinggemma"
uri64 := "encoderfile://"

cl, _ := embeddings.NewEmbedder32(ctx, uri32)
cl, _ := embeddings.NewEmbedder64(ctx, uri64)

The NewEmbedder, NewEmbedder32 and NewEmbedder64 all have the same signature: A context.Context instance and a URI string used to configure and instantiate the underlying embeddings provider implementation. These are discussed in detail below.

Both the TextEmbeddings and ImageEmbeddings methods take the same input, a EmbeddingsRequest struct:

type EmbeddingsRequest struct {
	Id    string `json:"id,omitempty"`
	Model string `json:"model"`
	Body  []byte `json:"body"`
}

As mentioned both methods return an EmbeddingsResponse[T] instance. The default implementation of the EmbeddingsResponse[T] interface used by this package is the CommonEmbeddingsResponse type. See response.go for details.

Example

Error handling omitted for the sake of brevity.

import (
	"context"
	"encoding/json"
	"os"

	"github.com/sfomuseum/go-embeddings"
)

func main() {

	ctx := context.Background()

	emb, _ := embeddings.NewEmbedder32(ctx, "ollama://?model=embeddinggemma")

	req := &embeddings.EmbeddingsRequest{
		Body: []byte("Hello world"),
	}

	rsp, _ := emb.TextEmbeddings(ctx, req)

	enc := json.NewEncoder(os.Stdout)
	enc.Encode(rsp)

Which would return the following:

{
  "embeddings": [
    -0.21400317549705505,
    0.02651195414364338,
    ... more embeddings
    -0.04678588733077049,
    -0.042774248868227005
  ],
  "model": "ollama/embeddinggemma",
  "created": 1771985811,
  "precision": "float32"
}

Precision

The convention for precision values is a string, for example "float32". Typically an embeddings service will return vector embeddings with a single precision but the Embedder interface allows you to derive embeddings as either float32 or float64 value. In order to preserve the origin precision information if embeddings are requested in a precision other than that generated by a service the requested precision will be appened to the origin value.

For example, if you request float64 values from a service that returns float32 values those data will be recast and the precision string will be updated to read "float32#as-float64".

Implementations

encoderfile://

Derive vector embeddings from an instance of the Mozilla encoderfile application, running as an HTTP server.

encoderfile://?{PARAMETERS}
Name Value Required Notes
client-uri string no The URI for the embedderfile HTTP server endpoint. Default is http://localhost:8080. The gRPC server endpoint provided by encoderfile is not supported yet.
See also
llamafile://

Derive vector embedding from an instance of the Mozilla llamafile application. Note that newer versions of llamafile not longer expose an interface for deriving embeddings so this implementation will only work with older builds. See the encoderfile:// implementation for an alternative.

llamafile://?{PARAMETERS}
Name Value Required Notes
client-uri string no The URI for the llamafile HTTP server endpoint. Default is http://localhost:8080.
See also
mlxclip://

Derive vector embeddings from a Python script using the harperreed/mlx_clip library which emits JSON-encoded embeddings to STDOUT.

The option requires a device using an Apple Silicon chip and involves a non-zero manual set up process discussed below.

mlxclip://{PATH_TO_EMBEDDINGS_DOT_PY}
Set up

As of this writing I am not sure I have working set up instructions. Specifically you want something like the include code in the mlxclip_py.txt file which in turn loads the mlx_clip library. Nothing fancy but since first getting this to work something has changed (?) that prevents Python from importing the mlx_clip package. This remains to be resolved.

See also
mobileclip://

Derive vector embeddings from the MobileCLIP models exposed via an instance of the sfomuseum/swift-mobileclip gRPC endpoint.

mobileclip://?{PARAMETERS}
Name Value Required Notes
client-uri string yes The URI for the swift-mobileclip gRPC server endpoint. Default is grpc://localhosr:8080.
See also
null://

Derive null (empty) vector embeddings. This is a "placeholder" implementation that will always return a zero-length list of embeddings.

null://
ollama://

Derive vector embeddings from an instance of the Ollama application.

ollama://?{PARAMETERS}
Name Value Required Notes
client-uri string no Default is http://localhost:11434.
model string yes The name of the model to use for generating embeddings.
See also
openclip://

Derive vector embeddings from a web service exposing the OpenCLIP model and library.

The option involves a non-zero manual set up process discussed below.

openclip://?{PARAMETERS}
Name Value Required Notes
client-uri string no The URI of the HTTP endpoint exposing the OpenCLIP model functionality. Default is http://localhost:8080.
Set up

Using this implementation requires running a HTTP service exposing the OpenCLIP functionality. The easiest way to do that is in a Python "virtual environment" configured as follows:

$> python -m venv openclip
$> cd openclip/
$> bash bin/activate
$> bin/pip install flask
$> bin/pip install open_clip_torch
$> bin/pip install Pillow

Then, copy the included code in openclip_server.txt in to a file called openclip_server.py and launch it as follows:

$> bin/flask --app openclip_server run
 * Serving Flask app 'openclip_server'
 * Debug mode: off
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
INFO:werkzeug:Press CTRL+C to quit
See also

Tests

Because so many of the implementations above depend on the availability of external, third-party services their tests depend on the presence of Go build tags to run. They are :

Implementation Build tag
encoderfile:// encoderfile
llamafile:// llamafile
mlxclip:// mlxclip
mobileclip:// mobileclip
ollama:// ollama
openclip:// openclip

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var NotImplemented = errors.New("Not implemented")

Functions

func AsFloat32 added in v0.0.2 

func AsFloat32(data []float64) []float32

func AsFloat64 added in v0.0.2 

func AsFloat64(data []float32) []float64

func EmbedderSchemes 

func EmbedderSchemes() []string

Schemes returns the list of schemes that have been registered.

func RegisterEmbedder 

func RegisterEmbedder[T Float](ctx context.Context, scheme string, init_func EmbedderInitializationFunc[T]) error

RegisterEmbedder registers 'scheme' as a key pointing to 'init_func' in an internal lookup table used to create new `Embedder` instances by the `NewEmbedder` method.

Types

type CommonEmbeddingsResponse added in v0.1.0 

type CommonEmbeddingsResponse[T Float] struct {
	EmbeddingsResponse[T] `json:",omitempty"`
	CommonId              string `json:"id,omitempty"`
	CommonEmbeddings      []T    `json:"embeddings"`
	CommonModel           string `json:"model"`
	CommonCreated         int64  `json:"created"`
	CommonPrecision       string `json:"precision"`
}

func (*CommonEmbeddingsResponse[T]) Created added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Created() int64

func (*CommonEmbeddingsResponse[T]) Dimensions added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Dimensions() int32

func (*CommonEmbeddingsResponse[T]) Embeddings added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Embeddings() []T

func (*CommonEmbeddingsResponse[T]) Id added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Id() string

func (*CommonEmbeddingsResponse[T]) Model added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Model() string

func (*CommonEmbeddingsResponse[T]) Precision added in v0.1.0 

func (r *CommonEmbeddingsResponse[T]) Precision() string

type Embedder 

type Embedder[T Float] interface {
	TextEmbeddings(context.Context, *EmbeddingsRequest) (EmbeddingsResponse[T], error)
	ImageEmbeddings(context.Context, *EmbeddingsRequest) (EmbeddingsResponse[T], error)
}

Embedder defines an interface for generating (vector) embeddings

func NewEmbedder 

func NewEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

newEmbedder returns a new `Embedder` instance configured by 'uri'. The value of 'uri' is parsed as a `url.URL` and its scheme is used as the key for a corresponding `EmbedderInitializationFunc` function used to instantiate the new `Embedder`. It is assumed that the scheme (and initialization function) have been registered by the `RegisterEmbedder` method.

func NewEmbedder32 added in v0.1.0 

func NewEmbedder32(ctx context.Context, uri string) (Embedder[float32], error)

func NewEmbedder64 added in v0.1.0 

func NewEmbedder64(ctx context.Context, uri string) (Embedder[float64], error)

func NewEncoderfileEmbedder added in v0.1.0 

func NewEncoderfileEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewLlamafileEmbedder added in v0.1.0 

func NewLlamafileEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewMLXClipEmbedder added in v0.1.0 

func NewMLXClipEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewMobileCLIPEmbedder added in v0.1.0 

func NewMobileCLIPEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewNullEmbedder 

func NewNullEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewOllamaEmbedder added in v0.1.0 

func NewOllamaEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

func NewOpenCLIPEmbedder added in v0.1.0 

func NewOpenCLIPEmbedder[T Float](ctx context.Context, uri string) (Embedder[T], error)

type EmbedderInitializationFunc 

type EmbedderInitializationFunc[T Float] func(ctx context.Context, uri string) (Embedder[T], error)

EmbedderInitializationFunc is a function defined by individual embedder package and used to create an instance of that embedder

type EmbeddingsRequest added in v0.1.0 

type EmbeddingsRequest struct {
	Id    string `json:"id,omitempty"`
	Model string `json:"model"`
	Body  []byte `json:"body"`
}

type EmbeddingsResponse added in v0.1.0 

type EmbeddingsResponse[T Float] interface {
	Id() string
	Model() string
	Embeddings() []T
	Dimensions() int32
	Precision() string
	Created() int64
}

type EncoderfileEmbedder added in v0.1.0 

type EncoderfileEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

EncoderfileEmbedder implements the `Embedder` interface using an Encoderfile API endpoint to derive embeddings.

func (*EncoderfileEmbedder[T]) ImageEmbeddings added in v0.1.0 

func (e *EncoderfileEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*EncoderfileEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *EncoderfileEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type Float added in v0.1.0 

type Float interface{ ~float32 | ~float64 }

type LlamafileEmbedder added in v0.1.0 

type LlamafileEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

LlamafileEmbedder implements the `Embedder` interface using an Llamafile API endpoint to derive embeddings.

func (*LlamafileEmbedder[T]) ImageEmbeddings added in v0.1.0 

func (e *LlamafileEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*LlamafileEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *LlamafileEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type MLXClipEmbedder added in v0.1.0 

type MLXClipEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

func (*MLXClipEmbedder[T]) ImageEmbeddings added in v0.1.1 

func (e *MLXClipEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*MLXClipEmbedder[T]) TextEmbeddings added in v0.1.1 

func (e *MLXClipEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type MobileCLIPEmbedder added in v0.1.0 

type MobileCLIPEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

MobileCLIPEmbedder implements the `Embedder` interface using an MobileCLIP API endpoint to derive embeddings.

func (*MobileCLIPEmbedder[T]) ImageEmbeddings added in v0.1.0 

func (e *MobileCLIPEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*MobileCLIPEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *MobileCLIPEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type NullEmbedder 

type NullEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

NullEmbedder implements the `Embedder` interface using an Null API endpoint to derive embeddings.

func (*NullEmbedder[T]) ImageEmbeddings 

func (e *NullEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*NullEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *NullEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type OllamaEmbedder added in v0.1.0 

type OllamaEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

OllamaEmbedder implements the `Embedder` interface using an Ollama API endpoint to derive embeddings.

func (*OllamaEmbedder[T]) ImageEmbeddings added in v0.1.0 

func (e *OllamaEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*OllamaEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *OllamaEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

type OpenCLIPEmbedder added in v0.1.0 

type OpenCLIPEmbedder[T Float] struct {
	Embedder[T]
	// contains filtered or unexported fields
}

OpenCLIPEmbedder implements the `Embedder` interface using an OpenCLIP API endpoint to derive embeddings.

func (*OpenCLIPEmbedder[T]) ImageEmbeddings added in v0.1.0 

func (e *OpenCLIPEmbedder[T]) ImageEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

func (*OpenCLIPEmbedder[T]) TextEmbeddings added in v0.1.0 

func (e *OpenCLIPEmbedder[T]) TextEmbeddings(ctx context.Context, req *EmbeddingsRequest) (EmbeddingsResponse[T], error)

Source Files

View all Source files

Directories

Path Synopsis
app
embeddings
cmd
embeddings command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.