embed

Documentation

Overview

Package embed defines the EmbedEngine interface for vector embedding backends.

Unlike the generation-oriented engine.Engine (workflow.Graph → Result), EmbedEngine converts raw content (text, image bytes, video bytes) into dense float32 vectors suitable for similarity search.

Index

• func Retry(fn func() error, maxRetries int, initialDelay time.Duration) error
• type Capability
• type ContentType
• type Describer
• type EmbedEngine
• type RateLimiter
  • func NewRateLimiter(maxPerMinute int) *RateLimiter
  • func (rl *RateLimiter) Wait(ctx context.Context) error
• type Request
  • func ImageRequest(data []byte, taskType string) Request
  • func TextRequest(text string, taskType string) Request
  • func VideoRequest(data []byte, taskType string) Request
• type Result

Functions

func Retry

func Retry(fn func() error, maxRetries int, initialDelay time.Duration) error

Retry calls fn with exponential back-off on retryable errors.

Types

type Capability

type Capability struct {
	// SupportedTypes lists content types this engine can embed.
	SupportedTypes []ContentType
	// Models lists available model identifiers.
	Models []string
	// MaxDimensions is the maximum output dimensionality.
	MaxDimensions int
	// SupportsMRL indicates Matryoshka Representation Learning support
	// (embeddings can be truncated to fewer dimensions).
	SupportsMRL bool
}

Capability describes what an embedding engine supports.

type ContentType

type ContentType int

ContentType identifies the kind of content being embedded.

const (
	ContentText  ContentType = iota // plain text
	ContentImage                    // image bytes (JPEG, PNG, WebP)
	ContentVideo                    // video bytes (MP4, MOV)
)

type Describer

type Describer interface {
	EmbedCapabilities() Capability
}

Describer is an optional interface for engines that advertise capabilities.

type EmbedEngine

type EmbedEngine interface {
	// Embed produces a vector for the given request.
	Embed(ctx context.Context, req Request) (Result, error)

	// EmbedBatch produces vectors for multiple requests.
	// Backends that support batch APIs can implement this efficiently;
	// the default implementation calls Embed sequentially.
	EmbedBatch(ctx context.Context, reqs []Request) ([]Result, error)

	// Dimensions returns the output vector dimensionality.
	Dimensions() int
}

EmbedEngine converts content into vector embeddings.

type RateLimiter

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter implements a sliding-window rate limiter. Shared by all embedding backends that need request throttling.

func NewRateLimiter

func NewRateLimiter(maxPerMinute int) *RateLimiter

NewRateLimiter creates a rate limiter allowing max requests per minute.

func (*RateLimiter) Wait

func (rl *RateLimiter) Wait(ctx context.Context) error

Wait blocks until the caller is allowed to proceed. Respects context cancellation.

type Request

type Request struct {
	// Content holds the raw data: a string for text, or []byte for image/video.
	Content any
	// Type classifies the content.
	Type ContentType
	// TaskType hints how the embedding will be used (e.g., "RETRIEVAL_DOCUMENT",
	// "RETRIEVAL_QUERY"). Backends that support task-type-aware embedding
	// (Gemini, Voyage) use this; others ignore it.
	TaskType string
}

Request describes what to embed.

func ImageRequest

func ImageRequest(data []byte, taskType string) Request

ImageRequest is a convenience constructor for image embedding.

func TextRequest

func TextRequest(text string, taskType string) Request

TextRequest is a convenience constructor for text embedding.

func VideoRequest

func VideoRequest(data []byte, taskType string) Request

VideoRequest is a convenience constructor for video embedding.

type Result

type Result struct {
	// Vector is the dense embedding.
	Vector []float32
	// Dimensions is the vector length (len(Vector)).
	Dimensions int
	// Model identifies which model produced this embedding.
	Model string
}

Result is the output of an embedding operation.