types

package
v1.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 13, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package types provides type-safe data structures for ML inference operations.

The types package defines all request/response types, builders, and parsers for ML operations in the Lumen SDK. It provides:

  • Request builders for clean, fluent API construction
  • Response parsers for type-safe result handling
  • Data structures for embeddings, classifications, and face detection
  • MIME type constants for supported formats

Request Builders

Use builders to construct type-safe inference requests: 

inferReq := types.NewInferRequest("text_embedding").
    WithCorrelationID("req-123").
    WithMeta("model", "v2").
    ForEmbedding(embeddingReq, "text_embedding").
    Build()

Response Parsers

Parse responses into strongly-typed structures: 

result, _ := client.Infer(ctx, inferReq)
embedding, err := types.ParseInferResponse(result).
    AsEmbeddingResponse()
if err != nil {
    log.Fatal(err)
}

Embedding Operations

Work with vector embeddings for semantic search: 

// Generate embedding
text := []byte("semantic search query")
embReq, _ := types.NewEmbeddingRequest(text)

// Compare embeddings
similarity, _ := emb1.CosineSimilarity(emb2)
if similarity > 0.9 {
    fmt.Println("Highly similar!")
}

Classification

Classify images into categories: 

imageData, _ := os.ReadFile("photo.jpg")
classReq, _ := types.NewClassificationRequest(imageData)
inferReq := types.NewInferRequest("classification").
    ForClassification(classReq, "image_classification").
    Build()

result, _ := client.Infer(ctx, inferReq)
labels, _ := types.ParseInferResponse(result).
    AsClassificationResponse()
topLabels := labels.TopK(5)

Face Detection

Detect and recognize faces in images: 

imageData, _ := os.ReadFile("photo.jpg")
faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithDetectionConfidenceThreshold(0.85),
    types.WithMaxFaces(10),
)
inferReq := types.NewInferRequest("face_detection").
    ForFaceDetection(faceReq, "face_detection").
    Build()

Role in Project

The types package provides the data layer for ML operations, ensuring type safety and clean APIs. It bridges between application code and the protobuf-based gRPC communication layer.

Index

Constants

View Source 
const (
	TaskSemanticTextEmbed  = "semantic_text_embed"
	TaskSemanticImageEmbed = "semantic_image_embed"
	TaskBioCLIPClassify    = "bioclip_classify"
	TaskOCR                = "ocr"
	TaskFaceRecognition    = "face_recognition"

	ServiceCLIP    = "clip"
	ServiceBioCLIP = "bioclip"
	ServiceSigLIP  = "siglip"
	ServiceOCR     = "ocr"
	ServiceFace    = "face"

	PreprocessBioCLIP224Image               = "bioclip2_224_image_v1"
	PreprocessSigLIP2BasePatch16_224Image   = "siglip2_base_patch16_224_image_v1"
	PreprocessSigLIP2SO400MPatch14_384Image = "siglip2_so400m_patch14_384_image_v1"
	PreprocessCLIPImage                     = PreprocessBioCLIP224Image
	PreprocessSigLIPImage                   = PreprocessSigLIP2BasePatch16_224Image
	PreprocessPPOCRDetection                = "ppocr_det_v1"
	PreprocessInsightFaceDet                = "insightface_det_v1"

	MetaService        = "service"
	MetaTopK           = "top_k"
	MetaSourceWidth    = "lumen.source.width"
	MetaSourceHeight   = "lumen.source.height"
	MetaLetterboxScale = "lumen.letterbox.scale"
	MetaLetterboxPadX  = "lumen.letterbox.pad_x"
	MetaLetterboxPadY  = "lumen.letterbox.pad_y"

	DeprecatedTensorJSONMIME = "application/vnd.lumen.tensor+json"
)
View Source 
const (
	// Reserved metadata keys for Lumen tensor fast-path requests and responses.
	MetaInputKind             = "lumen.input.kind"
	MetaTensorDType           = "lumen.tensor.dtype"
	MetaTensorShape           = "lumen.tensor.shape"
	MetaTensorLayout          = "lumen.tensor.layout"
	MetaTensorFormat          = "lumen.tensor.format"
	MetaTensorByteOrder       = "lumen.tensor.byte_order"
	MetaPreprocessID          = "lumen.preprocess.id"
	MetaPreprocessSkip        = "lumen.preprocess.skip"
	MetaModelID               = "lumen.model.id"
	MetaModelVersion          = "lumen.model.version"
	MetaOutputKind            = "lumen.output.kind"
	MetaOutputTensorDType     = "lumen.output.tensor.dtype"
	MetaOutputTensorShape     = "lumen.output.tensor.shape"
	MetaOutputTensorLayout    = "lumen.output.tensor.layout"
	MetaOutputTensorFormat    = "lumen.output.tensor.format"
	MetaOutputTensorByteOrder = "lumen.output.tensor.byte_order"
	InputKindRaw              = "raw"
	InputKindTensor           = "tensor"
	OutputKindRaw             = "raw"
	OutputKindTensor          = "tensor"
	TensorFormatContig        = "contiguous"
	TensorByteOrderLittle     = "little"
	DefaultTensorMIME         = "application/octet-stream"
)

Variables

View Source 
var SupportedImageMimeTypes = []string{
	"image/jpeg",
	"image/png",
	"image/webp",
	"image/avif",
}

SupportedImageMimeTypes lists the image MIME types accepted by Lumen ML services.

These formats are supported for image-based operations including:

  • Image embedding generation
  • Image classification
  • Face detection and recognition

Role in project: Defines the contract between clients and ML nodes for image data. Used for validation in request builders (NewClassificationRequest, etc.).

Example: 

imageData, _ := os.ReadFile("photo.jpg")
mime := mimetype.Detect(imageData).String()
if !mimetype.EqualsAny(mime, types.SupportedImageMimeTypes...) {
    log.Fatal("Unsupported image format")
}
View Source 
var SupportedTextMimeTypes = []string{
	"text/plain",
	"text/markdown",
	"text/html",
}

SupportedTextMimeTypes lists the text MIME types accepted by Lumen ML services.

These formats are supported for text-based operations including:

  • Text embedding generation
  • Semantic search
  • Text analysis tasks

Role in project: Defines acceptable text formats for ML operations. Used for validation in NewEmbeddingRequest and other text-processing functions.

Example: 

textData := []byte("Machine learning is transforming AI")
embReq, err := types.NewEmbeddingRequest(textData)
// Automatically validates against SupportedTextMimeTypes
View Source 
var TopKMetaAliases = []string{"TopK", "topK", "top_k", "top-k", "lumen.top_k"}
View Source 
var ValidFinishReasons = []string{
	"stop",
	"length",
	"eos_token",
	"stop_sequence",
	"error",
}

ValidFinishReasons defines the acceptable values for the FinishReason field.

Functions

func AssembleInferResponses added in v1.1.4 

func AssembleInferResponses(responses []*pb.InferResponse) (*pb.InferResponse, error)

AssembleInferResponses returns the final semantic response from a response stream. Responses with Total > 1 are treated as transport chunks and strictly reassembled. Legacy or semantic streaming responses without Total > 1 return the final response, or the last response when no final marker is present.

func IOTaskHasTensorPath added in v1.2.8 

func IOTaskHasTensorPath(task *pb.IOTask) bool

func IOTaskTensorBatchingSupported added in v1.2.8 

func IOTaskTensorBatchingSupported(task *pb.IOTask) bool

func IOTaskTensorPreprocessID added in v1.2.8 

func IOTaskTensorPreprocessID(task *pb.IOTask) string

func ServiceFromMeta added in v1.1.5 

func ServiceFromMeta(meta map[string]string) string

func TensorBatchingKey added in v1.1.5 

func TensorBatchingKey(req *pb.InferRequest) (string, bool, error)

func TensorElementSize added in v1.1.4 

func TensorElementSize(dtype string) (int, bool)

TensorElementSize returns the byte size for supported fast-path tensor dtypes.

func ValidateTaskRequest added in v1.1.5 

func ValidateTaskRequest(req *pb.InferRequest) error

ValidateTaskRequest validates the public task request contract.

Types

type ClassificationRequest 

type ClassificationRequest struct {
	Payload     []byte `json:"payload"`
	PayloadMime string `json:"payload_mime"`
}

ClassificationRequest represents a request for image classification.

This structure encapsulates the image payload and its MIME type for classification. Only image types are supported (see SupportedImageMimeTypes). Use with the InferRequest builder's ForClassification() method.

Role in project: Input data structure for image classification operations. Works with NewClassificationRequest() for automatic MIME type detection and validation.

Example: 

imageData, _ := os.ReadFile("photo.jpg")
classReq, err := types.NewClassificationRequest(imageData)
if err != nil {
    log.Fatal(err)
}

func NewClassificationRequest 

func NewClassificationRequest(payload []byte) (*ClassificationRequest, error)

NewClassificationRequest creates a new ClassificationRequest with automatic MIME detection.

This function analyzes the payload to detect its image format and validates that it's a supported type for classification. Supported formats include JPEG, PNG, GIF, BMP, WebP, and other common image formats (see SupportedImageMimeTypes).

Parameters:

  • payload: The raw image bytes to clip_classify

Returns:

  • *ClassificationRequest: Request object ready for ForClassification()
  • error: Non-nil if the payload is not a supported image type

Role in project: Factory function that simplifies classification request creation with automatic format detection. Prevents errors from incorrect MIME type specification.

Example: 

// Classify an image file
imageData, err := os.ReadFile("nature_scene.jpg")
if err != nil {
    log.Fatal(err)
}

classReq, err := types.NewClassificationRequest(imageData)
if err != nil {
    log.Fatalf("Unsupported image format: %v", err)
}

inferReq := types.NewInferRequest("scene_classification").
    ForClassification(classReq, "scene_classification").
    Build()

result, _ := client.Infer(ctx, inferReq)
labels, _ := types.ParseInferResponse(result).AsClassificationResponse()
fmt.Printf("Scene: %s\n", labels.TopK(1)[0].Label)

type EmbeddingRequest 

type EmbeddingRequest struct {
	Payload     []byte `json:"payload"`
	PayloadMime string `json:"payload_mime"`
}

EmbeddingRequest represents a request for embedding generation.

This structure encapsulates the payload (text or image bytes) and its MIME type for embedding generation. Additional parameters like ModelID, CorrelationID, and metadata are set through the InferRequest builder.

Role in project: Input data structure for embedding operations. Works with NewEmbeddingRequest() for automatic MIME type detection and validation.

Example: 

// Create embedding request with auto-detected MIME type
textData := []byte("semantic search query")
embReq, err := types.NewEmbeddingRequest(textData)
if err != nil {
    log.Fatal(err)
}

func NewEmbeddingRequest 

func NewEmbeddingRequest(payload []byte) (*EmbeddingRequest, error)

NewEmbeddingRequest creates a new EmbeddingRequest with automatic MIME type detection.

This function analyzes the payload to determine if it's text or image data and validates that the MIME type is supported for embedding generation. Supported types include text/plain, text/html, image/jpeg, image/png, and others.

Parameters:

  • payload: The raw bytes of text or image data to embed

Returns:

  • *EmbeddingRequest: Request object ready to use with ForEmbedding()
  • error: Non-nil if MIME type is unsupported

Role in project: Factory function that simplifies embedding request creation by automatically detecting and validating content types. Prevents common errors from incorrect MIME type specification.

Example: 

// Text embedding
text := []byte("Natural language processing")
embReq, err := types.NewEmbeddingRequest(text)
if err != nil {
    log.Fatalf("Unsupported content type: %v", err)
}

// Image embedding
imageData, _ := os.ReadFile("photo.jpg")
embReq, err := types.NewEmbeddingRequest(imageData)
if err != nil {
    log.Fatalf("Unsupported image type: %v", err)
}

type EmbeddingV1 

type EmbeddingV1 struct {
	Vector  []float32 `json:"vector" example:"[0.1, 0.2, 0.3]"`
	Dim     int       `json:"dim" example:"3"`
	ModelID string    `json:"model_id" example:"embedding_model_1"`

	// AestheticScore is an optional aesthetic quality score (teacher-distilled,
	// roughly 1–10) returned alongside an image embedding when the model ships an
	// aesthetic head. It is nil for text embeddings and for image models without a
	// head. Use AestheticScoreValue for a presence-checked read.
	AestheticScore *float32 `json:"aesthetic_score,omitempty" example:"6.4"`
}

EmbeddingV1 represents a high-dimensional vector embedding from ML models.

Embeddings are dense vector representations that capture semantic meaning of text, images, or other data types. They enable similarity comparisons, semantic search, clustering, and recommendation systems.

The vector values are typically normalized or can be normalized using the Normalize() method. Common embedding dimensions range from 128 to 1536 depending on the model.

Role in project: Core data structure for embedding operations, the most fundamental ML output in the Lumen SDK. Embeddings power semantic search, image similarity, recommendation engines, and clustering applications.

Example: 

// Generate and use an embedding
result, _ := client.Infer(ctx, embeddingRequest)
embedding, _ := types.ParseInferResponse(result).AsEmbeddingResponse()

fmt.Printf("Dimensions: %d\n", embedding.DimValue())
fmt.Printf("Model: %s\n", embedding.ModelID)
fmt.Printf("Magnitude: %.4f\n", embedding.Magnitude())

// Compare with another embedding
similarity, _ := embedding.CosineSimilarity(otherEmbedding)
if similarity > 0.9 {
    fmt.Println("Highly similar!")
}

func (EmbeddingV1) AestheticScoreValue added in v1.2.9 

func (e EmbeddingV1) AestheticScoreValue() (float32, bool)

AestheticScoreValue returns the aesthetic score and whether it was present.

Returns:

  • float32: the score (0 when absent)
  • bool: true when the model provided an aesthetic score

Example: 

embedding, _ := types.ParseInferResponse(result).AsEmbeddingResponse()
if score, ok := embedding.AestheticScoreValue(); ok {
    fmt.Printf("aesthetic score: %.2f\n", score)
}

func (EmbeddingV1) CosineSimilarity 

func (e EmbeddingV1) CosineSimilarity(other EmbeddingV1) (float32, error)

CosineSimilarity computes the cosine similarity between two embeddings.

Cosine similarity measures the cosine of the angle between two vectors, ranging from -1 (opposite) to +1 (identical), with 0 indicating orthogonality. This is the standard similarity metric for embeddings and is invariant to vector magnitude.

The vectors must have the same dimensions. For best results, use normalized embeddings.

Parameters:

  • other: The embedding to compare against

Returns:

  • float32: Similarity score in range [-1, 1], where higher is more similar
  • error: Non-nil if dimensions don't match

Role in project: Primary method for comparing embeddings in semantic search, similarity ranking, duplicate detection, and recommendation systems. This is the most commonly used distance metric in the Lumen SDK.

Example: 

// Compare two text embeddings
text1 := []byte("machine learning")
text2 := []byte("artificial intelligence")
emb1, _ := generateEmbedding(text1)
emb2, _ := generateEmbedding(text2)

similarity, err := emb1.CosineSimilarity(emb2)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Similarity: %.4f\n", similarity)
if similarity > 0.8 {
    fmt.Println("Highly similar concepts!")
}

func (EmbeddingV1) DimValue 

func (e EmbeddingV1) DimValue() int

DimValue returns the actual dimension of the embedding vector.

This is computed from the vector length and may differ from the Dim field if the model output was truncated or padded. Always use DimValue() for accurate dimension information.

Returns:

  • int: The number of dimensions in the vector

Example: 

embedding, _ := types.ParseInferResponse(result).AsEmbeddingResponse()
dim := embedding.DimValue()
fmt.Printf("Vector has %d dimensions\n", dim)

func (EmbeddingV1) Dot 

func (e EmbeddingV1) Dot(other EmbeddingV1) (float32, error)

Dot Computes the dot product of two embeddings

func (EmbeddingV1) EuclideanDistance 

func (e EmbeddingV1) EuclideanDistance(other EmbeddingV1) (float32, error)

EuclideanDistance Computes the Euclidean distance between two embeddings

func (EmbeddingV1) IsEmpty 

func (e EmbeddingV1) IsEmpty() bool

IsEmpty Returns true if the embedding is empty

func (EmbeddingV1) Magnitude 

func (e EmbeddingV1) Magnitude() float32

func (EmbeddingV1) ManhattanDistance 

func (e EmbeddingV1) ManhattanDistance(other EmbeddingV1) (float32, error)

ManhattanDistance Computes the Manhattan distance between two embeddings

func (EmbeddingV1) Normalize 

func (e EmbeddingV1) Normalize() EmbeddingV1

Normalize performs L2 normalization on the embedding vector.

L2 normalization scales the vector to unit length (magnitude = 1.0), which is essential for computing cosine similarity and ensures consistent distance metrics. Many embedding models output pre-normalized vectors, but this method can be used to ensure normalization or to re-normalize after vector arithmetic.

Returns:

  • EmbeddingV1: A new embedding with normalized vector (original is unchanged)

Role in project: Prepares embeddings for similarity calculations. Normalized vectors allow cosine similarity to be computed using just dot product, which is much faster.

Example: 

embedding, _ := types.ParseInferResponse(result).AsEmbeddingResponse()
normalized := embedding.Normalize()
fmt.Printf("Original magnitude: %.4f\n", embedding.Magnitude())
fmt.Printf("Normalized magnitude: %.4f\n", normalized.Magnitude()) // Should be ~1.0

type Face 

type Face struct {
	BBox       []float32 `json:"bbox"` //  [x1, y1, x2, y2]
	Confidence float32   `json:"confidence"`
	Landmarks  []float32 `json:"landmarks,omitempty"`
	Embedding  []float32 `json:"embedding,omitempty"`
}

Face represents a single detected face with its attributes.

Each face includes:

  • BBox: Bounding box as [x1, y1, x2, y2] in image coordinates
  • Confidence: Detection confidence score (0.0 to 1.0)
  • Landmarks: Optional facial keypoints (eyes, nose, mouth corners, etc.)
  • Embedding: Optional face embedding vector for recognition/comparison

Role in project: Individual face detection result containing location, confidence, and optional biometric data for recognition tasks.

type FaceRecognitionOption 

type FaceRecognitionOption func(*FaceRecognitionRequest)

FaceRecognitionOption is a function type for configuring face detection requests.

This option pattern allows clean, readable configuration of detection parameters without requiring many constructor variants or builder methods.

Role in project: Provides flexible configuration mechanism for face detection requests using the functional options pattern.

func WithDetectionConfidenceThreshold 

func WithDetectionConfidenceThreshold(threshold float32) FaceRecognitionOption

WithDetectionConfidenceThreshold sets the minimum confidence threshold for face detection.

Only faces detected with confidence above this threshold will be included in results. Higher values reduce false positives but may miss some faces. Typical range: 0.5-0.95.

Parameters:

  • threshold: Confidence threshold (0.0 to 1.0)

Returns:

  • FaceRecognitionOption: Option function for NewFaceRecognitionRequest

Example: 

faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithDetectionConfidenceThreshold(0.9), // Only high-confidence faces
)

func WithFaceSizeMax 

func WithFaceSizeMax(size float32) FaceRecognitionOption

WithFaceSizeMax 设置最大人脸尺寸

func WithFaceSizeMin 

func WithFaceSizeMin(size float32) FaceRecognitionOption

WithFaceSizeMin 设置最小人脸尺寸

func WithMaxFaces 

func WithMaxFaces(maxFaces int) FaceRecognitionOption

WithMaxFaces sets the maximum number of faces to detect in the image.

Limits the number of faces returned in the response. Use -1 for unlimited faces. This is useful for performance optimization and when you only need a fixed number of the most confident detections.

Parameters:

  • maxFaces: Maximum faces to return (-1 for unlimited)

Returns:

  • FaceRecognitionOption: Option function for NewFaceRecognitionRequest

Example: 

// Detect at most 5 faces
faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithMaxFaces(5),
)

// Detect all faces
faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithMaxFaces(-1),
)

func WithNmsThreshold 

func WithNmsThreshold(threshold float32) FaceRecognitionOption

WithNmsThreshold 设置 NMS 阈值

type FaceRecognitionRequest 

type FaceRecognitionRequest struct {
	Payload                      []byte  `json:"payload"`
	PayloadMime                  string  `json:"payload_mime"`
	DetectionConfidenceThreshold float32 `json:"detection_confidence_threshold,omitempty"`
	NmsThreshold                 float32 `json:"nms_threshold,omitempty"`
	FaceSizeMin                  float32 `json:"face_size_min,omitempty"`
	FaceSizeMax                  float32 `json:"face_size_max,omitempty"`
	MaxFaces                     int     `json:"max_faces,omitempty"` // -1 means no limit
}

FaceRecognitionRequest represents a request for face detection and recognition.

This structure encapsulates the image payload and various detection parameters:

  • DetectionConfidenceThreshold: Minimum confidence for accepting detections (0.0-1.0)
  • NmsThreshold: Non-maximum suppression threshold for overlapping faces
  • FaceSizeMin/Max: Constraints on face sizes to detect
  • MaxFaces: Maximum number of faces to return (-1 for unlimited)

Use the WithXxx option functions to set these parameters cleanly.

Role in project: Input structure for face detection tasks with fine-grained control over detection parameters. Supports both simple detection and advanced recognition with facial embeddings.

Example: 

imageData, _ := os.ReadFile("group_photo.jpg")
faceReq, err := types.NewFaceRecognitionRequest(imageData,
    types.WithDetectionConfidenceThreshold(0.85),
    types.WithMaxFaces(10),
    types.WithFaceSizeMin(20.0),
)

func NewFaceRecognitionRequest 

func NewFaceRecognitionRequest(payload []byte, opts ...FaceRecognitionOption) (*FaceRecognitionRequest, error)

NewFaceRecognitionRequest creates a new face detection request with optional configuration.

This function analyzes the payload to detect the image format and validates it's a supported type. Configuration options can be passed to customize detection behavior using the WithXxx option functions.

Parameters:

  • payload: The raw image bytes to process
  • opts: Optional configuration functions (WithDetectionConfidenceThreshold, WithMaxFaces, etc.)

Returns:

  • *FaceRecognitionRequest: Configured request ready for ForFaceDetection()
  • error: Non-nil if the payload is not a supported image type

Role in project: Factory function for creating face detection requests with clean, flexible configuration. Automatically detects image format and validates MIME type.

Example: 

// Basic face detection
imageData, _ := os.ReadFile("photo.jpg")
faceReq, err := types.NewFaceRecognitionRequest(imageData)

// Advanced face detection with custom parameters
faceReq, err := types.NewFaceRecognitionRequest(imageData,
    types.WithDetectionConfidenceThreshold(0.85),
    types.WithMaxFaces(10),
    types.WithFaceSizeMin(20.0),
    types.WithNmsThreshold(0.4),
)
if err != nil {
    log.Fatalf("Invalid image: %v", err)
}

inferReq := types.NewInferRequest("face_detection").
    ForFaceDetection(faceReq, "face_detection").
    Build()

type FaceV1 

type FaceV1 struct {
	Faces   []Face `json:"faces"`
	Count   int    `json:"count"`
	ModelID string `json:"model_id"`
}

FaceV1 represents face detection and recognition results from ML models.

This structure contains all detected faces with their locations, confidence scores, facial landmarks, and optional embeddings for recognition. The Count field indicates the total number of faces detected.

Role in project: Output structure for face detection and recognition tasks. Used in security systems, photo organization, attendance tracking, biometric authentication, and identity verification applications.

Example: 

result, _ := client.Infer(ctx, faceDetectionRequest)
faceResp, _ := types.ParseInferResponse(result).AsFaceResponse()
fmt.Printf("Detected %d faces\n", faceResp.Count)
fmt.Printf("Model: %s\n", faceResp.ModelID)
for i, face := range faceResp.Faces {
    fmt.Printf("Face %d: confidence=%.2f, location=%v\n",
        i+1, face.Confidence, face.BBox)
}

type ImageInput added in v1.2.8 

type ImageInput struct {
	Encoded     []byte
	PayloadMIME string

	Data       []byte
	Width      int
	Height     int
	Channels   int
	Layout     string
	DType      string
	ColorSpace string
}

ImageInput is the SDK-owned preprocessor input shape. Callers can provide an encoded image, an already-decoded HWC RGB uint8 image, or both. Decoded input matching the preprocessor's target dimensions is preferred: it skips the in-process decode/resize entirely (the caller's image pipeline, e.g. libvips, already produced model-sized pixels). Encoded input is the fallback whenever decoded input is absent or does not match the expected shape.

type ImageTextGenerationRequest 

type ImageTextGenerationRequest struct {
	Payload     []byte            `json:"payload"`
	PayloadMime string            `json:"payload_mime"`
	Meta        map[string]string `json:"meta"`
}

ImageTextGenerationRequest represents a request for image+text generation (VLM).

This structure encapsulates an image payload, prompt or messages, and generation parameters for vision-language model tasks.

Role in project: Input structure for VLM text generation tasks.

Example: 

imageData, _ := os.ReadFile("cat.jpg")
req, err := types.NewImageTextGenerationRequest(imageData, "image/jpeg").
	WithMaxTokens(512).
	WithTemperature(0.0)

inferReq := types.NewInferRequest("vlm").
    ForImageTextGeneration(req, "fastvlm-2b-onnx").
    Build()

func NewImageTextGenerationRequest 

func NewImageTextGenerationRequest(payload []byte, opts ...ImageTextGenerationRequestOption) (*ImageTextGenerationRequest, error)

NewImageTextGenerationRequest creates a new image+text generation request.

This function initializes a request with the provided image data and MIME type, with optional generation parameters configured via functional options.

Parameters:

  • payload: The raw image bytes to process
  • payloadMime: The MIME type of the image (e.g., "image/jpeg", "image/png")
  • opts: Optional functions to configure generation parameters

Returns:

  • *ImageTextGenerationRequest: Configured request ready for ForImageTextGeneration()
  • error: Non-nil if the payload is not a supported image type

Role in project: Factory function for creating VLM requests.

Example: 

imageData, _ := os.ReadFile("cat.jpg")
req, err := types.NewImageTextGenerationRequest(imageData, "image/jpeg",
	types.WithMaxTokens(512),
	types.WithTemperature(0.0),
	types.WithMessages([]map[string]string{
		{"role": "user", "content": "What's in this image?"},
	}))

if err != nil {
	log.Fatalf("Failed to create request: %v", err)
}

type ImageTextGenerationRequestOption 

type ImageTextGenerationRequestOption func(*ImageTextGenerationRequest)

func WithAddGenerationPrompt 

func WithAddGenerationPrompt(addPrompt bool) ImageTextGenerationRequestOption

WithAddGenerationPrompt adds a generation prompt to the input.

func WithDoSample 

func WithDoSample(doSample bool) ImageTextGenerationRequestOption

WithDoSample enables or disables sampling for token generation.

func WithMaxTokens 

func WithMaxTokens(maxTokens int) ImageTextGenerationRequestOption

WithMaxTokens sets the maximum number of new tokens to generate.

func WithMessages 

func WithMessages(messages []map[string]string) ImageTextGenerationRequestOption

WithMessages sets the messages in chat format for the generation request.

func WithPrompt 

func WithPrompt(prompt string) ImageTextGenerationRequestOption

WithPrompt sets the text prompt for the generation request.

func WithRepetitionPenalty 

func WithRepetitionPenalty(penalty float64) ImageTextGenerationRequestOption

WithRepetitionPenalty sets the repetition penalty to discourage repetitive text.

func WithStopSequences 

func WithStopSequences(stopSequences []string) ImageTextGenerationRequestOption

WithStopSequences sets the stop sequences for generation.

func WithTemperature 

func WithTemperature(temperature float64) ImageTextGenerationRequestOption

WithTemperature sets the sampling temperature for the generation request. Higher values (e.g., 0.8) make output more random, while lower values (e.g., 0.2) make it more deterministic.

func WithTopP 

func WithTopP(topP float64) ImageTextGenerationRequestOption

WithTopP sets the nucleus sampling parameter for the generation request. Controls diversity via probability threshold: 0.9 means considering the top 90% probability mass.

type InferRequestBuilder 

type InferRequestBuilder struct {
	// contains filtered or unexported fields
}

InferRequestBuilder provides a fluent interface for constructing inference requests.

This builder pattern implementation allows for clean, readable request construction with method chaining. It handles the complexity of setting up various request types (embedding, classification, face detection) with appropriate metadata and configurations.

Role in project: Simplifies the creation of well-formed inference requests for end users. The builder pattern prevents common mistakes and ensures all required fields are set correctly.

Example: 

req := types.NewInferRequest("text_embedding").
    WithCorrelationID("my-request-123").
    WithMeta("model_version", "v2").
    ForEmbedding(embeddingReq, "text_embedding").
    Build()

func NewInferRequest 

func NewInferRequest(task string) *InferRequestBuilder

NewInferRequest creates a new InferRequestBuilder for the specified task.

The task name is used by the client to select appropriate ML nodes from the catalog based on their capabilities. Each node advertises the tasks it supports, and the client's load balancer uses this information for routing.

Parameters:

  • task: The ML task identifier (e.g., "text_embedding", "face_detection", "classification")

Returns:

  • *InferRequestBuilder: A new builder instance ready for method chaining

Role in project: Entry point for creating type-safe inference requests. This is typically the first method called when preparing an ML inference operation.

Example: 

// Create a builder for text embedding task
builder := types.NewInferRequest("text_embedding")

// Or with immediate chaining
req := types.NewInferRequest("face_detection").
    WithCorrelationID("detection-001").
    Build()

func (*InferRequestBuilder) Build 

func (b *InferRequestBuilder) Build() *pb.InferRequest

Build finalizes and returns the constructed inference request.

This method completes the builder chain and produces the protobuf InferRequest that can be sent to the LumenClient for processing. After calling Build(), the builder should not be reused.

Returns:

  • *pb.InferRequest: The fully constructed inference request ready for submission

Role in project: Final step in the builder pattern that produces the actual request object consumed by the client's Infer() or InferStream() methods.

Example: 

req := types.NewInferRequest("classification").
    WithCorrelationID("img-001").
    ForClassification(classReq, "image_classification").
    Build()
result, err := client.Infer(ctx, req)

func (*InferRequestBuilder) ForBioCLIPClassify added in v1.1.5 

func (b *InferRequestBuilder) ForBioCLIPClassify(payload []byte, mime string, topK int) *InferRequestBuilder

func (*InferRequestBuilder) ForBioCLIPTensor added in v1.1.5 

func (b *InferRequestBuilder) ForBioCLIPTensor(payload []byte, dtype string, topK int) *InferRequestBuilder

func (*InferRequestBuilder) ForClassification 

func (b *InferRequestBuilder) ForClassification(req *ClassificationRequest, task string) *InferRequestBuilder

ForClassification configures the builder for an image classification request.

Classification requests categorize images into predefined classes with confidence scores. This is commonly used for content moderation, scene detection, object recognition, and automated tagging systems.

Parameters:

  • req: The classification request with image payload (use NewClassificationRequest for MIME detection)
  • task: The classification task name from node capabilities (e.g., "lumen_clip_classify", "scene_classification")

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Specialized builder for classification tasks. Classification is widely used for categorizing visual content, detecting objects, identifying scenes, and content filtering applications.

Example: 

// Basic image classification
imageData, _ := os.ReadFile("photo.jpg")
classReq, _ := types.NewClassificationRequest(imageData)
req := types.NewInferRequest("image_classification").
    ForClassification(classReq, "lumen_clip_classify").
    Build()

result, err := client.Infer(ctx, req)
classResp, _ := types.ParseInferResponse(result).AsClassificationResponse()
topLabels := classResp.TopK(5)
for _, label := range topLabels {
    fmt.Printf("%s: %.2f\n", label.Label, label.Score)
}

func (*InferRequestBuilder) ForEmbedding 

func (b *InferRequestBuilder) ForEmbedding(req *EmbeddingRequest, task string) *InferRequestBuilder

ForEmbedding configures the builder for an embedding generation request.

Embedding requests transform text or images into high-dimensional vector representations useful for semantic search, similarity comparisons, and clustering. This method sets the appropriate payload and MIME type for the embedding operation.

Parameters:

  • req: The embedding request containing the payload (use NewEmbeddingRequest for automatic MIME detection)
  • task: The specific embedding task name from node capabilities (e.g., TaskSemanticTextEmbed)

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Specialized builder method for embedding tasks, the most commonly used ML operation in the Lumen SDK. Embeddings power features like semantic search, image similarity, and content recommendation.

Example: 

// Text embedding
textData := []byte("Machine learning is fascinating")
embReq, _ := types.NewEmbeddingRequest(textData)
req := types.NewInferRequest(types.TaskSemanticTextEmbed).
    ForEmbedding(embReq, types.TaskSemanticTextEmbed).
    Build()

// Image embedding
imageData, _ := os.ReadFile("photo.jpg")
embReq, _ := types.NewEmbeddingRequest(imageData)
req := types.NewInferRequest("image_embedding").
    ForEmbedding(embReq, "lumen_clip_image_embed").
    Build()

func (*InferRequestBuilder) ForFaceDetection 

func (b *InferRequestBuilder) ForFaceDetection(req *FaceRecognitionRequest, task string) *InferRequestBuilder

ForFaceDetection configures the builder for face detection and recognition requests.

Face detection requests locate and analyze faces in images, optionally returning facial landmarks, bounding boxes, and face embeddings for recognition. This method automatically sets task-specific metadata from the FaceRecognitionRequest configuration.

Supported parameters (set via metadata):

  • detection_confidence_threshold: Minimum confidence for face detection (0.0-1.0)
  • nms_threshold: Non-maximum suppression threshold for overlapping detections
  • face_size_min/max: Constraints on detected face sizes
  • max_faces: Maximum number of faces to detect (-1 for unlimited)

Parameters:

  • req: Face detection request with image and optional configuration parameters
  • task: The face detection task name (e.g., "face_detection", "face_recognition")

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Specialized builder for face detection/recognition tasks. Used in applications like security systems, photo organization, attendance tracking, and identity verification.

Example: 

// Face detection with custom thresholds
imageData, _ := os.ReadFile("group_photo.jpg")
faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithDetectionConfidenceThreshold(0.8),
    types.WithMaxFaces(10),
)
req := types.NewInferRequest("face_detection").
    ForFaceDetection(faceReq, "face_detection").
    Build()

result, err := client.Infer(ctx, req)
faceResp, _ := types.ParseInferResponse(result).AsFaceResponse()
fmt.Printf("Detected %d faces\n", faceResp.Count)
for i, face := range faceResp.Faces {
    fmt.Printf("Face %d: confidence=%.2f, bbox=%v\n",
        i+1, face.Confidence, face.BBox)
}

func (*InferRequestBuilder) ForFaceRecognitionRaw added in v1.1.5 

func (b *InferRequestBuilder) ForFaceRecognitionRaw(payload []byte, mime string) *InferRequestBuilder

func (*InferRequestBuilder) ForFaceRecognitionTensor added in v1.1.5 

func (b *InferRequestBuilder) ForFaceRecognitionTensor(payload []byte, dtype string, h, w int64, sourceWidth, sourceHeight int, scale, padX, padY float64) *InferRequestBuilder

func (*InferRequestBuilder) ForImageTextGeneration 

func (b *InferRequestBuilder) ForImageTextGeneration(req *ImageTextGenerationRequest, task string) *InferRequestBuilder

ForImageTextGeneration configures the builder for an image+text generation (VLM) request.

Image+text generation requests process an image along with a text prompt or messages to generate descriptive text about the image. This method sets the appropriate payload and transfers all metadata from the ImageTextGenerationRequest to the inference request.

Parameters:

  • req: The VLM request with image payload, prompt/messages, and generation parameters
  • task: The VLM task name from node capabilities (e.g., "vlm", "image_text_generation")

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Specialized builder for vision-language model tasks. Used in applications like image description, visual question answering, content analysis, and multimodal AI systems.

Example: 

// Basic VLM request with custom parameters
imageData, _ := os.ReadFile("cat.jpg")
vlmReq, _ := types.NewImageTextGenerationRequest(imageData, "image/jpeg",
    types.WithMaxTokens(256),
    types.WithTemperature(0.2),
    types.WithMessages([]map[string]string{
        {"role": "user", "content": "Describe what's in this image"},
    }))
req := types.NewInferRequest("vlm").
    ForImageTextGeneration(vlmReq, "fastvlm-2b-onnx").
    Build()

result, err := client.Infer(ctx, req)
genResp, _ := types.ParseInferResponse(result).AsTextGenerationResponse()
fmt.Printf("Generated text: %s\n", genResp.Text)
fmt.Printf("Finish reason: %s\n", genResp.FinishReason)

func (*InferRequestBuilder) ForOCR 

func (b *InferRequestBuilder) ForOCR(req *OCRRequest, task string) *InferRequestBuilder

ForOCR configures the builder for an optical character recognition (OCR) request.

OCR requests detect and recognize text in images. This method sets the appropriate payload and task-specific metadata from the OCRRequest configuration.

Supported parameters (set via metadata):

  • detection_threshold: Minimum confidence for text detection
  • recognition_threshold: Minimum confidence for text recognition
  • use_angle_cls: Whether to use angle classification for rotated text

Parameters:

  • req: OCR request with image and optional configuration parameters
  • task: The OCR task name (e.g., "ocr", "text_detection")

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Specialized builder for OCR tasks. Used in document digitization, license plate recognition, and scene text understanding.

Example: 

// OCR with custom thresholds
imageData, _ := os.ReadFile("document.jpg")
ocrReq, _ := types.NewOCRRequest(imageData,
    types.WithDetectionThreshold(0.7),
    types.WithUseAngleCls(true),
)
req := types.NewInferRequest("ocr").
    ForOCR(ocrReq, "ocr_system").
    Build()

result, err := client.Infer(ctx, req)
ocrResp, _ := types.ParseInferResponse(result).AsOCRResponse()
fmt.Printf("Detected %d text regions\n", ocrResp.Count)

func (*InferRequestBuilder) ForOCRRaw added in v1.1.5 

func (b *InferRequestBuilder) ForOCRRaw(payload []byte, mime string) *InferRequestBuilder

func (*InferRequestBuilder) ForOCRTensor added in v1.1.5 

func (b *InferRequestBuilder) ForOCRTensor(payload []byte, dtype string, h, w int64, sourceWidth, sourceHeight int) *InferRequestBuilder

func (*InferRequestBuilder) ForSemanticImageEmbed added in v1.1.5 

func (b *InferRequestBuilder) ForSemanticImageEmbed(payload []byte, mime string) *InferRequestBuilder

func (*InferRequestBuilder) ForSemanticImageTensor added in v1.1.5 

func (b *InferRequestBuilder) ForSemanticImageTensor(payload []byte, service string, dtype string) *InferRequestBuilder

func (*InferRequestBuilder) ForSemanticTextEmbed added in v1.1.5 

func (b *InferRequestBuilder) ForSemanticTextEmbed(text string) *InferRequestBuilder

func (*InferRequestBuilder) ForTensorInput added in v1.1.4 

func (b *InferRequestBuilder) ForTensorInput(payload []byte, mime string, descriptor TensorDescriptor) *InferRequestBuilder

ForTensorInput configures the request with a model-ready tensor payload. Routing still uses Task; model metadata is only a validation hint for task backends.

func (*InferRequestBuilder) WithCorrelationID 

func (b *InferRequestBuilder) WithCorrelationID(id string) *InferRequestBuilder

WithCorrelationID sets a custom correlation ID for request tracking and logging.

Correlation IDs enable request tracing across distributed components and make debugging easier. If not provided, the system automatically generates a unique correlation ID. Using custom IDs is recommended for production systems with distributed tracing.

Parameters:

  • id: A unique identifier for this request (e.g., UUID, trace ID)

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Enables distributed tracing and log correlation across the entire inference pipeline (client -> load balancer -> ML node -> response).

Example: 

req := types.NewInferRequest("embedding").
    WithCorrelationID(uuid.New().String()).
    Build()

func (*InferRequestBuilder) WithInputKind added in v1.1.4 

func (b *InferRequestBuilder) WithInputKind(kind string) *InferRequestBuilder

WithInputKind declares whether the payload is raw task input or a model-ready tensor.

func (*InferRequestBuilder) WithMeta 

func (b *InferRequestBuilder) WithMeta(key, value string) *InferRequestBuilder

WithMeta adds a metadata key-value pair to the inference request.

Metadata provides additional configuration and context for ML inference operations. Different tasks support different metadata fields. Common uses include model selection, confidence thresholds, and output format preferences.

Task-specific metadata examples:

  • Face detection: "detection_confidence_threshold", "max_faces", "nms_threshold"
  • Embedding: "model_version", "normalize", "output_format"
  • Classification: "top_k", "confidence_threshold"

Parameters:

  • key: Metadata field name (task-specific, see ML node documentation)
  • value: Metadata field value as a string

Returns:

  • *InferRequestBuilder: The builder instance for method chaining

Role in project: Provides flexible configuration mechanism for ML tasks without requiring API changes when new parameters are added to ML models.

Example: 

req := types.NewInferRequest("face_detection").
    WithMeta("detection_confidence_threshold", "0.8").
    WithMeta("max_faces", "10").
    Build()

func (*InferRequestBuilder) WithPayload added in v1.1.5 

func (b *InferRequestBuilder) WithPayload(payload []byte, mime string) *InferRequestBuilder

func (*InferRequestBuilder) WithPreprocessID added in v1.1.4 

func (b *InferRequestBuilder) WithPreprocessID(id string) *InferRequestBuilder

WithPreprocessID records the preprocessing contract that produced a tensor payload.

func (*InferRequestBuilder) WithService added in v1.1.5 

func (b *InferRequestBuilder) WithService(service string) *InferRequestBuilder

WithService records the target service in InferRequest.Meta.

func (*InferRequestBuilder) WithTensorDescriptor added in v1.1.4 

func (b *InferRequestBuilder) WithTensorDescriptor(dtype string, shape []int64, layout string) *InferRequestBuilder

WithTensorDescriptor writes the v1 tensor fast-path descriptor into request metadata.

type InferResponseParser 

type InferResponseParser struct {
	// contains filtered or unexported fields
}

InferResponseParser provides type-safe parsing of ML inference responses.

This parser handles the deserialization and type conversion of protobuf responses into Go structs. It validates response MIME types and ensures the correct schema is used for each response type (embedding, classification, face detection).

Role in project: Bridges the gap between protobuf responses and Go application code. Provides type safety and validation to prevent runtime errors from mismatched response types.

Example: 

result, _ := client.Infer(ctx, request)
parser := types.ParseInferResponse(result)
embeddingResp, err := parser.AsEmbeddingResponse()
if err != nil {
    log.Fatalf("Failed to parse response: %v", err)
}

func ParseInferResponse 

func ParseInferResponse(resp *pb.InferResponse) *InferResponseParser

ParseInferResponse creates a new parser for the given inference response.

This is the entry point for response parsing. After creating a parser, use the appropriate As* method to convert to the expected response type.

Parameters:

  • resp: The protobuf inference response from the ML node

Returns:

  • *InferResponseParser: A parser instance ready for type conversion

Role in project: Factory function that initiates the response parsing chain. This is typically called immediately after receiving a response from Infer() or InferStream().

Example: 

result, err := client.Infer(ctx, inferReq)
if err != nil {
    log.Fatal(err)
}
parser := types.ParseInferResponse(result)

func (*InferResponseParser) AsClassificationResponse 

func (p *InferResponseParser) AsClassificationResponse() (*LabelsV1, error)

AsClassificationResponse parses the response as image classification results.

This method validates the MIME type (application/json;schema=labels_v1) and deserializes the response into a LabelsV1 structure containing classification labels with confidence scores. The labels are typically sorted by confidence, and you can use TopK() to get the most likely categories.

Returns:

  • *LabelsV1: Parsed classification with labels, scores, and model ID
  • error: Non-nil if MIME type is incorrect or JSON parsing fails

Role in project: Type-safe conversion for classification responses. Used extensively in content categorization, object detection, scene recognition, and automated tagging systems.

Example: 

// Classify an image
imageData, _ := os.ReadFile("nature.jpg")
classReq, _ := types.NewClassificationRequest(imageData)
inferReq := types.NewInferRequest("image_classification").
    ForClassification(classReq, "scene_classification").
    Build()

result, _ := client.Infer(ctx, inferReq)
classification, err := types.ParseInferResponse(result).AsClassificationResponse()
if err != nil {
    log.Fatalf("Failed to parse classification: %v", err)
}

// Get top 3 predictions
topLabels := classification.TopK(3)
fmt.Println("Top predictions:")
for i, label := range topLabels {
    fmt.Printf("%d. %s (%.2f%%)\n", i+1, label.Label, label.Score*100)
}

func (*InferResponseParser) AsEmbeddingResponse 

func (p *InferResponseParser) AsEmbeddingResponse() (*EmbeddingV1, error)

AsEmbeddingResponse parses the response as an embedding vector result.

This method validates the MIME type (application/json;schema=embedding_v1) and converts the response into an EmbeddingV1 structure containing a float32 vector, dimension info, and model identifier. The resulting embedding can be used for similarity calculations.

Returns:

  • *EmbeddingV1: Parsed embedding with vector, dimension, and model ID
  • error: Non-nil if MIME type is incorrect or JSON parsing fails

Role in project: Type-safe conversion for embedding responses. Embeddings are the most common output type in the Lumen SDK, used for semantic search, recommendation systems, and similarity comparisons.

Example: 

// Generate text embedding
text := []byte("Machine learning transforms data into insights")
embReq, _ := types.NewEmbeddingRequest(text)
inferReq := types.NewInferRequest("text_embedding").
    ForEmbedding(embReq, "text_embedding").
    Build()

result, _ := client.Infer(ctx, inferReq)
embedding, err := types.ParseInferResponse(result).AsEmbeddingResponse()
if err != nil {
    log.Fatalf("Failed to parse embedding: %v", err)
}

fmt.Printf("Embedding dimension: %d\n", embedding.DimValue())
fmt.Printf("Model: %s\n", embedding.ModelID)

// Compare with another embedding
similarity, _ := embedding.CosineSimilarity(otherEmbedding)
fmt.Printf("Cosine similarity: %.4f\n", similarity)

func (*InferResponseParser) AsFaceResponse 

func (p *InferResponseParser) AsFaceResponse() (*FaceV1, error)

AsFaceResponse parses the response as face detection/recognition results.

This method validates that the response has the correct MIME type (application/json;schema=face_v1) and deserializes it into a FaceV1 structure containing detected faces with their bounding boxes, confidence scores, landmarks, and optional embeddings.

Returns:

  • *FaceV1: Parsed face detection results with all detected faces
  • error: Non-nil if MIME type is incorrect or JSON parsing fails

Role in project: Type-safe conversion for face detection responses. Essential for applications performing face detection, recognition, or biometric operations.

Example: 

// Detect faces in an image
imageData, _ := os.ReadFile("photo.jpg")
faceReq, _ := types.NewFaceRecognitionRequest(imageData,
    types.WithMaxFaces(5))
inferReq := types.NewInferRequest("face_detection").
    ForFaceDetection(faceReq, "face_detection").
    Build()

result, _ := client.Infer(ctx, inferReq)
faceResp, err := types.ParseInferResponse(result).AsFaceResponse()
if err != nil {
    log.Fatalf("Failed to parse face response: %v", err)
}

fmt.Printf("Found %d faces\n", faceResp.Count)
for i, face := range faceResp.Faces {
    fmt.Printf("Face %d: confidence=%.2f\n", i+1, face.Confidence)
}

func (*InferResponseParser) AsOCRResponse 

func (p *InferResponseParser) AsOCRResponse() (*OCRV1, error)

AsOCRResponse parses the response as OCR results.

This method validates that the response has the correct MIME type (application/json;schema=ocr_v1) and deserializes it into an OCRV1 structure containing detected text regions.

Returns:

  • *OCRV1: Parsed OCR results with all detected text items
  • error: Non-nil if MIME type is incorrect or JSON parsing fails

Role in project: Type-safe conversion for OCR responses.

func (*InferResponseParser) AsTensorResponse added in v1.1.4 

func (p *InferResponseParser) AsTensorResponse() (*TensorResponse, error)

AsTensorResponse parses the response as a validated tensor output.

func (*InferResponseParser) AsTextGenerationResponse 

func (p *InferResponseParser) AsTextGenerationResponse() (*TextGenerationV1, error)

AsTextGenerationResponse parses the response as text generation results.

This method validates that the response has the correct MIME type (application/json;schema=text_generation_v1) and deserializes it into a TextGenerationV1 structure containing generated text, token counts, finish reason, and optional metadata.

Returns:

  • *TextGenerationV1: Parsed text generation results with generated text and metadata
  • error: Non-nil if MIME type is incorrect or JSON parsing fails

Role in project: Type-safe conversion for text generation responses. Used extensively in vision-language model applications for image description, visual question answering, and multimodal content analysis.

Example: 

// Generate text about an image
imageData, _ := os.ReadFile("cat.jpg")
vlmReq, _ := types.NewImageTextGenerationRequest(imageData, "image/jpeg",
    types.WithMaxTokens(256),
    types.WithTemperature(0.2),
    types.WithMessages([]map[string]string{
        {"role": "user", "content": "Describe what's in this image"},
    }))
req := types.NewInferRequest("vlm").
    ForImageTextGeneration(vlmReq, "fastvlm-2b-onnx").
    Build()

result, err := client.Infer(ctx, req)
genResp, err := types.ParseInferResponse(result).AsTextGenerationResponse()
if err != nil {
    log.Fatalf("Failed to parse text generation response: %v", err)
}

fmt.Printf("Generated text: %s\n", genResp.Text)
fmt.Printf("Tokens generated: %d\n", genResp.GeneratedTokens)
fmt.Printf("Finish reason: %s\n", genResp.FinishReason)
if genResp.Metadata != nil {
    fmt.Printf("Generation time: %.2f ms\n", genResp.Metadata.GenerationTimeMs)
}

func (*InferResponseParser) Raw 

func (p *InferResponseParser) Raw() *pb.InferResponse

Raw returns the underlying protobuf response without parsing.

Use this method when you need direct access to the raw response fields, such as custom metadata, correlation IDs, or when implementing custom response handling logic.

Returns:

  • *pb.InferResponse: The original protobuf response

Role in project: Provides escape hatch for advanced use cases that need access to raw response data not covered by the typed parsers.

Example: 

result, _ := client.Infer(ctx, inferReq)
parser := types.ParseInferResponse(result)
raw := parser.Raw()
fmt.Printf("Correlation ID: %s\n", raw.CorrelationId)
fmt.Printf("Is final: %v\n", raw.IsFinal)

type Label 

type Label struct {
	Label string  `json:"label"`
	Score float32 `json:"score"`
}

Label represents a single classification category with its confidence score.

Role in project: Individual classification result pairing a label name with its confidence score. Used for ranking and filtering classification results.

type LabelsV1 

type LabelsV1 struct {
	Labels  []Label `json:"labels" example:"[{\"label\": \"cat\", \"score\": 0.9}, {\"label\": \"dog\", \"score\": 0.1}]"`
	ModelID string  `json:"model_id" example:"embedding_model_1"`
}

LabelsV1 represents image classification results with confidence scores.

Classification results consist of multiple labels (categories) with associated confidence scores indicating the likelihood that each label applies to the input. Labels are typically pre-sorted by confidence score in descending order.

Role in project: Output structure for image classification tasks. Used extensively in content categorization, object detection, scene recognition, and automated tagging.

Example: 

result, _ := client.Infer(ctx, classificationRequest)
classification, _ := types.ParseInferResponse(result).AsClassificationResponse()
fmt.Printf("Model: %s\n", classification.ModelID)
for _, label := range classification.TopK(5) {
    fmt.Printf("%s: %.2f%%\n", label.Label, label.Score*100)
}

func (LabelsV1) TopK 

func (l LabelsV1) TopK(k int) []Label

TopK returns the top K most confident labels from the classification results.

This method sorts labels by confidence score (if not already sorted) and returns the K highest scoring labels. If K exceeds the number of labels, all labels are returned.

Parameters:

  • k: Number of top labels to return

Returns:

  • []Label: Slice of top K labels sorted by confidence (descending)

Role in project: Enables easy extraction of most relevant classification results, commonly used for displaying top predictions to users or filtering low-confidence results.

Example: 

classification, _ := types.ParseInferResponse(result).AsClassificationResponse()
topLabels := classification.TopK(3)
fmt.Println("Top 3 predictions:")
for i, label := range topLabels {
    fmt.Printf("%d. %s (%.1f%% confidence)\n",
        i+1, label.Label, label.Score*100)
}

type OCRItem 

type OCRItem struct {
	Box        [][]int `json:"box"` // List of [x, y] points
	Text       string  `json:"text"`
	Confidence float32 `json:"confidence"`
}

OCRItem represents a single detected text region with its content.

Each item includes:

  • Box: Polygon coordinates defining the text region (usually 4 points: TL, TR, BR, BL). Each point is [x, y].
  • Text: Recognized text content.
  • Confidence: Recognition confidence score (0.0 to 1.0).

type OCRRequest 

type OCRRequest struct {
	Payload              []byte  `json:"payload"`
	PayloadMime          string  `json:"payload_mime"`
	DetectionThreshold   float32 `json:"detection_threshold,omitempty"`
	RecognitionThreshold float32 `json:"recognition_threshold,omitempty"`
	UseAngleCls          bool    `json:"use_angle_cls,omitempty"`
}

OCRRequest represents a request for optical character recognition.

This structure encapsulates the image payload for text detection and recognition. It automatically handles MIME type detection and validation.

Role in project: Input structure for OCR tasks.

Example: 

imageData, _ := os.ReadFile("document.jpg")
ocrReq, err := types.NewOCRRequest(imageData)

func NewOCRRequest 

func NewOCRRequest(payload []byte, opts ...OCRRequestOption) (*OCRRequest, error)

NewOCRRequest creates a new OCR request.

This function analyzes the payload to detect the image format and validates it's a supported type.

Parameters:

  • payload: The raw image bytes to process

Returns:

  • *OCRRequest: Configured request ready for ForOCR()
  • error: Non-nil if the payload is not a supported image type

Role in project: Factory function for creating OCR requests.

Example: 

imageData, _ := os.ReadFile("receipt.jpg")
ocrReq, err := types.NewOCRRequest(imageData)
if err != nil {
    log.Fatalf("Invalid image: %v", err)
}

inferReq := types.NewInferRequest("ocr").
    ForOCR(ocrReq, "ocr_model").
    Build()

type OCRRequestOption 

type OCRRequestOption func(*OCRRequest)

func WithDetectionThreshold 

func WithDetectionThreshold(threshold float32) OCRRequestOption

func WithRecognitionThreshold 

func WithRecognitionThreshold(threshold float32) OCRRequestOption

func WithUseAngleCls 

func WithUseAngleCls(useAngleCls bool) OCRRequestOption

type OCRV1 

type OCRV1 struct {
	Items   []OCRItem `json:"items"`
	Count   int       `json:"count"`
	ModelID string    `json:"model_id"`
}

OCRV1 represents optical character recognition (OCR) results.

This structure contains all detected text regions with their content, locations, and confidence scores. The Count field indicates the total number of text regions detected.

Role in project: Output structure for OCR tasks. Used in document digitization, text extraction, license plate recognition, and scene text understanding.

type TaskContract added in v1.2.8 

type TaskContract struct {
	Task *pb.IOTask
}

TaskContract is a small helper wrapper around the protobuf IOTask contract.

func NewTaskContract added in v1.2.8 

func NewTaskContract(task *pb.IOTask) TaskContract

func (TaskContract) HasTensorPath added in v1.2.8 

func (t TaskContract) HasTensorPath() bool

func (TaskContract) TensorBatchingSupported added in v1.2.8 

func (t TaskContract) TensorBatchingSupported() bool

func (TaskContract) TensorPreprocessID added in v1.2.8 

func (t TaskContract) TensorPreprocessID() string

type TensorDescriptor added in v1.1.4 

type TensorDescriptor struct {
	DType          string
	Shape          []int64
	Layout         string
	Format         string
	ByteOrder      string
	PreprocessID   string
	PreprocessSkip bool
	ModelID        string
	ModelVersion   string
}

TensorDescriptor describes a model-ready tensor carried through InferRequest.Meta. It intentionally mirrors the v1 tensor fast-path metadata contract without changing the protobuf wire shape.

func ParseOutputTensorDescriptor added in v1.1.4 

func ParseOutputTensorDescriptor(meta map[string]string) (*TensorDescriptor, bool, error)

ParseOutputTensorDescriptor parses InferResponse.Meta into an output tensor descriptor. The bool return value is true only for lumen.output.kind=tensor.

func ParseTensorDescriptor added in v1.1.4 

func ParseTensorDescriptor(meta map[string]string) (*TensorDescriptor, bool, error)

ParseTensorDescriptor parses InferRequest.Meta into a tensor descriptor. The bool return value is true only for lumen.input.kind=tensor.

func ValidateTensorFastPath added in v1.1.4 

func ValidateTensorFastPath(req *pb.InferRequest, opts TensorValidationOptions) (*TensorDescriptor, error)

ValidateTensorFastPath validates tensor fast-path metadata when present. Raw requests return (nil, nil); tensor requests return their parsed descriptor.

func ValidateTensorResponse added in v1.1.4 

func ValidateTensorResponse(resp *pb.InferResponse, opts TensorOutputValidationOptions) (*TensorDescriptor, error)

ValidateTensorResponse validates tensor output metadata when present. Raw and legacy JSON responses return (nil, nil); tensor responses return their descriptor.

type TensorOutputValidationOptions added in v1.1.4 

type TensorOutputValidationOptions struct {
	AllowedDTypes     []string
	AllowedLayouts    []string
	AllowedFormats    []string
	AllowedByteOrders []string

	// DisableSingleBatchCheck permits tensor outputs whose leading batch
	// dimension is greater than one. It is false by default for v1.
	DisableSingleBatchCheck bool
}

TensorOutputValidationOptions lets callers tighten tensor response validation. Empty allowlists use the SDK defaults.

type TensorPayload added in v1.2.8 

type TensorPayload struct {
	Payload     []byte
	PayloadMIME string
	Descriptor  TensorDescriptor
}

TensorPayload is a model-ready tensor payload plus the metadata needed to send it through InferRequest.

type TensorPreprocessor added in v1.2.8 

type TensorPreprocessor interface {
	ID() string
	Preprocess(ctx context.Context, input ImageInput) (*TensorPayload, error)
}

type TensorPreprocessorRegistry added in v1.2.8 

type TensorPreprocessorRegistry struct {
	// contains filtered or unexported fields
}

func DefaultTensorPreprocessorRegistry added in v1.2.8 

func DefaultTensorPreprocessorRegistry() *TensorPreprocessorRegistry

func NewTensorPreprocessorRegistry added in v1.2.8 

func NewTensorPreprocessorRegistry(preprocessors ...TensorPreprocessor) *TensorPreprocessorRegistry

func (*TensorPreprocessorRegistry) Lookup added in v1.2.8 

func (r *TensorPreprocessorRegistry) Lookup(id string) (TensorPreprocessor, bool)

func (*TensorPreprocessorRegistry) Register added in v1.2.8 

func (r *TensorPreprocessorRegistry) Register(preprocessor TensorPreprocessor)

type TensorResponse added in v1.1.4 

type TensorResponse struct {
	Descriptor *TensorDescriptor
	Data       []byte
	ResultMime string
	Meta       map[string]string
}

TensorResponse is a validated model-ready tensor returned by an InferResponse.

type TensorValidationOptions added in v1.1.4 

type TensorValidationOptions struct {
	AllowedDTypes        []string
	AllowedLayouts       []string
	AllowedFormats       []string
	AllowedByteOrders    []string
	AllowedPreprocessIDs []string

	// DisableSingleBatchCheck permits tensor payloads whose leading batch
	// dimension is greater than one. It is false by default for v1.
	DisableSingleBatchCheck bool
}

TensorValidationOptions lets task backends tighten the generic tensor contract with task-specific allowlists. Empty allowlists use the SDK defaults, except AllowedPreprocessIDs: when empty, any non-empty preprocess ID is accepted.

type TextGenerationMetadata 

type TextGenerationMetadata struct {
	Temperature      float64 `json:"temperature,omitempty"`
	TopP             float64 `json:"top_p,omitempty"`
	MaxTokens        int     `json:"max_tokens,omitempty"`
	Seed             int64   `json:"seed,omitempty"`
	GenerationTimeMs float64 `json:"generation_time_ms,omitempty"`
	StreamingChunks  int     `json:"streaming_chunks,omitempty"`
}

TextGenerationMetadata contains optional metadata about the text generation process.

This structure captures generation parameters and performance metrics that can be useful for debugging, optimization, and understanding the generation behavior.

type TextGenerationV1 

type TextGenerationV1 struct {
	Text            string                  `json:"text"`
	FinishReason    string                  `json:"finish_reason"`
	GeneratedTokens int                     `json:"generated_tokens"`
	InputTokens     int                     `json:"input_tokens,omitempty"`
	ModelID         string                  `json:"model_id"`
	Metadata        *TextGenerationMetadata `json:"metadata,omitempty"`
}

TextGenerationV1 represents a text generation response from Lumen VLM services.

This structure contains the generated text along with metadata about the generation process, including token counts, completion reasons, and optional generation parameters.

Role in project: Output structure for text generation tasks. Used in chat responses, text completion, summarization, and other natural language generation scenarios.

func (*TextGenerationV1) IsValidFinishReason 

func (t *TextGenerationV1) IsValidFinishReason() bool

IsValidFinishReason checks if the provided finish reason is valid.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.