model

Documentation

Overview

Package model handles CLIP ONNX model downloading, loading, and inference.

Index

• Constants
• Variables
• func EnsureModels(progressFn func(filename string, downloaded, total int64)) error
• func FilePath(name string) (string, error)
• func IsUnicodeLetter(r rune) bool
• func ModelsDir() (string, error)
• func PreprocessImage(path string) ([]float32, error)
• type CLIPSession
  • func NewCLIPSession(explicitPath string) (*CLIPSession, error)
  • func (c *CLIPSession) Classify(imagePath string, categories []string) (map[string]float32, error)
  • func (c *CLIPSession) Destroy()
• type ModelFile
• type Tokenizer
  • func LoadTokenizer(vocabPath, mergesPath string) (*Tokenizer, error)
  • func TokenizerFromModelsDir() (*Tokenizer, error)
  • func (t *Tokenizer) Encode(text string) []int64
  • func (t *Tokenizer) EncodeCategories(categories []string) []int64

Constants

const BaselineCategory = "uncategorized"

BaselineCategory is the internal label for the baseline "catch-all" prompt used to prevent false-positive classification.

Variables

var RequiredFiles = []ModelFile{
	{
		Name: "model.onnx",
		URL:  hfBaseURL + "/onnx/model.onnx",
	},
	{
		Name: "vocab.json",
		URL:  hfBaseURL + "/vocab.json",
	},
	{
		Name: "merges.txt",
		URL:  hfBaseURL + "/merges.txt",
	},
}

RequiredFiles defines all files needed for CLIP inference.

Functions

func EnsureModels

func EnsureModels(progressFn func(filename string, downloaded, total int64)) error

EnsureModels checks that all required files exist, downloading any that are missing.

func FilePath

func FilePath(name string) (string, error)

FilePath returns the full path to a named file in the models directory.

func IsUnicodeLetter

func IsUnicodeLetter(r rune) bool

IsUnicodeLetter checks if a rune is a unicode letter (exported for testing).

func ModelsDir

func ModelsDir() (string, error)

ModelsDir returns the path to the model storage directory (~/.imgsort/models/).

func PreprocessImage

func PreprocessImage(path string) ([]float32, error)

PreprocessImage loads an image file and returns a float32 tensor in [1, 3, 224, 224] CHW format, normalized for CLIP.

Types

type CLIPSession

type CLIPSession struct {
	// contains filtered or unexported fields
}

CLIPSession holds a loaded CLIP model ready for inference.

func NewCLIPSession

func NewCLIPSession(explicitPath string) (*CLIPSession, error)

NewCLIPSession creates a new CLIP inference session. If explicitPath is empty, it tries the embedded library first, then platform defaults.

func (*CLIPSession) Classify

func (c *CLIPSession) Classify(imagePath string, categories []string) (map[string]float32, error)

Classify runs zero-shot classification on an image against the given categories. A baseline "uncategorized" prompt is injected to prevent false positives (especially with few categories). Returns a map of category names to their similarity scores (after softmax), including the baseline.

func (*CLIPSession) Destroy

func (c *CLIPSession) Destroy()

Destroy releases resources held by the CLIP session.

type ModelFile

type ModelFile struct {
	Name   string
	URL    string
	SHA256 string // expected hash (empty = skip verification)
}

ModelFile describes a file to download.

type Tokenizer

type Tokenizer struct {
	// contains filtered or unexported fields
}

Tokenizer implements CLIP's BPE tokenization.

func LoadTokenizer

func LoadTokenizer(vocabPath, mergesPath string) (*Tokenizer, error)

LoadTokenizer loads the tokenizer from vocab.json and merges.txt files.

func TokenizerFromModelsDir

func TokenizerFromModelsDir() (*Tokenizer, error)

TokenizerFromModelsDir loads the tokenizer from the standard models directory.

func (*Tokenizer) Encode

func (t *Tokenizer) Encode(text string) []int64

Encode tokenizes a text string and returns token IDs padded/truncated to contextLen.

func (*Tokenizer) EncodeCategories

func (t *Tokenizer) EncodeCategories(categories []string) []int64

EncodeCategories tokenizes a batch of category labels using CLIP's prompt template.