Documentation
¶
Overview ¶
Package kjarni runs transformer models inside your Go program: text classification, embeddings, semantic search, reranking, chat, speech to text, summarization, translation and image search, with the same output as PyTorch and no Python or server. Kjarni is a transformer inference library; this package calls its native library. Models download automatically on first use and are cached on disk, so everything after that works offline.
The same engine ships for other runtimes: Kjarni on NuGet for C#, kjarni-wasm on npm for the browser, and a command line binary. Documentation and guides live at https://kjarni.ai and the source is at https://github.com/olafurjohannsson/kjarni
Getting started ¶
import "github.com/olafurjohannsson/kjarni-go"
emb, err := kjarni.NewEmbedder("minilm-l6-v2")
if err != nil {
log.Fatal(err)
}
defer emb.Close()
vec, err := emb.Encode("hello world")
Beyond text embeddings ¶
tr, _ := kjarni.NewTranscriber("", kjarni.WithLanguage("en"))
transcript, _ := tr.TranscribeFile("meeting.wav")
sum, _ := kjarni.NewSummarizer("")
summary, _ := sum.Summarize(article)
tl, _ := kjarni.NewTranslator("", kjarni.WithLanguages("en", "de"))
german, _ := tl.Translate("The house is small.")
photos, _ := kjarni.NewImageIndex("")
photos.AddDirectory("Pictures")
hits, _ := photos.Search("a red car", 5)
Index ¶
- func CosineSimilarity(a, b []float32) float32
- type Aggregation
- type Chat
- func (c *Chat) Close() error
- func (c *Chat) ContextSize() int
- func (c *Chat) Conversation() (*ChatConversation, error)
- func (c *Chat) ModelName() string
- func (c *Chat) Send(message string) (string, error)
- func (c *Chat) SendWithConfig(message string, config *GenerationConfig) (string, error)
- func (c *Chat) SendWithHistory(turns []ChatTurn, message string, config *GenerationConfig) (string, error)
- func (c *Chat) Stream(message string, onToken func(token string) bool) error
- func (c *Chat) StreamWithConfig(message string, config *GenerationConfig, onToken func(token string) bool) error
- func (c *Chat) StreamWithHistory(turns []ChatTurn, message string, config *GenerationConfig, ...) error
- type ChatConversation
- func (cc *ChatConversation) Clear(keepSystem bool)
- func (cc *ChatConversation) Close() error
- func (cc *ChatConversation) Length() int
- func (cc *ChatConversation) Send(message string) (string, error)
- func (cc *ChatConversation) SendWithConfig(message string, config *GenerationConfig) (string, error)
- func (cc *ChatConversation) Stream(message string, onToken func(token string) bool) error
- type ChatMode
- type ChatOption
- type ChatRole
- type ChatTurn
- type Classifier
- type ClassifyResult
- type Embedder
- type Entity
- type EntityRecognizer
- type ErrorCode
- type GenerationConfig
- type ImageHit
- type ImageIndex
- func (x *ImageIndex) Add(path string) error
- func (x *ImageIndex) AddDirectory(dir string) (ScanReport, error)
- func (x *ImageIndex) Close() error
- func (x *ImageIndex) Len() int
- func (x *ImageIndex) Load(path string) (int, error)
- func (x *ImageIndex) Save(path string) error
- func (x *ImageIndex) Search(query string, topK int) ([]ImageHit, error)
- type IndexStats
- type Indexer
- type KjarniError
- type LabelScore
- type Option
- func WithCacheDir(dir string) Option
- func WithChunking(size, overlap int) Option
- func WithDevice(device string) Option
- func WithExclude(patterns ...string) Option
- func WithExtensions(extensions ...string) Option
- func WithHidden(include bool) Option
- func WithLanguage(language string) Option
- func WithLanguages(from, to string) Option
- func WithModelPath(dir string) Option
- func WithQuiet(quiet bool) Option
- func WithRecursive(recursive bool) Option
- func WithSummaryLength(min, max int) Option
- func WithTranslate(translate bool) Option
- type RerankResult
- type Reranker
- type ScanReport
- type SearchMode
- type SearchOptions
- type SearchResult
- type Searcher
- type Segment
- type Summarizer
- type Transcriber
- type Transcript
- type Translator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CosineSimilarity ¶
CosineSimilarity computes cosine similarity between two vectors in Go.
Types ¶
type Aggregation ¶ added in v0.5.0
type Aggregation int32
Aggregation decides how tokens become entities, as transformers' aggregation_strategy.
const ( // AggregationFirst gives a word the label of its first token, so it is never split. AggregationFirst Aggregation = 0 // AggregationSimple keeps each token's label; neighbours with the same tag join. AggregationSimple Aggregation = 1 // AggregationMax gives a word the label of its most confident token. AggregationMax Aggregation = 2 // AggregationAverage gives a word the label with the highest mean probability. AggregationAverage Aggregation = 3 )
type Chat ¶ added in v0.1.9
type Chat struct {
// contains filtered or unexported fields
}
Chat runs a local language model in-process. It is safe for concurrent use; calls are serialized. Close releases the model.
func NewChat ¶ added in v0.1.9
NewChat loads a chat model by registry name, for example "llama3.2-3b-instruct", downloading it on first use. With WithModelPath it loads a local Hugging Face model folder instead, model may be empty, and the conversation is formatted with the folder's own chat template.
opts takes ChatOption values (WithSystemPrompt, WithChatMode, WithQuantized) and, of the shared options, WithQuiet, WithDevice, WithCacheDir and WithModelPath.
func (*Chat) ContextSize ¶ added in v0.1.9
returns the model's context window size
func (*Chat) Conversation ¶ added in v0.1.9
func (c *Chat) Conversation() (*ChatConversation, error)
create a stateful conversation with history management
func (*Chat) SendWithConfig ¶ added in v0.1.9
func (c *Chat) SendWithConfig(message string, config *GenerationConfig) (string, error)
sends a message with custom generation config
func (*Chat) SendWithHistory ¶ added in v0.2.5
func (c *Chat) SendWithHistory(turns []ChatTurn, message string, config *GenerationConfig) (string, error)
SendWithHistory replies to message given prior turns, oldest first, without keeping any state. It suits callers that own the transcript and replay it every turn; ChatConversation is simpler otherwise. A RoleSystem turn replaces the system prompt given to NewChat. config may be nil for the defaults.
func (*Chat) Stream ¶ added in v0.1.9
Stream sends a message and calls onToken for each generated token. Return false from onToken to stop generation.
func (*Chat) StreamWithConfig ¶ added in v0.1.9
func (c *Chat) StreamWithConfig(message string, config *GenerationConfig, onToken func(token string) bool) error
streams with custom generation config.
func (*Chat) StreamWithHistory ¶ added in v0.2.5
func (c *Chat) StreamWithHistory(turns []ChatTurn, message string, config *GenerationConfig, onToken func(token string) bool) error
StreamWithHistory is SendWithHistory, streamed through onToken. Return false from onToken to stop.
type ChatConversation ¶ added in v0.1.9
type ChatConversation struct {
// contains filtered or unexported fields
}
ChatConversation maintains chat history parent Chat must outlive this object
func (*ChatConversation) Clear ¶ added in v0.1.9
func (cc *ChatConversation) Clear(keepSystem bool)
clear the conversation history
func (*ChatConversation) Close ¶ added in v0.1.9
func (cc *ChatConversation) Close() error
release the conversation resources
func (*ChatConversation) Length ¶ added in v0.1.9
func (cc *ChatConversation) Length() int
returns the number of messages in history
func (*ChatConversation) Send ¶ added in v0.1.9
func (cc *ChatConversation) Send(message string) (string, error)
Send sends a message and returns a response Both user message and response are added to history
func (*ChatConversation) SendWithConfig ¶ added in v0.1.9
func (cc *ChatConversation) SendWithConfig(message string, config *GenerationConfig) (string, error)
send with custom generation config.
type ChatOption ¶ added in v0.1.9
type ChatOption func(*chatOptions)
ChatOption configures NewChat.
func WithChatMode ¶ added in v0.1.9
func WithChatMode(mode ChatMode) ChatOption
WithChatMode picks the sampling defaults; see ChatMode.
func WithQuantized ¶ added in v0.2.6
func WithQuantized(quantized bool) ChatOption
WithQuantized loads the model's 4-bit GGUF weights instead of full precision: a fraction of the memory and several times faster to generate on a CPU.
func WithSystemPrompt ¶ added in v0.1.9
func WithSystemPrompt(prompt string) ChatOption
WithSystemPrompt sets the instructions the model sees before every message.
type Classifier ¶
type Classifier struct {
// contains filtered or unexported fields
}
Classifier runs text classification using a pre-trained model.
func NewClassifier ¶
func NewClassifier(model string, opts ...Option) (*Classifier, error)
NewClassifier creates a classifier for the given model. Available models: distilbert-sentiment, roberta-sentiment, bert-sentiment-multilingual, distilroberta-emotion, roberta-emotions, toxic-bert. Models download automatically on first use and are cached locally.
func (*Classifier) Classify ¶
func (c *Classifier) Classify(text string) (*ClassifyResult, error)
Classify runs the model on the given text and returns scored labels.
func (*Classifier) ClassifyBatch ¶ added in v0.2.6
func (c *Classifier) ClassifyBatch(texts []string) ([]*ClassifyResult, error)
ClassifyBatch classifies several texts in one call, which is much faster than one at a time. It returns one result per text, in order.
func (*Classifier) Close ¶
func (c *Classifier) Close() error
Close releases the classifier resources. Safe to call multiple times.
func (*Classifier) NumLabels ¶
func (c *Classifier) NumLabels() int
NumLabels returns the number of labels the model supports.
type ClassifyResult ¶
type ClassifyResult struct {
Label string
Score float32
AllScores []LabelScore
}
ClassifyResult holds the output of a classification. Label and Score contain the top prediction. AllScores contains scores for every label.
func (*ClassifyResult) String ¶
func (r *ClassifyResult) String() string
String returns the result as "label (score%)".
func (*ClassifyResult) ToJSON ¶
func (r *ClassifyResult) ToJSON() string
ToJSON returns the result as a JSON string.
type Embedder ¶
type Embedder struct {
// contains filtered or unexported fields
}
Embedder encodes text into vector embeddings for similarity and search.
func NewEmbedder ¶
NewEmbedder creates an embedder for the given model. Available models: minilm-l6-v2 (384d), mpnet-base-v2 (768d), distilbert-base (768d). Models download automatically on first use and are cached locally.
func (*Embedder) EncodeBatch ¶
EncodeBatch encodes multiple texts and returns their embedding vectors.
type Entity ¶ added in v0.5.0
Entity is one named entity. Start and End are byte offsets into the input, so text[e.Start:e.End] == e.Text.
type EntityRecognizer ¶ added in v0.5.0
type EntityRecognizer struct {
// contains filtered or unexported fields
}
EntityRecognizer finds names of people, places, organizations and other entities with a token classification model. The entities are those transformers' pipeline("ner") gives for the same aggregation.
func NewEntityRecognizer ¶ added in v0.5.0
func NewEntityRecognizer(model string, aggregation Aggregation, opts ...Option) (*EntityRecognizer, error)
NewEntityRecognizer loads an entity recognizer. An empty model means the default, distilbert-ner, for English. WithModelPath loads any *ForTokenClassification model from a folder instead.
func (*EntityRecognizer) Close ¶ added in v0.5.0
func (r *EntityRecognizer) Close() error
Close frees the native model. Calling it twice is safe.
type ErrorCode ¶
type ErrorCode int32
ErrorCode represents error codes returned by the kjarni engine.
const ( ErrOk ErrorCode = 0 ErrNullPointer ErrorCode = 1 ErrInvalidUtf8 ErrorCode = 2 ErrModelNotFound ErrorCode = 3 ErrLoadFailed ErrorCode = 4 ErrInferenceFailed ErrorCode = 5 ErrInvalidConfig ErrorCode = 7 ErrCancelled ErrorCode = 8 ErrTimeout ErrorCode = 9 ErrStreamEnded ErrorCode = 10 // ErrPanic reports a panic in the native engine caught at the ABI boundary. The // operation failed and its outputs are undefined; the handle stays valid to Close. ErrPanic ErrorCode = 11 ErrUnknown ErrorCode = 255 )
type GenerationConfig ¶ added in v0.1.9
type GenerationConfig struct {
// Temperature: higher is more random.
Temperature float32
// TopK samples only from the k most likely tokens.
TopK int32
// TopP samples from the smallest set of tokens whose probability sums to p.
TopP float32
// MinP drops tokens less likely than this fraction of the most likely one.
MinP float32
// RepetitionPenalty above 1 discourages repeating earlier tokens; 1 disables it.
RepetitionPenalty float32
// MaxNewTokens caps the length of the reply.
MaxNewTokens int32
// DoSample is 1 to sample, 0 to decode greedily, -1 for the default.
DoSample int32
}
GenerationConfig controls how a reply is generated. A negative value keeps the model's default for that field.
func CreativeGenerationConfig ¶ added in v0.1.9
func CreativeGenerationConfig() GenerationConfig
CreativeGenerationConfig samples at temperature 0.9 with top-k 50 and top-p 0.95.
func DefaultGenerationConfig ¶ added in v0.1.9
func DefaultGenerationConfig() GenerationConfig
DefaultGenerationConfig keeps every model default.
func GreedyGenerationConfig ¶ added in v0.1.9
func GreedyGenerationConfig() GenerationConfig
GreedyGenerationConfig always picks the most likely token, so the same prompt gives the same reply every time.
type ImageHit ¶ added in v0.2.5
type ImageHit struct {
Path string
// Score is a cosine similarity, small even for good matches, so only the
// order is meaningful.
Score float32
}
ImageHit is an image matching a search.
type ImageIndex ¶ added in v0.2.5
type ImageIndex struct {
// contains filtered or unexported fields
}
ImageIndex searches photos by description, with CLIP.
func NewImageIndex ¶ added in v0.2.5
func NewImageIndex(modelDir string) (*ImageIndex, error)
NewImageIndex loads a CLIP model from modelDir, or clip-vit-base-32 for "", downloaded on first use.
func (*ImageIndex) Add ¶ added in v0.2.5
func (x *ImageIndex) Add(path string) error
Add indexes one JPEG or PNG.
func (*ImageIndex) AddDirectory ¶ added in v0.2.5
func (x *ImageIndex) AddDirectory(dir string) (ScanReport, error)
AddDirectory indexes every JPEG and PNG under dir.
func (*ImageIndex) Close ¶ added in v0.2.5
func (x *ImageIndex) Close() error
Close releases the model. Safe to call more than once.
func (*ImageIndex) Len ¶ added in v0.2.5
func (x *ImageIndex) Len() int
Len reports how many images are indexed.
func (*ImageIndex) Load ¶ added in v0.2.5
func (x *ImageIndex) Load(path string) (int, error)
Load restores vectors written by Save and returns how many.
func (*ImageIndex) Save ¶ added in v0.2.5
func (x *ImageIndex) Save(path string) error
Save writes the vectors (not the images) to path.
type IndexStats ¶
type IndexStats struct {
DocumentsIndexed int
ChunksCreated int
Dimension int
SizeBytes uint64
FilesProcessed int
FilesSkipped int
ElapsedMs uint64
}
IndexStats holds statistics from an indexing operation.
type Indexer ¶
type Indexer struct {
// contains filtered or unexported fields
}
Indexer creates search indexes from files in a directory.
func NewIndexer ¶
NewIndexer creates an indexer using the given embedding model. The model is used to generate vectors for each text chunk during indexing.
func (*Indexer) Add ¶ added in v0.2.5
Add indexes more files and directories into the existing index at indexPath, returning how many chunks were added.
func (*Indexer) Create ¶
func (idx *Indexer) Create(indexPath string, inputs []string) (*IndexStats, error)
Create builds a new search index at indexPath from the given files and directories. Files are chunked, embedded, and stored for later retrieval with a Searcher. It fails if an index already exists there; see Recreate.
type KjarniError ¶
KjarniError is an error returned by the kjarni engine.
func (*KjarniError) Error ¶
func (e *KjarniError) Error() string
Error returns the engine's message and error code.
type LabelScore ¶
LabelScore is a single label with its confidence score.
type Option ¶
type Option func(*options)
Option configures a classifier, embedder, or other kjarni component.
func WithCacheDir ¶ added in v0.2.5
WithCacheDir sets where models are downloaded to and loaded from.
func WithChunking ¶ added in v0.2.5
WithChunking sets an Indexer's chunk size and the overlap between chunks, in characters.
func WithDevice ¶
WithDevice sets the compute device. Supported values: "cpu", "gpu".
func WithExclude ¶ added in v0.2.5
WithExclude makes an Indexer skip names or `*` patterns, e.g. "node_modules", "*.min.js".
func WithExtensions ¶ added in v0.2.5
WithExtensions limits an Indexer to files with these extensions, e.g. "md", "txt".
func WithHidden ¶ added in v0.2.5
WithHidden makes an Indexer include hidden files and directories.
func WithLanguage ¶ added in v0.2.5
WithLanguage sets the spoken language for a Transcriber as an ISO code such as "en" or "is". Without it the language is detected from the audio.
func WithLanguages ¶ added in v0.2.5
WithLanguages sets a Translator's default source and target languages.
func WithModelPath ¶ added in v0.5.0
WithModelPath loads the model from a local folder, such as one downloaded from the Hugging Face Hub, instead of by name. Embedder, Classifier, Reranker, EntityRecognizer, Chat, Summarizer, Translator and Transcriber accept it.
func WithRecursive ¶ added in v0.2.5
WithRecursive sets whether an Indexer descends into subdirectories (the default).
func WithSummaryLength ¶ added in v0.2.5
WithSummaryLength bounds a Summarizer's output in tokens; 0 keeps the model's default.
func WithTranslate ¶ added in v0.2.5
WithTranslate makes a Transcriber translate the speech into English.
type RerankResult ¶
RerankResult holds a single reranked document with its relevance score.
type Reranker ¶
type Reranker struct {
// contains filtered or unexported fields
}
Reranker scores query-document relevance using a cross-encoder model.
func NewReranker ¶
NewReranker creates a reranker using the default cross-encoder model. The model downloads automatically on first use and is cached locally.
func (*Reranker) Rerank ¶
func (r *Reranker) Rerank(query string, documents []string) ([]RerankResult, error)
Rerank scores all documents and returns them sorted by relevance to the query.
func (*Reranker) RerankTopK ¶
RerankTopK scores all documents and returns the top k sorted by relevance.
type ScanReport ¶ added in v0.2.5
ScanReport says what scanning a directory did.
type SearchMode ¶
type SearchMode int
SearchMode determines the search strategy.
const ( // Keyword uses BM25 term matching. Keyword SearchMode = 0 // Semantic uses vector similarity. Semantic SearchMode = 1 // Hybrid combines BM25 and vector similarity. Hybrid SearchMode = 2 )
type SearchOptions ¶ added in v0.2.5
type SearchOptions struct {
// TopK is the most results to return; 0 means 10.
TopK int
// NoRerank skips the reranker for this query when the Searcher has one.
NoRerank bool
// Threshold drops results scoring below it.
Threshold float32
// SourcePattern keeps only chunks from files matching this `*` pattern, e.g. "*.md".
SourcePattern string
// FilterKey and FilterValue keep only chunks whose metadata has that pair.
FilterKey string
FilterValue string
}
SearchOptions overrides a Searcher's defaults for one query. The zero value changes nothing.
type SearchResult ¶
type SearchResult struct {
Score float32
Text string
DocumentID int
// Metadata stored with the chunk, such as "source", the file it came from.
Metadata map[string]any
}
SearchResult holds a single search result with its relevance score.
type Searcher ¶
type Searcher struct {
// contains filtered or unexported fields
}
Searcher queries indexes created by an Indexer.
func NewSearcher ¶
NewSearcher creates a searcher using the given embedding model. Pass a non-empty rerankerModel to enable cross-encoder reranking of results. Pass an empty string to disable reranking.
func (*Searcher) Search ¶
func (s *Searcher) Search(indexPath string, query string, mode SearchMode) ([]SearchResult, error)
Search queries the index at indexPath and returns up to 10 results using the given mode, reranked when the Searcher has a reranker.
func (*Searcher) SearchWithOptions ¶ added in v0.2.5
func (s *Searcher) SearchWithOptions(indexPath string, query string, mode SearchMode, opts SearchOptions) ([]SearchResult, error)
SearchWithOptions is Search with per-query overrides.
type Segment ¶ added in v0.2.5
Segment is one timed piece of a transcript, in seconds from the start.
type Summarizer ¶ added in v0.2.5
type Summarizer struct {
// contains filtered or unexported fields
}
Summarizer summarizes article-length text.
func NewSummarizer ¶ added in v0.2.5
func NewSummarizer(model string, opts ...Option) (*Summarizer, error)
NewSummarizer loads a summarization model, downloading it on first use. Pass "" for distilbart-cnn. See WithSummaryLength.
func (*Summarizer) Close ¶ added in v0.2.5
func (s *Summarizer) Close() error
Close releases the model. Safe to call more than once.
type Transcriber ¶ added in v0.2.5
type Transcriber struct {
// contains filtered or unexported fields
}
Transcriber turns speech into text with Whisper.
func NewTranscriber ¶ added in v0.2.5
func NewTranscriber(model string, opts ...Option) (*Transcriber, error)
NewTranscriber loads a Whisper model, downloading it on first use. Pass "" for whisper-small. See WithLanguage and WithTranslate.
func (*Transcriber) Close ¶ added in v0.2.5
func (t *Transcriber) Close() error
Close releases the model. Safe to call more than once.
func (*Transcriber) Transcribe ¶ added in v0.2.5
func (t *Transcriber) Transcribe(samples []float32, sampleRate int) (*Transcript, error)
Transcribe transcribes mono samples in -1..1 at any sample rate.
func (*Transcriber) TranscribeFile ¶ added in v0.2.5
func (t *Transcriber) TranscribeFile(path string) (*Transcript, error)
TranscribeFile transcribes a wav, mp3, flac or ogg file.
type Transcript ¶ added in v0.2.5
type Transcript struct {
Text string
// Language is the ISO code transcribed: the one given, or the one detected.
Language string
DurationSecs float32
Segments []Segment
}
Transcript is the result of transcribing audio.
type Translator ¶ added in v0.2.5
type Translator struct {
// contains filtered or unexported fields
}
Translator translates text between languages.
func NewTranslator ¶ added in v0.2.5
func NewTranslator(model string, opts ...Option) (*Translator, error)
NewTranslator loads a translation model, downloading it on first use. Pass "" to pick from WithLanguages, which sets the languages Translate uses by default: Opus-MT for English and German, otherwise flan-t5-base, which often returns the input unchanged.
func (*Translator) Close ¶ added in v0.2.5
func (t *Translator) Close() error
Close releases the model. Safe to call more than once.
func (*Translator) Translate ¶ added in v0.2.5
func (t *Translator) Translate(text string) (string, error)
Translate translates text between the default languages given to NewTranslator.
func (*Translator) TranslateBetween ¶ added in v0.2.5
func (t *Translator) TranslateBetween(text, from, to string) (string, error)
TranslateBetween translates text from one language to another, e.g. "en" to "de" or "english" to "german". An empty language falls back to the default; with neither, the error has code ErrInvalidConfig.
