Documentation
¶
Overview ¶
Package kjarni provides text classification, embeddings, semantic search, reranking, chat, speech to text, summarization, translation and image search using pre-trained models, running inside your process against a native library. Models download automatically on first use and are cached on disk, so everything after that works offline.
The same engine ships for other runtimes: Kjarni on NuGet for C#, kjarni-wasm on npm for the browser, and a command line binary. Documentation and guides live at https://kjarni.ai and the source is at https://github.com/olafurjohannsson/kjarni
Getting started ¶
import "github.com/olafurjohannsson/kjarni-go"
emb, err := kjarni.NewEmbedder("minilm-l6-v2")
if err != nil {
log.Fatal(err)
}
defer emb.Close()
vec, err := emb.Encode("hello world")
Beyond text embeddings ¶
tr, _ := kjarni.NewTranscriber("", kjarni.WithLanguage("en"))
transcript, _ := tr.TranscribeFile("meeting.wav")
sum, _ := kjarni.NewSummarizer("")
summary, _ := sum.Summarize(article)
tl, _ := kjarni.NewTranslator("", kjarni.WithLanguages("en", "de"))
german, _ := tl.Translate("The house is small.")
photos, _ := kjarni.NewImageIndex("")
photos.AddDirectory("Pictures")
hits, _ := photos.Search("a red car", 5)
Index ¶
- func CosineSimilarity(a, b []float32) float32
- type Chat
- func (c *Chat) Close() error
- func (c *Chat) ContextSize() int
- func (c *Chat) Conversation() (*ChatConversation, error)
- func (c *Chat) ModelName() string
- func (c *Chat) Send(message string) (string, error)
- func (c *Chat) SendWithConfig(message string, config *GenerationConfig) (string, error)
- func (c *Chat) SendWithHistory(turns []ChatTurn, message string, config *GenerationConfig) (string, error)
- func (c *Chat) Stream(message string, onToken func(token string) bool) error
- func (c *Chat) StreamWithConfig(message string, config *GenerationConfig, onToken func(token string) bool) error
- func (c *Chat) StreamWithHistory(turns []ChatTurn, message string, config *GenerationConfig, ...) error
- type ChatConversation
- func (cc *ChatConversation) Clear(keepSystem bool)
- func (cc *ChatConversation) Close() error
- func (cc *ChatConversation) Length() int
- func (cc *ChatConversation) Send(message string) (string, error)
- func (cc *ChatConversation) SendWithConfig(message string, config *GenerationConfig) (string, error)
- func (cc *ChatConversation) Stream(message string, onToken func(token string) bool) error
- type ChatMode
- type ChatOption
- type ChatRole
- type ChatTurn
- type Classifier
- type ClassifyResult
- type Embedder
- type ErrorCode
- type GenerationConfig
- type ImageHit
- type ImageIndex
- func (x *ImageIndex) Add(path string) error
- func (x *ImageIndex) AddDirectory(dir string) (ScanReport, error)
- func (x *ImageIndex) Close() error
- func (x *ImageIndex) Len() int
- func (x *ImageIndex) Load(path string) (int, error)
- func (x *ImageIndex) Save(path string) error
- func (x *ImageIndex) Search(query string, topK int) ([]ImageHit, error)
- type IndexStats
- type Indexer
- type KjarniError
- type LabelScore
- type Option
- func WithCacheDir(dir string) Option
- func WithChunking(size, overlap int) Option
- func WithDevice(device string) Option
- func WithExclude(patterns ...string) Option
- func WithExtensions(extensions ...string) Option
- func WithHidden(include bool) Option
- func WithLanguage(language string) Option
- func WithLanguages(from, to string) Option
- func WithQuiet(quiet bool) Option
- func WithRecursive(recursive bool) Option
- func WithSummaryLength(min, max int) Option
- func WithTranslate(translate bool) Option
- type RerankResult
- type Reranker
- type ScanReport
- type SearchMode
- type SearchOptions
- type SearchResult
- type Searcher
- type Segment
- type Summarizer
- type Transcriber
- type Transcript
- type Translator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CosineSimilarity ¶
CosineSimilarity computes cosine similarity between two vectors in Go.
Types ¶
type Chat ¶ added in v0.1.9
type Chat struct {
// contains filtered or unexported fields
}
func (*Chat) ContextSize ¶ added in v0.1.9
returns the model's context window size
func (*Chat) Conversation ¶ added in v0.1.9
func (c *Chat) Conversation() (*ChatConversation, error)
create a stateful conversation with history management
func (*Chat) SendWithConfig ¶ added in v0.1.9
func (c *Chat) SendWithConfig(message string, config *GenerationConfig) (string, error)
sends a message with custom generation config
func (*Chat) SendWithHistory ¶ added in v0.2.5
func (c *Chat) SendWithHistory(turns []ChatTurn, message string, config *GenerationConfig) (string, error)
SendWithHistory replies to message given prior turns, oldest first, without keeping any state. It suits callers that own the transcript and replay it every turn; ChatConversation is simpler otherwise. A RoleSystem turn replaces the system prompt given to NewChat. config may be nil for the defaults.
func (*Chat) Stream ¶ added in v0.1.9
Stream sends a message and calls onToken for each generated token. Return false from onToken to stop generation.
func (*Chat) StreamWithConfig ¶ added in v0.1.9
func (c *Chat) StreamWithConfig(message string, config *GenerationConfig, onToken func(token string) bool) error
streams with custom generation config.
func (*Chat) StreamWithHistory ¶ added in v0.2.5
func (c *Chat) StreamWithHistory(turns []ChatTurn, message string, config *GenerationConfig, onToken func(token string) bool) error
StreamWithHistory is SendWithHistory, streamed through onToken. Return false from onToken to stop.
type ChatConversation ¶ added in v0.1.9
type ChatConversation struct {
// contains filtered or unexported fields
}
ChatConversation maintains chat history parent Chat must outlive this object
func (*ChatConversation) Clear ¶ added in v0.1.9
func (cc *ChatConversation) Clear(keepSystem bool)
clear the conversation history
func (*ChatConversation) Close ¶ added in v0.1.9
func (cc *ChatConversation) Close() error
release the conversation resources
func (*ChatConversation) Length ¶ added in v0.1.9
func (cc *ChatConversation) Length() int
returns the number of messages in history
func (*ChatConversation) Send ¶ added in v0.1.9
func (cc *ChatConversation) Send(message string) (string, error)
Send sends a message and returns a response Both user message and response are added to history
func (*ChatConversation) SendWithConfig ¶ added in v0.1.9
func (cc *ChatConversation) SendWithConfig(message string, config *GenerationConfig) (string, error)
send with custom generation config.
type ChatOption ¶ added in v0.1.9
type ChatOption func(*chatOptions)
func WithChatMode ¶ added in v0.1.9
func WithChatMode(mode ChatMode) ChatOption
func WithQuantized ¶ added in v0.2.6
func WithQuantized(quantized bool) ChatOption
WithQuantized loads the model's 4-bit GGUF weights instead of full precision: a fraction of the memory and several times faster to generate on a CPU.
func WithSystemPrompt ¶ added in v0.1.9
func WithSystemPrompt(prompt string) ChatOption
type Classifier ¶
type Classifier struct {
// contains filtered or unexported fields
}
Classifier runs text classification using a pre-trained model.
func NewClassifier ¶
func NewClassifier(model string, opts ...Option) (*Classifier, error)
NewClassifier creates a classifier for the given model. Available models: distilbert-sentiment, roberta-sentiment, bert-sentiment-multilingual, distilroberta-emotion, roberta-emotions, toxic-bert. Models download automatically on first use and are cached locally.
func (*Classifier) Classify ¶
func (c *Classifier) Classify(text string) (*ClassifyResult, error)
Classify runs the model on the given text and returns scored labels.
func (*Classifier) ClassifyBatch ¶ added in v0.2.6
func (c *Classifier) ClassifyBatch(texts []string) ([]*ClassifyResult, error)
ClassifyBatch classifies several texts in one call, which is much faster than one at a time. It returns one result per text, in order.
func (*Classifier) Close ¶
func (c *Classifier) Close() error
Close releases the classifier resources. Safe to call multiple times.
func (*Classifier) NumLabels ¶
func (c *Classifier) NumLabels() int
NumLabels returns the number of labels the model supports.
type ClassifyResult ¶
type ClassifyResult struct {
Label string
Score float32
AllScores []LabelScore
}
ClassifyResult holds the output of a classification. Label and Score contain the top prediction. AllScores contains scores for every label.
func (*ClassifyResult) String ¶
func (r *ClassifyResult) String() string
String returns the result as "label (score%)".
func (*ClassifyResult) ToJSON ¶
func (r *ClassifyResult) ToJSON() string
ToJSON returns the result as a JSON string.
type Embedder ¶
type Embedder struct {
// contains filtered or unexported fields
}
Embedder encodes text into vector embeddings for similarity and search.
func NewEmbedder ¶
NewEmbedder creates an embedder for the given model. Available models: minilm-l6-v2 (384d), mpnet-base-v2 (768d), distilbert-base (768d). Models download automatically on first use and are cached locally.
func (*Embedder) EncodeBatch ¶
EncodeBatch encodes multiple texts and returns their embedding vectors.
type ErrorCode ¶
type ErrorCode int32
ErrorCode represents error codes returned by the kjarni engine.
const ( ErrOk ErrorCode = 0 ErrNullPointer ErrorCode = 1 ErrInvalidUtf8 ErrorCode = 2 ErrModelNotFound ErrorCode = 3 ErrLoadFailed ErrorCode = 4 ErrInferenceFailed ErrorCode = 5 ErrInvalidConfig ErrorCode = 7 ErrCancelled ErrorCode = 8 ErrTimeout ErrorCode = 9 ErrStreamEnded ErrorCode = 10 // ErrPanic reports a panic in the native engine caught at the ABI boundary. The // operation failed and its outputs are undefined; the handle stays valid to Close. ErrPanic ErrorCode = 11 ErrUnknown ErrorCode = 255 )
type GenerationConfig ¶ added in v0.1.9
type GenerationConfig struct {
Temperature float32
TopK int32
TopP float32
MinP float32
RepetitionPenalty float32
MaxNewTokens int32
DoSample int32
}
func CreativeGenerationConfig ¶ added in v0.1.9
func CreativeGenerationConfig() GenerationConfig
func DefaultGenerationConfig ¶ added in v0.1.9
func DefaultGenerationConfig() GenerationConfig
func GreedyGenerationConfig ¶ added in v0.1.9
func GreedyGenerationConfig() GenerationConfig
type ImageHit ¶ added in v0.2.5
type ImageHit struct {
Path string
// Score is a cosine similarity, small even for good matches, so only the
// order is meaningful.
Score float32
}
ImageHit is an image matching a search.
type ImageIndex ¶ added in v0.2.5
type ImageIndex struct {
// contains filtered or unexported fields
}
ImageIndex searches photos by description, with CLIP.
func NewImageIndex ¶ added in v0.2.5
func NewImageIndex(modelDir string) (*ImageIndex, error)
NewImageIndex loads a CLIP model from modelDir, or clip-vit-base-32 for "", downloaded on first use.
func (*ImageIndex) Add ¶ added in v0.2.5
func (x *ImageIndex) Add(path string) error
Add indexes one JPEG or PNG.
func (*ImageIndex) AddDirectory ¶ added in v0.2.5
func (x *ImageIndex) AddDirectory(dir string) (ScanReport, error)
AddDirectory indexes every JPEG and PNG under dir.
func (*ImageIndex) Close ¶ added in v0.2.5
func (x *ImageIndex) Close() error
Close releases the model. Safe to call more than once.
func (*ImageIndex) Len ¶ added in v0.2.5
func (x *ImageIndex) Len() int
Len reports how many images are indexed.
func (*ImageIndex) Load ¶ added in v0.2.5
func (x *ImageIndex) Load(path string) (int, error)
Load restores vectors written by Save and returns how many.
func (*ImageIndex) Save ¶ added in v0.2.5
func (x *ImageIndex) Save(path string) error
Save writes the vectors (not the images) to path.
type IndexStats ¶
type IndexStats struct {
DocumentsIndexed int
ChunksCreated int
Dimension int
SizeBytes uint64
FilesProcessed int
FilesSkipped int
ElapsedMs uint64
}
IndexStats holds statistics from an indexing operation.
type Indexer ¶
type Indexer struct {
// contains filtered or unexported fields
}
Indexer creates search indexes from files in a directory.
func NewIndexer ¶
NewIndexer creates an indexer using the given embedding model. The model is used to generate vectors for each text chunk during indexing.
func (*Indexer) Add ¶ added in v0.2.5
Add indexes more files and directories into the existing index at indexPath, returning how many chunks were added.
func (*Indexer) Create ¶
func (idx *Indexer) Create(indexPath string, inputs []string) (*IndexStats, error)
Create builds a new search index at indexPath from the given files and directories. Files are chunked, embedded, and stored for later retrieval with a Searcher. It fails if an index already exists there; see Recreate.
type KjarniError ¶
KjarniError is an error returned by the kjarni engine.
func (*KjarniError) Error ¶
func (e *KjarniError) Error() string
type LabelScore ¶
LabelScore is a single label with its confidence score.
type Option ¶
type Option func(*options)
Option configures a classifier, embedder, or other kjarni component.
func WithCacheDir ¶ added in v0.2.5
WithCacheDir sets where models are downloaded to and loaded from.
func WithChunking ¶ added in v0.2.5
WithChunking sets an Indexer's chunk size and the overlap between chunks, in characters.
func WithDevice ¶
WithDevice sets the compute device. Supported values: "cpu", "gpu".
func WithExclude ¶ added in v0.2.5
WithExclude makes an Indexer skip names or `*` patterns, e.g. "node_modules", "*.min.js".
func WithExtensions ¶ added in v0.2.5
WithExtensions limits an Indexer to files with these extensions, e.g. "md", "txt".
func WithHidden ¶ added in v0.2.5
WithHidden makes an Indexer include hidden files and directories.
func WithLanguage ¶ added in v0.2.5
WithLanguage sets the spoken language for a Transcriber as an ISO code such as "en" or "is". Without it the language is detected from the audio.
func WithLanguages ¶ added in v0.2.5
WithLanguages sets a Translator's default source and target languages.
func WithRecursive ¶ added in v0.2.5
WithRecursive sets whether an Indexer descends into subdirectories (the default).
func WithSummaryLength ¶ added in v0.2.5
WithSummaryLength bounds a Summarizer's output in tokens; 0 keeps the model's default.
func WithTranslate ¶ added in v0.2.5
WithTranslate makes a Transcriber translate the speech into English.
type RerankResult ¶
RerankResult holds a single reranked document with its relevance score.
type Reranker ¶
type Reranker struct {
// contains filtered or unexported fields
}
Reranker scores query-document relevance using a cross-encoder model.
func NewReranker ¶
NewReranker creates a reranker using the default cross-encoder model. The model downloads automatically on first use and is cached locally.
func (*Reranker) Rerank ¶
func (r *Reranker) Rerank(query string, documents []string) ([]RerankResult, error)
Rerank scores all documents and returns them sorted by relevance to the query.
func (*Reranker) RerankTopK ¶
RerankTopK scores all documents and returns the top k sorted by relevance.
type ScanReport ¶ added in v0.2.5
ScanReport says what scanning a directory did.
type SearchMode ¶
type SearchMode int
SearchMode determines the search strategy.
const ( // Keyword uses BM25 term matching. Keyword SearchMode = 0 // Semantic uses vector similarity. Semantic SearchMode = 1 // Hybrid combines BM25 and vector similarity. Hybrid SearchMode = 2 )
type SearchOptions ¶ added in v0.2.5
type SearchOptions struct {
// TopK is the most results to return; 0 means 10.
TopK int
// NoRerank skips the reranker for this query when the Searcher has one.
NoRerank bool
// Threshold drops results scoring below it.
Threshold float32
// SourcePattern keeps only chunks from files matching this `*` pattern, e.g. "*.md".
SourcePattern string
// FilterKey and FilterValue keep only chunks whose metadata has that pair.
FilterKey string
FilterValue string
}
SearchOptions overrides a Searcher's defaults for one query. The zero value changes nothing.
type SearchResult ¶
type SearchResult struct {
Score float32
Text string
DocumentID int
// Metadata stored with the chunk, such as "source", the file it came from.
Metadata map[string]any
}
SearchResult holds a single search result with its relevance score.
type Searcher ¶
type Searcher struct {
// contains filtered or unexported fields
}
Searcher queries indexes created by an Indexer.
func NewSearcher ¶
NewSearcher creates a searcher using the given embedding model. Pass a non-empty rerankerModel to enable cross-encoder reranking of results. Pass an empty string to disable reranking.
func (*Searcher) Search ¶
func (s *Searcher) Search(indexPath string, query string, mode SearchMode) ([]SearchResult, error)
Search queries the index at indexPath and returns up to 10 results using the given mode, reranked when the Searcher has a reranker.
func (*Searcher) SearchWithOptions ¶ added in v0.2.5
func (s *Searcher) SearchWithOptions(indexPath string, query string, mode SearchMode, opts SearchOptions) ([]SearchResult, error)
SearchWithOptions is Search with per-query overrides.
type Segment ¶ added in v0.2.5
Segment is one timed piece of a transcript, in seconds from the start.
type Summarizer ¶ added in v0.2.5
type Summarizer struct {
// contains filtered or unexported fields
}
Summarizer summarizes article-length text.
func NewSummarizer ¶ added in v0.2.5
func NewSummarizer(model string, opts ...Option) (*Summarizer, error)
NewSummarizer loads a summarization model, downloading it on first use. Pass "" for distilbart-cnn. See WithSummaryLength.
func (*Summarizer) Close ¶ added in v0.2.5
func (s *Summarizer) Close() error
Close releases the model. Safe to call more than once.
type Transcriber ¶ added in v0.2.5
type Transcriber struct {
// contains filtered or unexported fields
}
Transcriber turns speech into text with Whisper.
func NewTranscriber ¶ added in v0.2.5
func NewTranscriber(model string, opts ...Option) (*Transcriber, error)
NewTranscriber loads a Whisper model, downloading it on first use. Pass "" for whisper-small. See WithLanguage and WithTranslate.
func (*Transcriber) Close ¶ added in v0.2.5
func (t *Transcriber) Close() error
Close releases the model. Safe to call more than once.
func (*Transcriber) Transcribe ¶ added in v0.2.5
func (t *Transcriber) Transcribe(samples []float32, sampleRate int) (*Transcript, error)
Transcribe transcribes mono samples in -1..1 at any sample rate.
func (*Transcriber) TranscribeFile ¶ added in v0.2.5
func (t *Transcriber) TranscribeFile(path string) (*Transcript, error)
TranscribeFile transcribes a wav, mp3, flac or ogg file.
type Transcript ¶ added in v0.2.5
type Transcript struct {
Text string
// Language is the ISO code transcribed: the one given, or the one detected.
Language string
DurationSecs float32
Segments []Segment
}
Transcript is the result of transcribing audio.
type Translator ¶ added in v0.2.5
type Translator struct {
// contains filtered or unexported fields
}
Translator translates text between languages.
func NewTranslator ¶ added in v0.2.5
func NewTranslator(model string, opts ...Option) (*Translator, error)
NewTranslator loads a translation model, downloading it on first use. Pass "" for flan-t5-base. WithLanguages sets the languages Translate uses by default.
func (*Translator) Close ¶ added in v0.2.5
func (t *Translator) Close() error
Close releases the model. Safe to call more than once.
func (*Translator) Translate ¶ added in v0.2.5
func (t *Translator) Translate(text string) (string, error)
Translate translates text between the default languages given to NewTranslator.
func (*Translator) TranslateBetween ¶ added in v0.2.5
func (t *Translator) TranslateBetween(text, from, to string) (string, error)
TranslateBetween translates text from one language to another, e.g. "en" to "de" or "english" to "german". An empty language falls back to the default; with neither, the error has code ErrInvalidConfig.
