semanticrouter

package module
v0.9.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2024 License: MIT Imports: 7 Imported by: 5

README

semanticrouter-go

go.dev Build Status Go Report Card

Go Semantic Router is a superfast decision-making layer for your LLMs and agents written in pure Go .

Rather than waiting for slow LLM generations to make tool-use decisions, use the magic of semantic vector space to make those decisions — routing requests using configurable semantic meaning.

A pure-go package for abstractly computing similarity scores between a query vector embedding and a set of vector embeddings.

Installation

go get github.com/conneroisu/semanticrouter-go
Conversational Agents Example
// Package main shows how to use the semantic router to find the best route for a given utterance
// in the context of a veterinarian appointment.
package main

import (
	"context"
	"fmt"
	"os"

	"github.com/conneroisu/semanticrouter-go"
	"github.com/conneroisu/semanticrouter-go/encoders/ollama"
	"github.com/conneroisu/semanticrouter-go/stores/memory"
	"github.com/ollama/ollama/api"
)

// NoteworthyRoutes represents a set of routes that are noteworthy.
// noteworthy here means that the routes are likely to be relevant to a noteworthy conversation in a veterinarian appointment.
var NoteworthyRoutes = semanticrouter.Route{
	Name: "noteworthy",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is the best way to treat a dog with a cold?"},
		{Utterance: "my cat has been limping, what should I do?"},
	},
}

// ChitchatRoutes represents a set of routes that are chitchat.
// chitchat here means that the routes are likely to be relevant to a chitchat conversation in a veterinarian appointment.
var ChitchatRoutes = semanticrouter.Route{
	Name: "chitchat",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is your favorite color?"},
		{Utterance: "what is your favorite animal?"},
	},
}

// main runs the example.
func main() {
	if err := run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

// run runs the example.
func run() error {
	ctx := context.Background()
	cli, err := api.ClientFromEnvironment()
	if err != nil {
		return fmt.Errorf("error creating client: %w", err)
	}
	router, err := semanticrouter.NewRouter(
		[]semanticrouter.Route{NoteworthyRoutes, ChitchatRoutes},
		&ollama.Encoder{
			Client: cli,
			Model:  "mxbai-embed-large",
		},
		memory.NewStore(),
	)
	if err != nil {
		return fmt.Errorf("error creating router: %w", err)
	}
	finding, p, err := router.Match(ctx, "how's the weather today?")
	if err != nil {
		fmt.Println("Error:", err)
	}
	fmt.Println("Found:", finding)
	fmt.Println("p:", p)
	return nil
}

Output:

Found: chitchat
Veterinarian Example

The following example shows how to use the semantic router to find the best route for a given utterance in the context of a veterinarian appointment.

The goal of the example is to decide whether spoken utterances are relevant to a noteworthy conversation or a chitchat conversation.

The Code Example:
// Package main shows how to use the semantic router to find the best route for a given utterance
// in the context of a veterinarian appointment.
package main

import (
	"context"
	"fmt"
	"os"

	"github.com/conneroisu/semanticrouter-go"
	"github.com/conneroisu/semanticrouter-go/encoders/ollama"
	"github.com/conneroisu/semanticrouter-go/stores/memory"
	"github.com/ollama/ollama/api"
)

// NoteworthyRoutes represents a set of routes that are noteworthy.
// noteworthy here means that the routes are likely to be relevant to a noteworthy conversation in a veterinarian appointment.
var NoteworthyRoutes = semanticrouter.Route{
	Name: "noteworthy",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is the best way to treat a dog with a cold?"},
		{Utterance: "my cat has been limping, what should I do?"},
	},
}

// ChitchatRoutes represents a set of routes that are chitchat.
// chitchat here means that the routes are likely to be relevant to a chitchat conversation in a veterinarian appointment.
var ChitchatRoutes = semanticrouter.Route{
	Name: "chitchat",
	Utterances: []semanticrouter.Utterance{
		{Utterance: "what is your favorite color?"},
		{Utterance: "what is your favorite animal?"},
	},
}

// main runs the example.
func main() {
	if err := run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

// run runs the example.
func run() error {
	ctx := context.Background()
	cli, err := api.ClientFromEnvironment()
	if err != nil {
		return fmt.Errorf("error creating client: %w", err)
	}
	router, err := semanticrouter.NewRouter(
		[]semanticrouter.Route{NoteworthyRoutes, ChitchatRoutes},
		&ollama.Encoder{
			Client: cli,
			Model:  "mxbai-embed-large",
		},
		memory.NewStore(),
	)
	if err != nil {
		return fmt.Errorf("error creating router: %w", err)
	}
	finding, p, err := router.Match(ctx, "how's the weather today?")
	if err != nil {
		fmt.Println("Error:", err)
	}
	fmt.Println("Found:", finding)
	fmt.Println("p:", p)
	return nil
}
The Output

The output of the veterinarian example is:

Found: chitchat

Development

Testing

To run the tests, run the following command:

make test
Making a new Store Implementation

Implement the Store interface in the stores package.

The store interface is defined as follows:

type Store interface { // size=16 (0x10)
	Storer
	Getter
	io.Closer
}

Store is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store, and a method, Get, which takes a string and returns a []float64 from the data store.

func (io.Closer) Close() error
func (Getter) Get(ctx context.Context, key string) ([]float64, error)
func (Storer) Store(ctx context.Context, keyValPair Utterance) error

semanticrouter.Store on pkg.go.dev

Code Generated Documentation

semanticrouter

import "github.com/conneroisu/semanticrouter-go"

Package semanticrouter is a package for abstractly computing similarity scores between a query vector embedding and a set of vector embeddings.

It provides a simple key-value store for embeddings and a router that can be used to find the best route for a given utterance.

The router uses a similarity score to determine the best route for a given utterance.

The semantic router is designed to be used in conjunction with LLMs and agents to provide a superfast decision-making layer.

Index

func NormalizeScores

func NormalizeScores(sim []float64) []float64

NormalizeScores normalizes the similarity scores to a 0-1 range. The function takes a slice of float64 values representing the similarity scores.

The function takes a slice of float64 values representing the similarity scores and returns a slice of float64 values representing the normalized similarity scores.

type Encoder

Encoder represents a encoding driver in the semantic router.

It is an interface that defines a single method, Encode, which takes a string and returns a []float64 representing the embedding of the string.

type Encoder interface {
    Encode(ctx context.Context, utterance string) ([]float64, error)
}

type ErrEncoding

ErrEncoding is an error that is returned when an error occurs during encoding.

type ErrEncoding struct {
    Message string
}

func (ErrEncoding) Error
func (e ErrEncoding) Error() string

Error returns the error message.

type ErrGetEmbedding

ErrGetEmbedding is an error that is returned when an error occurs during getting an embedding.

type ErrGetEmbedding struct {
    Message string
    Storage Store
}

func (ErrGetEmbedding) Error
func (e ErrGetEmbedding) Error() string

Error returns the error message.

type ErrNoRouteFound

ErrNoRouteFound is an error that is returned when no route is found.

type ErrNoRouteFound struct {
    Message   string
    Utterance string
}

func (ErrNoRouteFound) Error
func (e ErrNoRouteFound) Error() string

Error returns the error message.

type Getter

Getter is an interface that defines a method, Get, which takes a string and returns a []float64 from the data store.

If the key does not exist, it returns an error.

type Getter interface {
    Get(ctx context.Context, key string) ([]float64, error)
}

type Option

Option is a function that configures a Router.

type Option func(*Router)

func WithEuclideanDistance
func WithEuclideanDistance(coefficient float64) Option

WithEuclideanDistance sets the EuclideanDistance function with a coefficient.

$$d(x, y) = \sqrt{\sum_{i=1}^{n}(x_i - y_i)^2}$$

func WithJaccardSimilarity
func WithJaccardSimilarity(coefficient float64) Option

WithJaccardSimilarity sets the JaccardSimilarity function with a coefficient.

$$J(A, B)=\frac{|A \cap B|}{|A \cup B|}$$

It adds the jaccard similarity to the comparision functions with the given coefficient.

func WithManhattanDistance
func WithManhattanDistance(coefficient float64) Option

WithManhattanDistance sets the ManhattanDistance function with a coefficient.

$$d(x, y) = |x_1 - y_1| + |x_2 - y_2| + ... + |x_n - y_n|$$

It adds the manhatten distance to the comparision functions with the given coefficient.

func WithPearsonCorrelation
func WithPearsonCorrelation(coefficient float64) Option

WithPearsonCorrelation sets the PearsonCorrelation function with a coefficient.

$$r=\frac{\sum\left(x_{i}-\bar{x}\right)\left(y_{i}-\bar{y}\right)}{\sqrt{\sum\left(x_{i}-\bar{x}\right)^{2} \sum\left(y_{i}-\bar{y}\right)^{2}}}$$

It adds the pearson correlation to the comparision functions with the given coefficient.

func WithSimilarityDotMatrix
func WithSimilarityDotMatrix(coefficient float64) Option

WithSimilarityDotMatrix sets the similarity function to use with a coefficient.

$$a \cdot b=\sum_{i=1}^{n} a_{i} b_{i}$$

It adds the similarity dot matrix to the comparision functions with the given coefficient.

func WithWorkers
func WithWorkers(workers int) Option

WithWorkers sets the number of workers to use for computing similarity scores.

type Route

Route represents a route in the semantic router.

It is a struct that contains a name and a slice of Utterances.

type Route struct {
    Name       string      // Name is the name of the route.
    Utterances []Utterance // Utterances is a slice of Utterances.
}

type Router

Router represents a semantic router.

Router is a struct that contains a slice of Routes and an Encoder.

Match can be called on a Router to find the best route for a given utterance.

type Router struct {
    Routes  []Route // Routes is a slice of Routes.
    Encoder Encoder // Encoder is an Encoder that encodes utterances into vectors.
    Storage Store   // Storage is a Store that stores the utterances.
    // contains filtered or unexported fields
}

func NewRouter
func NewRouter(routes []Route, encoder Encoder, store Store, opts ...Option) (router *Router, err error)

NewRouter creates a new semantic router.

func (*Router) Match
func (r *Router) Match(ctx context.Context, utterance string) (bestRoute *Route, bestScore float64, err error)

Match returns the route that matches the given utterance.

The score is the similarity score between the query vector and the index vector.

If the given context is canceled, the context's error is returned if it is non-nil.

type Store

Store is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store, and a method, Get, which takes a string and returns a []float64 from the data store.

type Store interface {
    Storer
    Getter
    io.Closer
}

type Storer

Storer is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store.

type Storer interface {
    Store(ctx context.Context, keyValPair Utterance) error
}

type Utterance

Utterance represents a utterance in the semantic router.

type Utterance struct {
    // ID is the ID of the utterance.
    ID  int
    // Utterance is the text of the utterance.
    Utterance string
    // Embed is the embedding of the utterance. It is a vector of floats.
    Embed embedding
}

Generated by gomarkdoc

Documentation

Overview

Package semanticrouter is a package for abstractly computing similarity scores between a query vector embedding and a set of vector embeddings.

It provides a simple key-value store for embeddings and a router that can be used to find the best route for a given utterance.

The router uses a similarity score to determine the best route for a given utterance.

The semantic router is designed to be used in conjunction with LLMs and agents to provide a superfast decision-making layer.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NormalizeScores 

func NormalizeScores(sim []float64) []float64

NormalizeScores normalizes the similarity scores to a 0-1 range. The function takes a slice of float64 values representing the similarity scores.

The function takes a slice of float64 values representing the similarity scores and returns a slice of float64 values representing the normalized similarity scores.

Types

type Encoder 

type Encoder interface {
	Encode(ctx context.Context, utterance string) ([]float64, error)
}

Encoder represents a encoding driver in the semantic router.

It is an interface that defines a single method, Encode, which takes a string and returns a []float64 representing the embedding of the string.

type ErrEncoding 

type ErrEncoding struct {
	Message string
}

ErrEncoding is an error that is returned when an error occurs during encoding.

func (ErrEncoding) Error 

func (e ErrEncoding) Error() string

Error returns the error message.

type ErrGetEmbedding 

type ErrGetEmbedding struct {
	Message string
	Storage Store
}

ErrGetEmbedding is an error that is returned when an error occurs during getting an embedding.

func (ErrGetEmbedding) Error 

func (e ErrGetEmbedding) Error() string

Error returns the error message.

type ErrNoRouteFound 

type ErrNoRouteFound struct {
	Message   string
	Utterance string
}

ErrNoRouteFound is an error that is returned when no route is found.

func (ErrNoRouteFound) Error 

func (e ErrNoRouteFound) Error() string

Error returns the error message.

type Getter 

type Getter interface {
	Get(ctx context.Context, key string) ([]float64, error)
}

Getter is an interface that defines a method, Get, which takes a string and returns a []float64 from the data store.

If the key does not exist, it returns an error.

type Option 

type Option func(*Router)

Option is a function that configures a Router.

func WithEuclideanDistance 

func WithEuclideanDistance(coefficient float64) Option

WithEuclideanDistance sets the EuclideanDistance function with a coefficient.

$$d(x, y) = \sqrt{\sum_{i=1}^{n}(x_i - y_i)^2}$$

func WithJaccardSimilarity 

func WithJaccardSimilarity(coefficient float64) Option

WithJaccardSimilarity sets the JaccardSimilarity function with a coefficient.

$$J(A, B)=\frac{|A \cap B|}{|A \cup B|}$$

It adds the jaccard similarity to the comparision functions with the given coefficient.

func WithManhattanDistance 

func WithManhattanDistance(coefficient float64) Option

WithManhattanDistance sets the ManhattanDistance function with a coefficient.

$$d(x, y) = |x_1 - y_1| + |x_2 - y_2| + ... + |x_n - y_n|$$

It adds the manhatten distance to the comparision functions with the given coefficient.

func WithPearsonCorrelation 

func WithPearsonCorrelation(coefficient float64) Option

WithPearsonCorrelation sets the PearsonCorrelation function with a coefficient.

$$r=\frac{\sum\left(x_{i}-\bar{x}\right)\left(y_{i}-\bar{y}\right)}{\sqrt{\sum\left(x_{i}-\bar{x}\right)^{2} \sum\left(y_{i}-\bar{y}\right)^{2}}}$$

It adds the pearson correlation to the comparision functions with the given coefficient.

func WithSimilarityDotMatrix 

func WithSimilarityDotMatrix(coefficient float64) Option

WithSimilarityDotMatrix sets the similarity function to use with a coefficient.

$$a \cdot b=\sum_{i=1}^{n} a_{i} b_{i}$$

It adds the similarity dot matrix to the comparision functions with the given coefficient.

func WithWorkers 

func WithWorkers(workers int) Option

WithWorkers sets the number of workers to use for computing similarity scores.

type Route 

type Route struct {
	Name       string      // Name is the name of the route.
	Utterances []Utterance // Utterances is a slice of Utterances.
}

Route represents a route in the semantic router.

It is a struct that contains a name and a slice of Utterances.

type Router 

type Router struct {
	Routes  []Route // Routes is a slice of Routes.
	Encoder Encoder // Encoder is an Encoder that encodes utterances into vectors.
	Storage Store   // Storage is a Store that stores the utterances.
	// contains filtered or unexported fields
}

Router represents a semantic router.

Router is a struct that contains a slice of Routes and an Encoder.

Match can be called on a Router to find the best route for a given utterance.

func NewRouter 

func NewRouter(
	routes []Route,
	encoder Encoder,
	store Store,
	opts ...Option,
) (router *Router, err error)

NewRouter creates a new semantic router.

func (*Router) Match 

func (r *Router) Match(
	ctx context.Context,
	utterance string,
) (bestRoute *Route, bestScore float64, err error)

Match returns the route that matches the given utterance.

The score is the similarity score between the query vector and the index vector.

If the given context is canceled, the context's error is returned if it is non-nil.

type Setter 

type Setter interface {
	Set(ctx context.Context, keyValPair Utterance) error
}

Setter is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store.

type Store 

type Store interface {
	Setter
	Getter
	io.Closer
}

Store is an interface that defines a method, Store, which takes a []float64 and stores it in a some sort of data store, and a method, Get, which takes a string and returns a []float64 from the data store.

type Utterance 

type Utterance struct {
	// ID is the ID of the utterance.
	ID int
	// Utterance is the text of the utterance.
	Utterance string
	// Embed is the embedding of the utterance. It is a vector of floats.
	Embed embedding
}

Utterance represents a utterance in the semantic router.

Source Files

View all Source files

Directories

Path Synopsis
encoders
closedai Module
google Module
ollama Module
voyageai Module
examples
chit-chat
Package main shows how to use the semantic router to find the best route for a given utterance in the context of a chat bot or other conversational application.
 Package main shows how to use the semantic router to find the best route for a given utterance in the context of a chat bot or other conversational application.
veterinarian Module
stores
memory Module
mongo Module
valkey Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.