semantic_search_go

README

semantic-search-go

Text similarity search for Go

2x faster than sentence-transformers on 500×10000

Installation

go get github.com/acheong08/semantic-search-go

Example usage

Simple:

package main

import (
	semantic_search "github.com/acheong08/semantic-search-go"
)

func main() {
	corpus := []string{"Google Chrome", "Firefox", "Dumpster Fire", "Garbage", "Brave"}
	results, _ := semantic_search.SemanticSearch([]string{"I need a web browser"}, corpus, 2)
	for _, result := range results[0] {
		println(corpus[result.CorpusID])
		println(result.Score)
	}
}

Raw:

package main

import (
	"github.com/acheong08/semantic-search-go/rank"
	"github.com/acheong08/semantic-search-go/typings"
	"github.com/acheong08/semantic-search-go/vectors"
)

func main() {
	corpus := []string{
		"Google Chrome",
		"Firefox",
		"Dumpster Fire",
		"Garbage",
		"Brave",
	}
	// Encode the corpus
	encodedCorpus, err := vectors.EncodeMulti(corpus)
	if err != nil {
		panic(err)
	}
	query := "What is a good web browser?"
	encodedQuery, err := vectors.Encode(query)
	if err != nil {
		panic(err)
	}
	// Convert query from []float64 to [][]float64 (tensor)
	queryTensor := [][]float64{encodedQuery}
	// Semantic search
	searchResult := rank.Rank(queryTensor, encodedCorpus, 2, false)
	// Print results
	for _, result := range searchResult[0] {
		println(corpus[result.CorpusID])
		println(result.Score)
	}
}

You can store your vectors from semantic-search-go/vectors in a database and run semantic-search-go/rank as needed rather than encoding every single time.

Documentation

Index

• func SemanticSearch(query []string, corpus []string, results int, sorted bool) ([][]typings.SearchResult, error)

Functions

func SemanticSearch

func SemanticSearch(query []string, corpus []string, results int, sorted bool) ([][]typings.SearchResult, error)