externalfunctions

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2024 License: MIT Imports: 18 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ExternalFunctionsMap = map[string]interface{}{
	"PerformVectorEmbeddingRequest":       PerformVectorEmbeddingRequest,
	"PerformKeywordExtractionRequest":     PerformKeywordExtractionRequest,
	"PerformGeneralRequest":               PerformGeneralRequest,
	"PerformCodeLLMRequest":               PerformCodeLLMRequest,
	"BuildLibraryContext":                 BuildLibraryContext,
	"SendVectorsToKnowledgeDB":            SendVectorsToKnowledgeDB,
	"GetListCollections":                  GetListCollections,
	"RetrieveDependencies":                RetrieveDependencies,
	"GeneralNeo4jQuery":                   GeneralNeo4jQuery,
	"GeneralQuery":                        GeneralQuery,
	"BuildFinalQueryForGeneralLLMRequest": BuildFinalQueryForGeneralLLMRequest,
	"BuildFinalQueryForCodeLLMRequest":    BuildFinalQueryForCodeLLMRequest,
	"SimilaritySearch":                    SimilaritySearch,
	"CreateKeywordsDbFilter":              CreateKeywordsDbFilter,
	"CreateTagsDbFilter":                  CreateTagsDbFilter,
	"CreateMetadataDbFilter":              CreateMetadataDbFilter,
	"CreateDbFilter":                      CreateDbFilter,
	"AppendMessageHistory":                AppendMessageHistory,
}

Functions

func BuildFinalQueryForCodeLLMRequest 

func BuildFinalQueryForCodeLLMRequest(request string, knowledgedbResponse []DbResponse) (finalQuery string)

BuildFinalQueryForCodeLLMRequest builds the final query for a code generation request to LLM. The final query is a markdown string that contains the original request and the code examples from the KnowledgeDB.

Parameters:

  • request: the original request
  • knowledgedbResponse: the KnowledgeDB response

Returns:

  • finalQuery: the final query

func BuildFinalQueryForGeneralLLMRequest added in v0.2.0 

func BuildFinalQueryForGeneralLLMRequest(request string, knowledgedbResponse []DbResponse) (finalQuery string)

BuildFinalQueryForGeneralLLMRequest builds the final query for a general request to LLM. The final query is a markdown string that contains the original request and the examples from the KnowledgeDB.

Parameters:

  • request: the original request
  • knowledgedbResponse: the KnowledgeDB response

Returns:

  • finalQuery: the final query

func BuildLibraryContext 

func BuildLibraryContext(message string, libraryContext string) (messageWithContext string)

BuildLibraryContext builds the context string for the query

Parameters:

  • message: the message string
  • libraryContext: the library context string

Returns:

  • messageWithContext: the message with context

func GeneralNeo4jQuery 

func GeneralNeo4jQuery(query string) (databaseResponse neo4jResponse)

GeneralNeo4jQuery executes the given Neo4j query and returns the response.

The function returns the neo4j response.

Parameters:

  • query: the Neo4j query to be executed.

Returns:

  • databaseResponse: the Neo4j response

func GetListCollections 

func GetListCollections() (collectionsList []string)

GetListCollections retrieves the list of collections from the KnowledgeDB.

The function returns the list of collections.

Parameters:

  • knowledgeDbEndpoint: the KnowledgeDB endpoint

Returns:

  • collectionsList: the list of collections

func PerformCodeLLMRequest 

func PerformCodeLLMRequest(input string, history []HistoricMessage, isStream bool, validateCode bool) (message string, stream *chan string)

PerformCodeLLMRequest performs a code generation request to LLM

Parameters:

  • input: the input string
  • history: the conversation history
  • isStream: the stream flag

Returns:

  • message: the generated code
  • stream: the stream channel

func PerformGeneralRequest 

func PerformGeneralRequest(input string, history []HistoricMessage, isStream bool, systemPrompt string) (message string, stream *chan string)

PerformGeneralRequest performs a general chat completion request to LLM

Parameters:

  • input: the input string
  • history: the conversation history
  • isStream: the stream flag
  • systemPrompt: the system prompt

Returns:

  • message: the generated message
  • stream: the stream channel

func PerformKeywordExtractionRequest 

func PerformKeywordExtractionRequest(input string, maxKeywordsSearch uint32) (keywords []string)

PerformSummaryRequest performs a keywords summary request to LLM

Parameters:

  • input: the input string
  • maxKeywordsSearch: the maximum number of keywords to search for

Returns:

  • keywords: the keywords extracted from the input string as a slice of strings

func PerformVectorEmbeddingRequest 

func PerformVectorEmbeddingRequest(input string) (embeddedVector []float32)

PerformVectorEmbeddingRequest performs a vector embedding request to LLM

Parameters:

  • input: the input string

Returns:

  • embeddedVector: the embedded vector in float32 format

func RetrieveDependencies 

func RetrieveDependencies(
	collectionName string,
	relationshipName string,
	relationshipDirection string,
	sourceDocumentId string,
	nodeTypesFilter DbArrayFilter,
	maxHopsNumber int) (dependenciesIds []string)

RetrieveDependencies retrieves the dependencies of the specified source node.

The function returns the list of dependencies.

Parameters:

  • collectionName: the name of the collection to which the data objects will be added.
  • relationshipName: the name of the relationship to retrieve dependencies for.
  • relationshipDirection: the direction of the relationship to retrieve dependencies for.
  • sourceDocumentId: the document ID of the source node.
  • nodeTypesFilter: filter based on node types.
  • maxHopsNumber: maximum number of hops to traverse.

Returns:

  • dependenciesIds: the list of dependencies

Types

type AppendMessageHistoryRole 

type AppendMessageHistoryRole string

AppendMessageHistoryInput represents the input for the AppendMessageHistory function.

type DBListCollectionsOutput 

type DBListCollectionsOutput struct {
	Success     bool     `json:"success" description:"Returns true if the collections were listed successfully. Returns false or an error if not."`
	Collections []string `json:"collections" description:"A list of collection names."`
}

DBListCollectionsOutput represents the output for the listCollections function.

type DbArrayFilter 

type DbArrayFilter struct {
	NeedAll    bool     `json:"needAll"`
	FilterData []string `json:"filterData"`
}

DbArrayFilter is used to filter array fields in the database.

func CreateKeywordsDbFilter 

func CreateKeywordsDbFilter(keywords []string, needAll bool) (databaseFilter DbArrayFilter)

CreateKeywordsDbFilter creates a keywords filter for the KnowledgeDB.

The function returns the keywords filter.

Parameters:

  • keywords: the keywords to be used for the filter
  • needAll: flag to indicate whether all keywords are needed

Returns:

  • databaseFilter: the keywords filter

func CreateTagsDbFilter 

func CreateTagsDbFilter(tags []string, needAll bool) (databaseFilter DbArrayFilter)

CreateTagsDbFilter creates a tags filter for the KnowledgeDB.

The function returns the tags filter.

Parameters:

  • tags: the tags to be used for the filter
  • needAll: flag to indicate whether all tags are needed

Returns:

  • databaseFilter: the tags filter

type DbData 

type DbData struct {
	Guid              string                 `json:"guid"`
	DocumentId        string                 `json:"document_id"`
	DocumentName      string                 `json:"document_name"`
	Text              string                 `json:"text"`
	Keywords          []string               `json:"keywords"`
	Summary           string                 `json:"summary"`
	Embedding         []float32              `json:"embeddings"`
	Tags              []string               `json:"tags"`
	Metadata          map[string]interface{} `json:"metadata"`
	ParentId          string                 `json:"parent_id"`
	ChildIds          []string               `json:"child_ids"`
	PreviousSiblingId string                 `json:"previous_sibling_id"`
	NextSiblingId     string                 `json:"next_sibling_id"`
	LastChildId       string                 `json:"last_child_id"`
	FirstChildId      string                 `json:"first_child_id"`
	Level             string                 `json:"level"`
	HasNeo4jEntry     bool                   `json:"has_neo4j_entry"`
}

DbData represents the data structure for the database.

type DbFilters 

type DbFilters struct {
	// Filters for string fields
	GuidFilter         []string `json:"guid,omitempty"`
	DocumentIdFilter   []string `json:"document_id,omitempty"`
	DocumentNameFilter []string `json:"document_name,omitempty"`
	LevelFilter        []string `json:"level,omitempty"`

	// Filters for array fields
	TagsFilter     DbArrayFilter `json:"tags,omitempty"`
	KeywordsFilter DbArrayFilter `json:"keywords,omitempty"`

	// Filters for JSON fields
	MetadataFilter []DbJsonFilter `json:"metadata,omitempty"`
}

func CreateDbFilter 

func CreateDbFilter(
	guid []string,
	documentId []string,
	documentName []string,
	level []string,
	tags DbArrayFilter,
	keywords DbArrayFilter,
	metadata []DbJsonFilter) (databaseFilter DbFilters)

CreateDbFilter creates a filter for the KnowledgeDB.

The function returns the filter.

Parameters:

  • guid: the guid filter
  • documentId: the document ID filter
  • documentName: the document name filter
  • level: the level filter
  • tags: the tags filter
  • keywords: the keywords filter
  • metadata: the metadata filter

Returns:

  • databaseFilter: the filter

type DbJsonFilter 

type DbJsonFilter struct {
	FieldName  string   `json:"fieldName"`
	FieldType  string   `json:"fieldType" description:"Can be either string or array."` // "string" or "array"
	FilterData []string `json:"filterData"`
	NeedAll    bool     `json:"needAll" description:"Only needed if the FieldType is array."` // only needed for array fields
}

DbJsonFilter is used to filter JSON fields in the database.

func CreateMetadataDbFilter 

func CreateMetadataDbFilter(fieldName string, fieldType string, filterData []string, needAll bool) (databaseFilter DbJsonFilter)

CreateMetadataDbFilter creates a metadata filter for the KnowledgeDB.

The function returns the metadata filter.

Parameters:

  • fieldName: the name of the field
  • fieldType: the type of the field
  • filterData: the filter data
  • needAll: flag to indicate whether all data is needed

Returns:

  • databaseFilter: the metadata filter

type DbResponse 

type DbResponse struct {
	Guid              string                 `json:"guid"`
	DocumentId        string                 `json:"document_id"`
	DocumentName      string                 `json:"document_name"`
	Text              string                 `json:"text"`
	Keywords          []string               `json:"keywords"`
	Summary           string                 `json:"summary"`
	Embedding         []float32              `json:"embeddings"`
	Tags              []string               `json:"tags"`
	Metadata          map[string]interface{} `json:"metadata"`
	ParentId          string                 `json:"parent_id"`
	ChildIds          []string               `json:"child_ids"`
	PreviousSiblingId string                 `json:"previous_sibling_id"`
	NextSiblingId     string                 `json:"next_sibling_id"`
	LastChildId       string                 `json:"last_child_id"`
	FirstChildId      string                 `json:"first_child_id"`
	Distance          float64                `json:"distance"`
	Level             string                 `json:"level"`
	HasNeo4jEntry     bool                   `json:"has_neo4j_entry"`

	// Siblings
	Parent    *DbData  `json:"parent,omitempty"`
	Children  []DbData `json:"children,omitempty"`
	LeafNodes []DbData `json:"leaf_nodes,omitempty"`
	Siblings  []DbData `json:"siblings,omitempty"`
}

DbResponse represents the response structure for the database.

func GeneralQuery 

func GeneralQuery(collectionName string, maxRetrievalCount int, outputFields []string, filters DbFilters) (databaseResponse []DbResponse)

GeneralQuery performs a general query in the KnowledgeDB.

The function returns the query results.

Parameters:

  • collectionName: the name of the collection to which the data objects will be added.
  • maxRetrievalCount: the maximum number of results to be retrieved.
  • outputFields: the fields to be included in the output.
  • filters: the filter for the query.

Returns:

  • databaseResponse: the query results

func SendVectorsToKnowledgeDB 

func SendVectorsToKnowledgeDB(vector []float32, keywords []string, keywordsSearch bool, collection string, similaritySearchResults int, similaritySearchMinScore float64) (databaseResponse []DbResponse)

SendVectorsToKnowledgeDB sends the given vector to the KnowledgeDB and returns the most relevant data. The number of results is specified in the config file. The keywords are used to filter the results. The min score filter is also specified in the config file. If it is not specified, the default value is used.

The function returns the most relevant data.

Parameters:

  • vector: the vector to be sent to the KnowledgeDB
  • keywords: the keywords to be used to filter the results
  • keywordsSearch: the flag to enable the keywords search
  • collection: the collection name
  • similaritySearchResults: the number of results to be returned
  • similaritySearchMinScore: the minimum score for the results

Returns:

  • databaseResponse: an array of the most relevant data

func SimilaritySearch 

func SimilaritySearch(
	collectionName string,
	embeddedVector []float32,
	maxRetrievalCount int,
	outputFields []string,
	filters DbFilters,
	minScore float64,
	getLeafNodes bool,
	getSiblings bool,
	getParent bool,
	getChildren bool) (databaseResponse []DbResponse)

SimilaritySearch performs a similarity search in the KnowledgeDB.

The function returns the similarity search results.

Parameters:

  • collectionName: the name of the collection to which the data objects will be added.
  • embeddedVector: the embedded vector used for searching.
  • maxRetrievalCount: the maximum number of results to be retrieved.
  • outputFields: the fields to be included in the output.
  • filters: the filter for the query.
  • minScore: the minimum score filter.
  • getLeafNodes: flag to indicate whether to retrieve all the leaf nodes in the result node branch.
  • getSiblings: flag to indicate whether to retrieve the previous and next node to the result nodes.
  • getParent: flag to indicate whether to retrieve the parent object.
  • getChildren: flag to indicate whether to retrieve the children objects.

Returns:

  • databaseResponse: the similarity search results

type ErrorResponse 

type ErrorResponse struct {
	Code    int    `json:"code"`
	Message string `json:"message"`
}

ErrorResponse represents the error response sent to the client when something fails during the processing of the request.

type GeneralNeo4jQueryInput 

type GeneralNeo4jQueryInput struct {
	Query string `json:"query" description:"Neo4j query to be executed. Required for executing a query." required:"true"`
}

GeneralNeo4jQueryInput represents the input for the generalNeo4jQuery function.

type GeneralNeo4jQueryOutput 

type GeneralNeo4jQueryOutput struct {
	Success  bool          `json:"success" description:"Returns true if the query was executed successfully. Returns false or an error if not."`
	Response neo4jResponse `json:"response" description:"Summary and records of the query execution."`
}

GeneralNeo4jQueryOutput represents the output for the generalNeo4jQuery function.

type HandlerRequest 

type HandlerRequest struct {
	Adapter             string            `json:"adapter"` // "chat", "embeddings"
	InstructionGuid     string            `json:"instructionGuid"`
	Data                string            `json:"data"`
	ChatRequestType     string            `json:"chatRequestType"`        // "summary", "code", "keywords", "general"; only relevant if "adapter" is "chat"
	DataStream          bool              `json:"dataStream"`             // only relevant if "adapter" is "chat"
	MaxNumberOfKeywords uint32            `json:"maxNumberOfKeywords"`    // only relevant if "chatRequestType" is "keywords"
	IsConversation      bool              `json:"isConversation"`         // only relevant if "chatRequestType" is "code"
	ConversationHistory []HistoricMessage `json:"conversationHistory"`    // only relevant if "isConversation" is true
	GeneralContext      string            `json:"generalContext"`         // any added context you might need
	MsgContext          string            `json:"msgContext"`             // any added context you might need
	SystemPrompt        string            `json:"systemPrompt"`           // only relevant if "chatRequestType" is "general"
	ModelOptions        ModelOptions      `json:"modelOptions,omitempty"` // only relevant if "adapter" is "chat"
	ClientGuid          string
}

HandlerRequest represents the client request for a specific chat or embeddings operation.

type HandlerResponse 

type HandlerResponse struct {
	// Common properties
	InstructionGuid string `json:"instructionGuid"`
	Type            string `json:"type"` // "info", "error", "chat", "embeddings"
	// Chat properties
	IsLast   *bool   `json:"isLast,omitempty"`
	Position *uint32 `json:"position,omitempty"`
	ChatData *string `json:"chatData,omitempty"`
	// Embeddings properties
	EmbeddedData []float32 `json:"embeddedData,omitempty"`
	// Error properties
	Error *ErrorResponse `json:"error,omitempty"`
	// Info properties
	InfoMessage *string `json:"infoMessage,omitempty"`
}

HandlerResponse represents the LLM Handler response for a specific request.

type HistoricMessage 

type HistoricMessage struct {
	Role    string `json:"role"` // "user", "assistant", "system"
	Content string `json:"content"`
}

HistoricMessage represents a past chat messages.

func AppendMessageHistory 

func AppendMessageHistory(newMessage string, role AppendMessageHistoryRole, history []HistoricMessage) (updatedHistory []HistoricMessage)

AppendMessageHistory appends a new message to the conversation history

Parameters:

  • newMessage: the new message
  • role: the role of the message
  • history: the conversation history

Returns:

  • updatedHistory: the updated conversation history

type ModelOptions 

type ModelOptions struct {
	FrequencyPenalty *float32 `json:"frequencyPenalty,omitempty"`
	MaxTokens        *int32   `json:"maxTokens,omitempty"`
	PresencePenalty  *float32 `json:"presencePenalty,omitempty"`
	Stop             []string `json:"stop,omitempty"`
	Temperature      *float32 `json:"temperature,omitempty"`
	TopP             *float32 `json:"topP,omitempty"`
}

OpenAIOption represents an option for an OpenAI API call.

type TransferDetails 

type TransferDetails struct {
	ResponseChannel chan HandlerResponse
	RequestChannel  chan HandlerRequest
}

TransferDetails holds communication channels for the websocket listener and writer.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.