Documentation
¶
Overview ¶
Package llms provides unified support for interacting with different Language Models (LLMs) from various providers. Designed with an extensible architecture, the package facilitates seamless integration of LLMs with a focus on modularity, encapsulation, and easy configurability.
The package includes the following subpackages for LLM providers: 1. Hugging Face: llms/huggingface/ 2. Local LLM: llms/local/ 3. OpenAI: llms/openai/ 4. Google AI: llms/googleai/ 5. Cohere: llms/cohere/
Each subpackage includes provider-specific LLM implementations and helper files for communication with supported LLM providers. The internal directories within these subpackages contain provider-specific client and API implementations.
The `llms.go` file contains the types and interfaces for interacting with different LLMs.
The `options.go` file provides various options and functions to configure the LLMs.
Index ¶
- func CalculateMaxTokens(model, text string) int
- func CountTokens(model, text string) int
- func GenerateFromSinglePrompt(ctx context.Context, llm Model, prompt string, options ...CallOption) (string, error)
- func GetModelContextSize(model string) int
- type BinaryContent
- type CallOption
- func WithCandidateCount(c int) CallOption
- func WithFrequencyPenalty(frequencyPenalty float64) CallOption
- func WithFunctionCallBehavior(behavior FunctionCallBehavior) CallOption
- func WithFunctions(functions []FunctionDefinition) CallOption
- func WithMaxLength(maxLength int) CallOption
- func WithMaxTokens(maxTokens int) CallOption
- func WithMinLength(minLength int) CallOption
- func WithModel(model string) CallOption
- func WithN(n int) CallOption
- func WithOptions(options CallOptions) CallOption
- func WithPresencePenalty(presencePenalty float64) CallOption
- func WithRepetitionPenalty(repetitionPenalty float64) CallOption
- func WithSeed(seed int) CallOption
- func WithStopWords(stopWords []string) CallOption
- func WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) CallOption
- func WithTemperature(temperature float64) CallOption
- func WithTopK(topK int) CallOption
- func WithTopP(topP float64) CallOption
- type CallOptions
- type ContentChoice
- type ContentPart
- type ContentResponse
- type FunctionCallBehavior
- type FunctionDefinition
- type ImageURLContent
- type LLMdeprecated
- type MessageContent
- type Model
- type TextContent
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CalculateMaxTokens ¶
CalculateMaxTokens calculates the max number of tokens that could be added to a text.
func CountTokens ¶
CountTokens gets the number of tokens the text contains.
func GenerateFromSinglePrompt ¶
func GenerateFromSinglePrompt(ctx context.Context, llm Model, prompt string, options ...CallOption) (string, error)
GenerateFromSinglePrompt is a convenience function for calling an LLM with a single string prompt, expecting a single string response. It's useful for simple, string-only interactions and provides a slightly more ergonomic API than the more general llms.Model.GenerateContent.
func GetModelContextSize ¶
ModelContextSize gets the max number of tokens for a language model. If the model name isn't recognized the default value 2048 is returned.
Types ¶
type BinaryContent ¶
BinaryContent is content holding some binary data with a MIME type.
func BinaryPart ¶
func BinaryPart(mime string, data []byte) BinaryContent
BinaryPart creates a new BinaryContent from the given MIME type (e.g. "image/png" and binary data).
type CallOption ¶
type CallOption func(*CallOptions)
CallOption is a function that configures a CallOptions.
func WithCandidateCount ¶
func WithCandidateCount(c int) CallOption
WithCandidateCount specifies the number of response candidates to generate.
func WithFrequencyPenalty ¶
func WithFrequencyPenalty(frequencyPenalty float64) CallOption
WithFrequencyPenalty will add an option to set the frequency penalty for sampling.
func WithFunctionCallBehavior ¶
func WithFunctionCallBehavior(behavior FunctionCallBehavior) CallOption
WithFunctionCallBehavior will add an option to set the behavior to use when calling functions.
func WithFunctions ¶
func WithFunctions(functions []FunctionDefinition) CallOption
WithFunctions will add an option to set the functions to include in the request.
func WithMaxLength ¶
func WithMaxLength(maxLength int) CallOption
WithMaxLength will add an option to set the maximum length of the generated text.
func WithMaxTokens ¶
func WithMaxTokens(maxTokens int) CallOption
WithMaxTokens specifies the max number of tokens to generate.
func WithMinLength ¶
func WithMinLength(minLength int) CallOption
WithMinLength will add an option to set the minimum length of the generated text.
func WithModel ¶
func WithModel(model string) CallOption
WithModel specifies which model name to use.
func WithN ¶
func WithN(n int) CallOption
WithN will add an option to set how many chat completion choices to generate for each input message.
func WithPresencePenalty ¶
func WithPresencePenalty(presencePenalty float64) CallOption
WithPresencePenalty will add an option to set the presence penalty for sampling.
func WithRepetitionPenalty ¶
func WithRepetitionPenalty(repetitionPenalty float64) CallOption
WithRepetitionPenalty will add an option to set the repetition penalty for sampling.
func WithSeed ¶
func WithSeed(seed int) CallOption
WithSeed will add an option to use deterministic sampling.
func WithStopWords ¶
func WithStopWords(stopWords []string) CallOption
WithStopWords specifies a list of words to stop generation on.
func WithStreamingFunc ¶
func WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) CallOption
WithStreamingFunc specifies the streaming function to use.
func WithTemperature ¶
func WithTemperature(temperature float64) CallOption
WithTemperature specifies the model temperature, a hyperparameter that regulates the randomness, or creativity, of the AI's responses.
func WithTopK ¶
func WithTopK(topK int) CallOption
WithTopK will add an option to use top-k sampling.
func WithTopP ¶
func WithTopP(topP float64) CallOption
WithTopP will add an option to use top-p sampling.
type CallOptions ¶
type CallOptions struct {
// Model is the model to use.
Model string `json:"model"`
// CandidateCount is the number of response candidates to generate.
CandidateCount int `json:"candidate_count"`
// MaxTokens is the maximum number of tokens to generate.
MaxTokens int `json:"max_tokens"`
// Temperature is the temperature for sampling, between 0 and 1.
Temperature float64 `json:"temperature"`
// StopWords is a list of words to stop on.
StopWords []string `json:"stop_words"`
// StreamingFunc is a function to be called for each chunk of a streaming response.
// Return an error to stop streaming early.
StreamingFunc func(ctx context.Context, chunk []byte) error
// TopK is the number of tokens to consider for top-k sampling.
TopK int `json:"top_k"`
// TopP is the cumulative probability for top-p sampling.
TopP float64 `json:"top_p"`
// Seed is a seed for deterministic sampling.
Seed int `json:"seed"`
// MinLength is the minimum length of the generated text.
MinLength int `json:"min_length"`
// MaxLength is the maximum length of the generated text.
MaxLength int `json:"max_length"`
// N is how many chat completion choices to generate for each input message.
N int `json:"n"`
// RepetitionPenalty is the repetition penalty for sampling.
RepetitionPenalty float64 `json:"repetition_penalty"`
// FrequencyPenalty is the frequency penalty for sampling.
FrequencyPenalty float64 `json:"frequency_penalty"`
// PresencePenalty is the presence penalty for sampling.
PresencePenalty float64 `json:"presence_penalty"`
// Function defitions to include in the request.
Functions []FunctionDefinition `json:"functions"`
// FunctionCallBehavior is the behavior to use when calling functions.
//
// If a specific function should be invoked, use the format:
// `{"name": "my_function"}`
FunctionCallBehavior FunctionCallBehavior `json:"function_call"`
}
CallOptions is a set of options for calling models. Not all models support all options.
type ContentChoice ¶
type ContentChoice struct {
// Content is the textual content of a response
Content string
// StopReason is the reason the model stopped generating output.
StopReason string
// GenerationInfo is arbitrary information the model adds to the response.
GenerationInfo map[string]any
// FuncCall is non-nil when the model asks to invoke a function/tool.
FuncCall *schema.FunctionCall
}
ContentChoice is one of the response choices returned by GenerateContent calls.
type ContentPart ¶
type ContentPart interface {
// contains filtered or unexported methods
}
ContentPart is an interface all parts of content have to implement.
type ContentResponse ¶
type ContentResponse struct {
Choices []*ContentChoice
}
ContentResponse is the response returned by a GenerateContent call. It can potentially return multiple content choices.
type FunctionCallBehavior ¶
type FunctionCallBehavior string
FunctionCallBehavior is the behavior to use when calling functions.
const ( // FunctionCallBehaviorNone will not call any functions. FunctionCallBehaviorNone FunctionCallBehavior = "none" // FunctionCallBehaviorAuto will call functions automatically. FunctionCallBehaviorAuto FunctionCallBehavior = "auto" )
type FunctionDefinition ¶
type FunctionDefinition struct {
// Name is the name of the function.
Name string `json:"name"`
// Description is a description of the function.
Description string `json:"description"`
// Parameters is a list of parameters for the function.
Parameters any `json:"parameters"`
}
FunctionDefinition is a definition of a function that can be called by the model.
type ImageURLContent ¶
type ImageURLContent struct {
URL string
}
ImageURLContent is content with an URL pointing to an image.
func ImageURLPart ¶
func ImageURLPart(url string) ImageURLContent
ImageURLPart creates a new ImageURLContent from the given URL.
func (ImageURLContent) MarshalJSON ¶
func (iuc ImageURLContent) MarshalJSON() ([]byte, error)
type MessageContent ¶
type MessageContent struct {
Role schema.ChatMessageType
Parts []ContentPart
Name string
}
MessageContent is the content of a message sent to a LLM. It has a role and a sequence of parts. For example, it can represent one message in a chat session sent by the user, in which case Role will be schema.ChatMessageTypeHuman and Parts will be the sequence of items sent in this specific message.
func TextParts ¶
func TextParts(role schema.ChatMessageType, parts ...string) MessageContent
TextParts is a helper function to create a MessageContent with a role and a list of text parts.
type Model ¶
type Model interface {
// GenerateContent asks the model to generate content from a sequence of
// messages. It's the most general interface for multi-modal LLMs that support
// chat-like interactions.
GenerateContent(ctx context.Context, messages []MessageContent, options ...CallOption) (*ContentResponse, error)
// Call is a simplified interface for a text-only Model, generating a single
// string response from a single string prompt.
//
// Deprecated: this method is retained for backwards compatibility. Use the
// more general [GenerateContent] instead. You can also use
// the [GenerateFromSinglePrompt] function which provides a similar capability
// to Call and is built on top of the new interface.
Call(ctx context.Context, prompt string, options ...CallOption) (string, error)
}
Model is an interface multi-modal models implement.
type TextContent ¶
type TextContent struct {
Text string
}
TextContent is content with some text.
func TextPart ¶
func TextPart(s string) TextContent
TextPart creates TextContent from a given string.
func (TextContent) MarshalJSON ¶
func (tc TextContent) MarshalJSON() ([]byte, error)
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package ernie wrapper around the Baidu Large Language Model Platform APIs.
|
Package ernie wrapper around the Baidu Large Language Model Platform APIs. |
|
package googleai implements a langchaingo provider for Google AI LLMs.
|
package googleai implements a langchaingo provider for Google AI LLMs. |
|
internal/cmd
Code generator for vertex.go from googleai.go nolint
|
Code generator for vertex.go from googleai.go nolint |
|
palm
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
|
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models. |
|
vertex
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.
|
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models. |
|
internal/localclient
Package localclient provides a client for local LLMs.
|
Package localclient provides a client for local LLMs. |