Help us shape the future of Go! Take the 2026 Go Developer Survey

gollm

package module
v0.0.91 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2024 License: AGPL-3.0 Imports: 15 Imported by: 62

README

gollm - Go Large Language Model

Gophers building a robot by Renee French

gollm is a Go package designed to help you build your own AI golems. Just as the mystical golem of legend was brought to life with sacred words, gollm empowers you to breathe life into your AI creations using the power of Large Language Models (LLMs). This package simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers to craft their own digital servants.

Documentation

Key Features

  • Unified API for Multiple LLM Providers: Shape your golem's mind using various providers, including OpenAI, Anthropic, and Groq. Seamlessly switch between models like GPT-4, GPT-4o-mini, Claude, and llama-3.1.
  • Easy Provider and Model Switching: Mold your golem's capabilities by configuring preferred providers and models with simple function calls.
  • Flexible Configuration Options: Customize your golem's essence using environment variables, code-based configuration, or configuration files to suit your project's needs.
  • Advanced Prompt Engineering: Craft sophisticated instructions to guide your golem's responses effectively.
  • Memory Retention: Maintain context across multiple interactions for more coherent conversations.
  • Structured Output and Validation: Ensure your golem's outputs are consistent and reliable with JSON schema generation and validation.
  • Provider Comparison Tools: Test your golem's performance across different LLM providers and models for the same task.
  • High-Level AI Functions: Empower your golem with pre-built functions like ChainOfThought for complex reasoning tasks.
  • Robust Error Handling and Retries: Build resilience into your golem with built-in retry mechanisms to handle API rate limits and transient errors.
  • Extensible Architecture: Easily expand your golem's capabilities by extending support for new LLM providers and features.
  • Debug Logging: Fine-tune your golem's performance with configurable debug levels.

Real-World Applications

Your gollm-powered golems can handle a wide range of AI-powered tasks, including:

  • Content Creation Workflows: Generate research summaries, article ideas, and refined paragraphs for writing projects.
  • Complex Reasoning Tasks: Use the ChainOfThought function to break down and analyze complex problems step-by-step.
  • Structured Data Generation: Create and validate complex data structures with customizable JSON schemas.
  • Model Performance Analysis: Compare different models' performance for specific tasks to optimize your AI pipeline.

Installation

go get github.com/teilomillet/gollm

Quick Start

Basic Usage
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/teilomillet/gollm"
)

func main() {
	llm, err := gollm.NewLLM(
		gollm.SetProvider("openai"),
		gollm.SetModel("gpt-4o-mini"),
		gollm.SetMaxTokens(100),
		gollm.SetAPIKey("your-api-key-here"),
	)
	if err != nil {
		log.Fatalf("Failed to create LLM: %v", err)
	}

	ctx := context.Background()

	prompt := gollm.NewPrompt("Tell me a short joke about programming.")
	response, err := llm.Generate(ctx, prompt)
	if err != nil {
		log.Fatalf("Failed to generate text: %v", err)
	}
	fmt.Printf("Response: %s\n", response)
}

For more advanced usage, including research and content refinement, check out the examples directory.

Advanced Usage

Using Memory with LLM

gollm now supports adding memory to your LLM instances. This allows the LLM to maintain context across multiple interactions. To use this feature:

  1. When creating a new LLM instance, use the SetMemory option:
llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-3.5-turbo"),
    gollm.SetAPIKey(os.Getenv("OPENAI_API_KEY")),
    gollm.SetMemory(4096), // Enable memory with a 4096 token limit
)
  1. Use the LLM as usual. The memory will automatically be maintained:
response, err := llm.Generate(context.Background(), gollm.NewPrompt("Hello, how are you?"))
// The next generation will include the context of the previous interaction
response, err = llm.Generate(context.Background(), gollm.NewPrompt("What did I just ask you?"))
  1. If needed, you can clear the memory:
if llmWithMemory, ok := llm.(*gollm.LLMWithMemory); ok {
    llmWithMemory.ClearMemory()
}

The memory feature uses tiktoken for accurate token counting and automatically manages the conversation history within the specified token limit.

Comparing Models

The CompareModels function allows you to easily compare responses from different LLM providers or models:

type JokeResponse struct {
	Setup    string `json:"setup"`
	Punchline string `json:"punchline"`
}

func validateJoke(joke JokeResponse) error {
	if joke.Setup == "" || joke.Punchline == "" {
		return fmt.Errorf("invalid joke structure")
	}
	return nil
}

configs := []*gollm.Config{
	{Provider: "openai", Model: "gpt-4o-mini", APIKey: "your-openai-api-key"},
	{Provider: "anthropic", Model: "claude-3-5-sonnet-20240620	", APIKey: "your-anthropic-api-key"},
	{Provider: "groq", Model: "llama-3.1-70b-versatile", APIKey: "your-anthropic-api-key"},

}

prompt := "Tell me a joke about programming. Respond in JSON format with 'setup' and 'punchline' fields."

results, err := gollm.CompareModels(context.Background(), prompt, validateJoke, configs...)
if err != nil {
	log.Fatalf("Error comparing models: %v", err)
}

fmt.Println(gollm.AnalyzeComparisonResults(results))

This example compares responses from OpenAI and Anthropic models, ensuring that each response is a valid joke with a setup and punchline.

JSON Output Validation

gollm now supports automatic validation of JSON outputs from LLMs. This is particularly useful when you expect a specific structure in the LLM's response:

type AnalysisResult struct {
	Topic       string   `json:"topic"`
	Pros        []string `json:"pros"`
	Cons        []string `json:"cons"`
	Conclusion  string   `json:"conclusion"`
}

func validateAnalysis(analysis AnalysisResult) error {
	if analysis.Topic == "" || len(analysis.Pros) == 0 || len(analysis.Cons) == 0 || analysis.Conclusion == "" {
		return fmt.Errorf("invalid analysis structure")
	}
	return nil
}

prompt := gollm.NewPrompt("Analyze the pros and cons of remote work.",
	gollm.WithOutput("Respond in JSON format with 'topic', 'pros', 'cons', and 'conclusion' fields."),
)

response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())
if err != nil {
	log.Fatalf("Failed to generate valid analysis: %v", err)
}

var result AnalysisResult
if err := json.Unmarshal([]byte(response), &result); err != nil {
	log.Fatalf("Failed to parse response: %v", err)
}

if err := validateAnalysis(result); err != nil {
	log.Fatalf("Invalid analysis: %v", err)
}

fmt.Printf("Analysis: %+v\n", result)

Find other examples that demonstrates how to use JSON schema validation to ensure that the LLM's response matches the expected structure in the examples/.

Streaming Support

Currently, gollm does not support streaming responses. This feature is on our roadmap for future development.

Project Status

gollm is actively maintained and under continuous development. We welcome contributions and feedback from the community.

Examples and Tutorials

Check out our examples directory for more usage examples, including:

  • Basic usage
  • Different prompt types
  • Comparing providers
  • Advanced prompt templates
  • Combining multiple gollm features
  • JSON output validation

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) - see the LICENSE file for details.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var QuestionAnswerTemplate = NewPromptTemplate(
	"QuestionAnswer",
	"Answer the given question",
	"Answer the following question:\n\n{{.Question}}",
	WithPromptOptions(
		WithDirectives("Provide a clear and concise answer"),
		WithOutput("Answer:"),
	),
)

Functions

func AnalyzeComparisonResults 

func AnalyzeComparisonResults[T any](results []ComparisonResult[T]) string

func ChainOfThought 

func ChainOfThought(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

ChainOfThought performs a chain of thought reasoning

func CleanResponse 

func CleanResponse(response string) string

CleanResponse removes markdown code block syntax and trims the JSON response

func ExtractStructuredData 

func ExtractStructuredData[T any](ctx context.Context, l LLM, text string, opts ...PromptOption) (*T, error)

ExtractStructuredData extracts structured data from text based on a given struct type

func GenerateJSONSchema 

func GenerateJSONSchema(v interface{}) ([]byte, error)

GenerateJSONSchema generates a JSON schema for the given struct

func QuestionAnswer 

func QuestionAnswer(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

QuestionAnswer performs question answering

func Summarize 

func Summarize(ctx context.Context, l LLM, text string, opts ...PromptOption) (string, error)

Summarize performs text summarization

func Validate 

func Validate(s interface{}) error

Validate checks if the given struct is valid according to its validation rules

Types

type ComparisonResult 

type ComparisonResult[T any] struct {
	Provider string
	Model    string
	Response string
	Data     T
	Error    error
	Attempts int
}

func CompareModels 

func CompareModels[T any](ctx context.Context, prompt string, validateFunc ValidateFunc[T], configs ...*Config) ([]ComparisonResult[T], error)

CompareModels now uses a generic type T and accepts a validation function

type Config 

type Config struct {
	Provider       string
	Model          string
	OllamaEndpoint string
	Temperature    float64
	MaxTokens      int
	Timeout        time.Duration
	MaxRetries     int
	RetryDelay     time.Duration
	APIKey         string
	DebugLevel     LogLevel
	MemoryOption   *MemoryOption
}

Config holds the configuration for the LLM

func LoadConfig 

func LoadConfig() (*Config, error)

LoadConfig loads the configuration from environment variables

type ConfigOption 

type ConfigOption func(*Config)

ConfigOption is a function type for modifying Config

func SetAPIKey 

func SetAPIKey(apiKey string) ConfigOption

SetAPIKey sets the API key for the current provider in the Config

func SetDebugLevel 

func SetDebugLevel(level LogLevel) ConfigOption

SetDebugLevel sets the debug level in the Config

func SetMaxRetries 

func SetMaxRetries(maxRetries int) ConfigOption

SetMaxRetries sets the maximum number of retries in the Config

func SetMaxTokens 

func SetMaxTokens(maxTokens int) ConfigOption

SetMaxTokens sets the max tokens in the Config

func SetMemory added in v0.0.8 

func SetMemory(maxTokens int) ConfigOption

SetMemory sets the memory option for the LLM

func SetModel 

func SetModel(model string) ConfigOption

SetModel sets the model in the Config

func SetOllamaEndpoint 

func SetOllamaEndpoint(endpoint string) ConfigOption

SetOllamaEndpoint sets the Ollama endpoint in the Config

func SetProvider 

func SetProvider(provider string) ConfigOption

SetProvider sets the provider in the Config

func SetRetryDelay 

func SetRetryDelay(retryDelay time.Duration) ConfigOption

SetRetryDelay sets the delay between retries in the Config

func SetTemperature 

func SetTemperature(temperature float64) ConfigOption

SetTemperature sets the temperature in the Config

func SetTimeout 

func SetTimeout(timeout time.Duration) ConfigOption

SetTimeout sets the timeout in the Config

type Example 

type Example struct {
	Content string `json:"content"`
}

type GenerateOption 

type GenerateOption func(*generateConfig)

GenerateOption is a function type for configuring generate options

func WithJSONSchemaValidation 

func WithJSONSchemaValidation() GenerateOption

WithJSONSchemaValidation returns a GenerateOption that enables JSON schema validation

type IterationCallback added in v0.0.9 

type IterationCallback func(iteration int, entry OptimizationEntry)

IterationCallback is a function type for the iteration callback

type LLM 

type LLM interface {
	// Generate produces a response given a context, prompt, and optional generate options
	Generate(ctx context.Context, prompt *Prompt, opts ...GenerateOption) (string, error)

	// SetOption sets an option for the LLM
	SetOption(key string, value interface{})

	// GetPromptJSONSchema returns the JSON schema for the prompt
	GetPromptJSONSchema(opts ...SchemaOption) ([]byte, error)

	// GetProvider returns the provider of the LLM
	GetProvider() string

	// GetModel returns the model of the LLM
	GetModel() string

	// UpdateDebugLevel updates the debug level of the LLM
	UpdateDebugLevel(level LogLevel)

	// Debug logs a debug message with optional key-value pairs
	Debug(msg string, keysAndValues ...interface{})

	// GetDebugLevel returns the current debug level of the LLM
	GetDebugLevel() LogLevel

	SetOllamaEndpoint(endpoint string) error
}

LLM is the interface that wraps the basic LLM operations

func NewLLM 

func NewLLM(opts ...ConfigOption) (LLM, error)

NewLLM creates a new LLM instance, potentially with memory if the option is set

type LogLevel 

type LogLevel int

LogLevel represents the level of logging 

const (
	LogLevelOff LogLevel = iota
	LogLevelError
	LogLevelWarn
	LogLevelInfo
	LogLevelDebug
)

type MemoryOption added in v0.0.8 

type MemoryOption struct {
	MaxTokens int
}

type Metric added in v0.0.9 

type Metric = llm.Metric

Type aliases to bridge public and internal types

type OptimizationEntry added in v0.0.9 

type OptimizationEntry = llm.OptimizationEntry

type OptimizerOption added in v0.0.9 

type OptimizerOption func(*PromptOptimizer)

OptimizerOption is a function type for configuring the PromptOptimizer

func WithCustomMetrics added in v0.0.9 

func WithCustomMetrics(metrics ...Metric) OptimizerOption

WithCustomMetrics adds custom metrics to the PromptOptimizer

func WithIterationCallback added in v0.0.9 

func WithIterationCallback(callback IterationCallback) OptimizerOption

Modify the existing WithIterationCallback function

func WithIterations added in v0.0.91 

func WithIterations(iterations int) OptimizerOption

func WithMaxRetries added in v0.0.9 

func WithMaxRetries(maxRetries int) OptimizerOption

WithMaxRetries sets the maximum number of retries for the PromptOptimizer

func WithMemorySize added in v0.0.91 

func WithMemorySize(size int) OptimizerOption

WithMemorySize sets the memory size for the PromptOptimizer

func WithOptimizationGoal added in v0.0.9 

func WithOptimizationGoal(goal string) OptimizerOption

WithOptimizationGoal sets the optimization goal for the PromptOptimizer

func WithRatingSystem added in v0.0.9 

func WithRatingSystem(system string) OptimizerOption

WithRatingSystem sets the rating system for the PromptOptimizer

func WithRetryDelay added in v0.0.9 

func WithRetryDelay(delay time.Duration) OptimizerOption

WithRetryDelay sets the delay between retries for the PromptOptimizer

func WithThreshold added in v0.0.9 

func WithThreshold(threshold float64) OptimizerOption

WithThreshold sets the threshold for the PromptOptimizer

type Prompt 

type Prompt struct {
	Input      string   `json:"input" jsonschema:"required,description=The main input text for the LLM" validate:"required"`
	Output     string   `json:"output,omitempty" jsonschema:"description=Specification for the expected output format"`
	Directives []string `json:"directives,omitempty" jsonschema:"description=List of directives to guide the LLM"`
	Context    string   `json:"context,omitempty" jsonschema:"description=Additional context for the LLM"`
	MaxLength  int      `` /* 128-byte string literal not displayed */
	Examples   []string `json:"examples,omitempty" jsonschema:"description=List of examples to guide the LLM"`
}

Prompt represents a structured prompt for an LLM

func NewPrompt 

func NewPrompt(input string, opts ...PromptOption) *Prompt

NewPrompt creates a new Prompt with the given input and applies any provided options

func (*Prompt) Apply 

func (p *Prompt) Apply(opts ...PromptOption)

Apply applies the given options to the Prompt

func (*Prompt) GenerateJSONSchema 

func (p *Prompt) GenerateJSONSchema(opts ...SchemaOption) ([]byte, error)

GenerateJSONSchema returns the JSON Schema for the Prompt struct

func (*Prompt) String 

func (p *Prompt) String() string

String returns the formatted prompt as a string

func (*Prompt) Validate 

func (p *Prompt) Validate() error

Validate checks if the Prompt is valid according to its validation rules

type PromptOptimizer added in v0.0.9 

type PromptOptimizer struct {
	// contains filtered or unexported fields
}

PromptOptimizer is the public interface for the prompt optimization system

func NewPromptOptimizer added in v0.0.9 

func NewPromptOptimizer(l LLM, initialPrompt string, taskDesc string, opts ...OptimizerOption) *PromptOptimizer

NewPromptOptimizer creates a new PromptOptimizer

func (*PromptOptimizer) GetOptimizationHistory added in v0.0.9 

func (po *PromptOptimizer) GetOptimizationHistory() []OptimizationEntry

GetOptimizationHistory returns the history of optimization attempts

func (*PromptOptimizer) OptimizePrompt added in v0.0.9 

func (po *PromptOptimizer) OptimizePrompt(ctx context.Context) (string, error)

OptimizePrompt runs the optimization process

type PromptOption 

type PromptOption func(*Prompt)

PromptOption is a function type that modifies a Prompt

func WithContext 

func WithContext(context string) PromptOption

WithContext adds context to the Prompt

func WithDirectives 

func WithDirectives(directives ...string) PromptOption

WithDirectives adds directives to the Prompt

func WithExamples 

func WithExamples(examples ...string) PromptOption

WithExamples adds examples to the Prompt

func WithMaxLength 

func WithMaxLength(length int) PromptOption

WithMaxLength sets the maximum length for the output

func WithOutput 

func WithOutput(output string) PromptOption

WithOutput adds an output specification to the Prompt

type PromptTemplate 

type PromptTemplate struct {
	Name        string
	Description string
	Template    string
	Options     []PromptOption
}

PromptTemplate represents a template for generating prompts

func NewPromptTemplate 

func NewPromptTemplate(name, description, template string, opts ...PromptTemplateOption) *PromptTemplate

NewPromptTemplate creates a new PromptTemplate with the given name, description, and template

func (*PromptTemplate) Execute 

func (pt *PromptTemplate) Execute(data map[string]interface{}) (*Prompt, error)

Execute generates a Prompt from the PromptTemplate with the given data

type PromptTemplateOption 

type PromptTemplateOption func(*PromptTemplate)

PromptTemplateOption is a function type that modifies a PromptTemplate

func WithPromptOptions 

func WithPromptOptions(options ...PromptOption) PromptTemplateOption

WithPromptOptions adds PromptOptions to the PromptTemplate

type SchemaOption 

type SchemaOption func(*jsonschema.Reflector)

SchemaOption is a function type for schema generation options

func WithExpandedStruct 

func WithExpandedStruct(expanded bool) SchemaOption

WithExpandedStruct sets the ExpandedStruct option for schema generation

type ValidateFunc 

type ValidateFunc[T any] func(T) error

ValidateFunc is a function type for custom validation logic

Source Files

View all Source files

Directories

Path Synopsis
cmd
goal command
internal
llm

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.