Help us shape the future of Go! Take the 2026 Go Developer Survey

gollm

package module
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 31, 2024 License: AGPL-3.0 Imports: 18 Imported by: 0

README

gollm - Go Abstract Language Model Interface

gollm is a Go package designed to simplify and streamline interactions with various Language Model (LLM) providers. It's built for AI engineers and developers who want a unified, flexible, and powerful interface to work with multiple LLM APIs.

Key Features

  • Unified API: Work with multiple LLM providers (OpenAI, Anthropic, Groq) through a single, consistent interface.
  • Easy Provider Switching: Seamlessly switch between different LLM providers or models with minimal code changes.
  • Flexible Configuration: Configure your LLM interactions via environment variables, code, or command-line flags.
  • High-Level AI Functions: Utilize pre-built functions for common AI tasks like question-answering, summarization, and chain-of-thought reasoning.
  • Advanced Prompt Engineering: Create sophisticated prompts with directives, context, and examples.
  • Provider Comparison: Easily compare responses from multiple LLM providers for the same prompt.
  • JSON Output Validation: Validate and ensure the structure of JSON outputs from LLMs.
  • Extensible Architecture: Add new LLM providers with minimal effort.
  • CLI Tool: Use gollm directly from the command line for quick experiments and workflows.

Installation

go get github.com/teilomillet/gollm

Quick Start

Basic Usage
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/teilomillet/gollm"
)

func main() {
	llm, err := gollm.NewLLM(
		gollm.SetProvider("openai"),
		gollm.SetModel("gpt-4o-mini"),
		gollm.SetMaxTokens(100),
		gollm.SetAPIKey("your-api-key-here"),
	)
	if err != nil {
		log.Fatalf("Failed to create LLM: %v", err)
	}

	ctx := context.Background()

	prompt := gollm.NewPrompt("Tell me a short joke about programming.")
	response, err := llm.Generate(ctx, prompt)
	if err != nil {
		log.Fatalf("Failed to generate text: %v", err)
	}
	fmt.Printf("Response: %s\n", response)
}

For more advanced usage, including research and content refinement, check out the examples directory.

Advanced Usage

Comparing Models

The CompareModels function allows you to easily compare responses from different LLM providers or models:

type JokeResponse struct {
	Setup    string `json:"setup"`
	Punchline string `json:"punchline"`
}

func validateJoke(joke JokeResponse) error {
	if joke.Setup == "" || joke.Punchline == "" {
		return fmt.Errorf("invalid joke structure")
	}
	return nil
}

configs := []*gollm.Config{
	{Provider: "openai", Model: "gpt-4o-mini", APIKey: "your-openai-api-key"},
	{Provider: "anthropic", Model: "claude-3-5-sonnet-20240620	", APIKey: "your-anthropic-api-key"},
}

prompt := "Tell me a joke about programming. Respond in JSON format with 'setup' and 'punchline' fields."

results, err := gollm.CompareModels(context.Background(), prompt, validateJoke, configs...)
if err != nil {
	log.Fatalf("Error comparing models: %v", err)
}

fmt.Println(gollm.AnalyzeComparisonResults(results))

This example compares responses from OpenAI and Anthropic models, ensuring that each response is a valid joke with a setup and punchline.

JSON Output Validation

gollm now supports automatic validation of JSON outputs from LLMs. This is particularly useful when you expect a specific structure in the LLM's response:

type AnalysisResult struct {
	Topic       string   `json:"topic"`
	Pros        []string `json:"pros"`
	Cons        []string `json:"cons"`
	Conclusion  string   `json:"conclusion"`
}

func validateAnalysis(analysis AnalysisResult) error {
	if analysis.Topic == "" || len(analysis.Pros) == 0 || len(analysis.Cons) == 0 || analysis.Conclusion == "" {
		return fmt.Errorf("invalid analysis structure")
	}
	return nil
}

prompt := gollm.NewPrompt("Analyze the pros and cons of remote work.",
	gollm.WithOutput("Respond in JSON format with 'topic', 'pros', 'cons', and 'conclusion' fields."),
)

response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())
if err != nil {
	log.Fatalf("Failed to generate valid analysis: %v", err)
}

var result AnalysisResult
if err := json.Unmarshal([]byte(response), &result); err != nil {
	log.Fatalf("Failed to parse response: %v", err)
}

if err := validateAnalysis(result); err != nil {
	log.Fatalf("Invalid analysis: %v", err)
}

fmt.Printf("Analysis: %+v\n", result)

This example demonstrates how to use JSON schema validation to ensure that the LLM's response matches the expected structure.

Performance Considerations

While gollm adds a thin abstraction layer, its impact on performance is minimal. The main performance factors will be the responsiveness of the chosen LLM provider and the complexity of your prompts.

Streaming Support

Currently, gollm does not support streaming responses. This feature is on our roadmap for future development.

Project Status

gollm is actively maintained and under continuous development. We welcome contributions and feedback from the community.

Examples and Tutorials

Check out our examples directory for more usage examples, including:

  • Basic usage
  • Different prompt types
  • Comparing providers
  • Advanced prompt templates
  • Combining multiple gollm features
  • JSON output validation

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) - see the LICENSE file for details.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var QuestionAnswerTemplate = NewPromptTemplate(
	"QuestionAnswer",
	"Answer the given question",
	"Answer the following question:\n\n{{.Question}}",
	WithPromptOptions(
		WithDirectives("Provide a clear and concise answer"),
		WithOutput("Answer:"),
	),
)

Functions

func AnalyzeComparisonResults added in v0.0.4 

func AnalyzeComparisonResults[T any](results []ComparisonResult[T]) string

func ChainOfThought 

func ChainOfThought(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

ChainOfThought performs a chain of thought reasoning

func CleanResponse added in v0.0.4 

func CleanResponse(response string) string

CleanResponse removes markdown code block syntax and trims the JSON response

func ExtractStructuredData added in v0.0.5 

func ExtractStructuredData[T any](ctx context.Context, l LLM, text string, opts ...PromptOption) (*T, error)

ExtractStructuredData extracts structured data from text based on a given struct type

func GenerateJSONSchema added in v0.0.4 

func GenerateJSONSchema(v interface{}, opts ...SchemaOption) ([]byte, error)

GenerateJSONSchema returns the JSON Schema for the given struct

func QuestionAnswer 

func QuestionAnswer(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

QuestionAnswer performs question answering

func Summarize 

func Summarize(ctx context.Context, l LLM, text string, opts ...PromptOption) (string, error)

Summarize performs text summarization

func Validate added in v0.0.4 

func Validate(s interface{}) error

Validate checks if the given struct is valid according to its validation rules

Types

type ComparisonResult added in v0.0.4 

type ComparisonResult[T any] struct {
	Provider string
	Model    string
	Response string
	Data     T
	Error    error
	Attempts int
}

func CompareModels added in v0.0.4 

func CompareModels[T any](ctx context.Context, prompt string, validateFunc ValidateFunc[T], configs ...*Config) ([]ComparisonResult[T], error)

CompareModels now uses a generic type T and accepts a validation function

type Config added in v0.0.4 

type Config struct {
	Provider    string
	Model       string
	Temperature float64
	MaxTokens   int
	Timeout     time.Duration
	MaxRetries  int
	RetryDelay  time.Duration
	APIKey      string
	DebugLevel  LogLevel
}

Config holds the configuration for the LLM

func LoadConfig added in v0.0.4 

func LoadConfig() (*Config, error)

LoadConfig loads the configuration from environment variables

type ConfigOption added in v0.0.3 

type ConfigOption func(*Config)

ConfigOption is a function type for modifying Config

func SetAPIKey added in v0.0.3 

func SetAPIKey(apiKey string) ConfigOption

SetAPIKey sets the API key for the current provider in the Config

func SetDebugLevel added in v0.0.4 

func SetDebugLevel(level LogLevel) ConfigOption

SetDebugLevel sets the debug level in the Config

func SetMaxRetries added in v0.0.4 

func SetMaxRetries(maxRetries int) ConfigOption

SetMaxRetries sets the maximum number of retries in the Config

func SetMaxTokens added in v0.0.3 

func SetMaxTokens(maxTokens int) ConfigOption

SetMaxTokens sets the max tokens in the Config

func SetModel added in v0.0.3 

func SetModel(model string) ConfigOption

SetModel sets the model in the Config

func SetProvider added in v0.0.3 

func SetProvider(provider string) ConfigOption

SetProvider sets the provider in the Config

func SetRetryDelay added in v0.0.4 

func SetRetryDelay(retryDelay time.Duration) ConfigOption

SetRetryDelay sets the delay between retries in the Config

func SetTemperature added in v0.0.3 

func SetTemperature(temperature float64) ConfigOption

SetTemperature sets the temperature in the Config

func SetTimeout added in v0.0.3 

func SetTimeout(timeout time.Duration) ConfigOption

SetTimeout sets the timeout in the Config

type Example 

type Example struct {
	Content string `json:"content"`
}

type GenerateOption added in v0.0.4 

type GenerateOption func(*generateConfig)

GenerateOption is a function type for configuring generate options

func WithJSONSchemaValidation added in v0.0.4 

func WithJSONSchemaValidation() GenerateOption

WithJSONSchemaValidation returns a GenerateOption that enables JSON schema validation

type LLM 

type LLM interface {
	// Generate produces a response given a context, prompt, and optional generate options
	Generate(ctx context.Context, prompt *Prompt, opts ...GenerateOption) (string, error)

	// SetOption sets an option for the LLM
	SetOption(key string, value interface{})

	// GetPromptJSONSchema returns the JSON schema for the prompt
	GetPromptJSONSchema(opts ...SchemaOption) ([]byte, error)

	// GetProvider returns the provider of the LLM
	GetProvider() string

	// GetModel returns the model of the LLM
	GetModel() string

	// UpdateDebugLevel updates the debug level of the LLM
	UpdateDebugLevel(level LogLevel)

	// Debug logs a debug message with optional key-value pairs
	Debug(msg string, keysAndValues ...interface{})

	// GetDebugLevel returns the current debug level of the LLM
	GetDebugLevel() LogLevel
}

LLM is the interface that wraps the basic LLM operations

func NewLLM 

func NewLLM(opts ...ConfigOption) (LLM, error)

NewLLM creates a new LLM instance

type LogLevel added in v0.0.4 

type LogLevel int

LogLevel represents the level of logging 

const (
	LogLevelOff LogLevel = iota
	LogLevelError
	LogLevelWarn
	LogLevelInfo
	LogLevelDebug
)

type Prompt 

type Prompt struct {
	Input      string   `json:"input" jsonschema:"required,description=The main input text for the LLM" validate:"required"`
	Output     string   `json:"output,omitempty" jsonschema:"description=Specification for the expected output format"`
	Directives []string `json:"directives,omitempty" jsonschema:"description=List of directives to guide the LLM"`
	Context    string   `json:"context,omitempty" jsonschema:"description=Additional context for the LLM"`
	MaxLength  int      `` /* 128-byte string literal not displayed */
	Examples   []string `json:"examples,omitempty" jsonschema:"description=List of examples to guide the LLM"`
}

Prompt represents a structured prompt for an LLM

func NewPrompt 

func NewPrompt(input string, opts ...PromptOption) *Prompt

NewPrompt creates a new Prompt with the given input and applies any provided options

func (*Prompt) Apply added in v0.0.3 

func (p *Prompt) Apply(opts ...PromptOption)

Apply applies the given options to the Prompt

func (*Prompt) GenerateJSONSchema added in v0.0.4 

func (p *Prompt) GenerateJSONSchema(opts ...SchemaOption) ([]byte, error)

GenerateJSONSchema returns the JSON Schema for the Prompt struct

func (*Prompt) String 

func (p *Prompt) String() string

String returns the formatted prompt as a string

func (*Prompt) Validate added in v0.0.4 

func (p *Prompt) Validate() error

Validate checks if the Prompt is valid according to its validation rules

type PromptOption added in v0.0.3 

type PromptOption func(*Prompt)

PromptOption is a function type that modifies a Prompt

func WithContext added in v0.0.3 

func WithContext(context string) PromptOption

WithContext adds context to the Prompt

func WithDirectives added in v0.0.3 

func WithDirectives(directives ...string) PromptOption

WithDirectives adds directives to the Prompt

func WithExamples added in v0.0.3 

func WithExamples(examples ...string) PromptOption

WithExamples adds examples to the Prompt

func WithMaxLength added in v0.0.3 

func WithMaxLength(length int) PromptOption

WithMaxLength sets the maximum length for the output

func WithOutput added in v0.0.3 

func WithOutput(output string) PromptOption

WithOutput adds an output specification to the Prompt

type PromptTemplate added in v0.0.3 

type PromptTemplate struct {
	Name        string
	Description string
	Template    string
	Options     []PromptOption
}

PromptTemplate represents a template for generating prompts

func NewPromptTemplate added in v0.0.3 

func NewPromptTemplate(name, description, template string, opts ...PromptTemplateOption) *PromptTemplate

NewPromptTemplate creates a new PromptTemplate with the given name, description, and template

func (*PromptTemplate) Execute added in v0.0.3 

func (pt *PromptTemplate) Execute(data map[string]interface{}) (*Prompt, error)

Execute generates a Prompt from the PromptTemplate with the given data

type PromptTemplateOption added in v0.0.3 

type PromptTemplateOption func(*PromptTemplate)

PromptTemplateOption is a function type that modifies a PromptTemplate

func WithPromptOptions added in v0.0.3 

func WithPromptOptions(options ...PromptOption) PromptTemplateOption

WithPromptOptions adds PromptOptions to the PromptTemplate

type SchemaOption added in v0.0.4 

type SchemaOption func(*jsonschema.Reflector)

SchemaOption is a function type for schema generation options

func WithExpandedStruct added in v0.0.4 

func WithExpandedStruct(expanded bool) SchemaOption

WithExpandedStruct sets the ExpandedStruct option for schema generation

type ValidateFunc added in v0.0.4 

type ValidateFunc[T any] func(T) error

ValidateFunc is a function type for custom validation logic

Source Files

View all Source files

Directories

Path Synopsis
cmd
goal command
internal
llm

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.