gollm

README

gollm - Go Large Language Model

gollm is a Go package designed to help you build your own AI golems. Just as the mystical golem of legend was brought to life with sacred words, gollm empowers you to breathe life into your AI creations using the power of Large Language Models (LLMs). This package simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers to craft their own digital servants.

Documentation

Table of Contents

• Key Features
• Real-World Applications
• Installation
• Quick Start
  • Basic Usage
• Quick Reference
  • LLM Creation and Configuration
  • Prompt Creation
  • Generate Response
  • Chain of Thought
  • Prompt Optimization
  • Model Comparison
• Advanced Usage
  • Prompt Engineering
  • Pre-built Functions (Chain of Thought)
  • Working with Examples
  • Prompt Templates
  • Structured Output (JSON Output Validation)
  • Prompt Optimizer
  • Model Comparison
  • Memory Retention
  • Structured Messages and Caching
• Best Practices
• Examples and Tutorials
• Project Status
• Philosophy
• Contributing
• License

Key Features

• Unified API for Multiple LLM Providers: Interact seamlessly with various providers, including OpenAI, Anthropic, Groq, Ollama, and OpenRouter. Easily switch between models like GPT-4, Claude, and Llama-3.1.
• Easy Provider and Model Switching: Configure preferred providers and models with simple options.
• Flexible Configuration Options: Customize using environment variables, code-based configuration, or configuration files.
• Advanced Prompt Engineering: Craft sophisticated instructions to guide your AI's responses effectively.
• Prompt Optimizer: Automatically refine and improve your prompts for better results, with support for custom metrics and different rating systems.
• Memory Retention: Maintain context across multiple interactions for more coherent conversations.
• Structured Output and Validation: Ensure outputs are consistent and reliable with JSON schema generation and validation.
• Provider Comparison Tools: Test performance across different LLM providers and models for the same task.
• High-Level AI Functions: Use pre-built functions like ChainOfThought for complex reasoning tasks.
• Robust Error Handling and Retries: Built-in retry mechanisms to handle API rate limits and transient errors.
• Extensible Architecture: Easily expand support for new LLM providers and features.

Supported Providers

gollm works with a variety of LLM providers:

• OpenAI: GPT-4o, GPT-4, GPT-3.5 Turbo
• Anthropic: Claude 3 (Opus, Sonnet, Haiku), Claude 2.1
• Groq: Llama-3, Mixtral models with high-speed inference
• Ollama: Local models support (Llama-3, Mistral, etc.)
• Mistral: Mistral Large, Mistral Medium
• OpenRouter: Access to multiple providers through a single API with:
  • Model fallback capabilities
  • Auto-routing between models
  • Prompt caching
  • Reasoning tokens
  • Provider routing preferences

Real-World Applications

gollm can handle a wide range of AI-powered tasks, including:

• Content Creation Workflows: Generate research summaries, article ideas, and refined paragraphs.
• Complex Reasoning Tasks: Use the ChainOfThought function to analyze complex problems step-by-step.
• Structured Data Generation: Create and validate complex data structures with customizable JSON schemas.
• Model Performance Analysis: Compare different models' performance for specific tasks.
• Prompt Optimization: Automatically improve prompts for various tasks.
• Mixture of Agents: Combine responses from multiple LLM providers.

Installation

go get github.com/teilomillet/gollm

Quick Start

Basic Usage

package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "time"

    "github.com/teilomillet/gollm"
)

func main() {
    // Load API key from environment variable
    apiKey := os.Getenv("OPENAI_API_KEY")
    if apiKey == "" {
        log.Fatalf("OPENAI_API_KEY environment variable is not set")
    }

    // Create a new LLM instance with custom configuration
    llm, err := gollm.NewLLM(
        gollm.SetProvider("openai"),
        gollm.SetModel("gpt-4o-mini"),
        gollm.SetAPIKey(apiKey),
        gollm.SetMaxTokens(200),
        gollm.SetMaxRetries(3),
        gollm.SetRetryDelay(time.Second*2),
        gollm.SetLogLevel(gollm.LogLevelInfo),
    )
    if err != nil {
        log.Fatalf("Failed to create LLM: %v", err)
    }

    ctx := context.Background()

    // Create a basic prompt
    prompt := gollm.NewPrompt("Explain the concept of 'recursion' in programming.")

    // Generate a response
    response, err := llm.Generate(ctx, prompt)
    if err != nil {
        log.Fatalf("Failed to generate text: %v", err)
    }
    fmt.Printf("Response:\n%s\n", response)
}

Quick Reference

Here's a quick reference guide for the most commonly used functions and options in the gollm package:

LLM Creation and Configuration

// Using OpenAI
llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-4"),
    gollm.SetAPIKey("your-api-key"),
    gollm.SetMaxTokens(100),
    gollm.SetTemperature(0.7),
    gollm.SetMemory(4096),
)

// Using OpenRouter (with model fallback)
llm, err := gollm.NewLLM(
    gollm.SetProvider("openrouter"),
    gollm.SetModel("anthropic/claude-3-5-sonnet"),
    gollm.SetAPIKey("your-openrouter-api-key"),
    gollm.SetMaxTokens(500),
)
// Enable fallback models if primary model is unavailable
llm.SetOption("fallback_models", []string{"openai/gpt-4o", "mistral/mistral-large"})
// Or use OpenRouter's auto-routing capability
// llm.SetOption("auto_route", true)

Prompt Creation

prompt := gollm.NewPrompt("Your prompt text here",
    gollm.WithContext("Additional context"),
    gollm.WithDirectives("Be concise", "Use examples"),
    gollm.WithOutput("Expected output format"),
    gollm.WithMaxLength(300),
)

Generate Response

response, err := llm.Generate(ctx, prompt)

Chain of Thought

response, err := tools.ChainOfThought(ctx, llm, "Your question here")

Prompt Optimization

optimizer := optimizer.NewPromptOptimizer(llm, initialPrompt, taskDescription,
    optimizer.WithCustomMetrics(/* custom metrics */),
    optimizer.WithRatingSystem("numerical"),
    optimizer.WithThreshold(0.8),
)
optimizedPrompt, err := optimizer.OptimizePrompt(ctx)

Model Comparison

results, err := tools.CompareModels(ctx, promptText, validateFunc, configs...)

Advanced Usage

The gollm package offers a range of advanced features to enhance your AI applications:

Prompt Engineering

Create sophisticated prompts to guide the AI's responses:

prompt := gollm.NewPrompt(
    "Explain the concept of recursion in programming",
    gollm.WithDirectives(
        "Be concise and clear", 
        "Include code examples in multiple languages",
        "Provide a practical example.",
    ),
    gollm.WithContext("This is for a beginner programmer who is just starting to learn."),
    gollm.WithOutput("Structure your response with sections: Definition, Example, Pitfalls, Best Practices."),
    gollm.WithMaxLength(1000),
)

Provider-Specific Features

OpenRouter Special Features

OpenRouter provides access to multiple LLM providers with additional capabilities:

// Create OpenRouter client
llm, err := gollm.NewLLM(
    gollm.SetProvider("openrouter"),
    gollm.SetModel("anthropic/claude-3-5-sonnet"),
    gollm.SetAPIKey(apiKey),
)

// Enable model fallbacks (tried in order if primary model fails)
llm.SetOption("fallback_models", []string{"openai/gpt-4o", "mistral/mistral-large"})

// Use auto-routing (automatically select best model)
llm.SetOption("auto_route", true)

// Enable prompt caching (improves performance and reduces costs)
llm.SetOption("enable_prompt_caching", true)

// Enable reasoning tokens (step-by-step thinking)
llm.SetOption("enable_reasoning", true)

// Specify provider routing preferences
llm.SetOption("provider_preferences", map[string]interface{}{
    "openai": map[string]interface{}{
        "weight": 1.0,
    },
})

Pre-built Functions (Chain of Thought)

Use the ChainOfThought function for step-by-step reasoning:

question := "What is the result of 15 * 7 + 22?"
response, err := tools.ChainOfThought(ctx, llm, question)
if err != nil {
    log.Fatalf("Failed to perform chain of thought: %v", err)
}
fmt.Printf("Chain of Thought:\n%s\n", response)

Working with Examples

Load examples directly from files:

examples, err := utils.ReadExamplesFromFile("examples.txt")
if err != nil {
    log.Fatalf("Failed to read examples: %v", err)
}

prompt := gollm.NewPrompt("Generate a similar example:",
    gollm.WithExamples(examples...),
)

response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to generate example: %v", err)
}
fmt.Printf("Generated Example:\n%s\n", response)

Prompt Templates

Create reusable prompt templates for consistent prompt generation:

// Create a new prompt template
template := gollm.NewPromptTemplate(
    "AnalysisTemplate",
    "A template for analyzing topics",
    "Provide a comprehensive analysis of {{.Topic}}. Consider the following aspects:\n" +
    "1. Historical context\n" +
    "2. Current relevance\n" +
    "3. Future implications",
    gollm.WithPromptOptions(
        gollm.WithDirectives(
            "Use clear and concise language",
            "Provide specific examples where appropriate",
        ),
        gollm.WithOutput("Structure your analysis with clear headings for each aspect."),
    ),
)

// Use the template to create a prompt
data := map[string]interface{}{
    "Topic": "artificial intelligence in healthcare",
}
prompt, err := template.Execute(data)
if err != nil {
    log.Fatalf("Failed to execute template: %v", err)
}

// Generate a response using the created prompt
response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}

fmt.Printf("Analysis:\n%s\n", response)

Structured Output (JSON Output Validation)

Ensure your LLM outputs are in a valid JSON format:

prompt := gollm.NewPrompt("Analyze the pros and cons of remote work.",
    gollm.WithOutput("Respond in JSON format with 'topic', 'pros', 'cons', and 'conclusion' fields."),
)

response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())
if err != nil {
    log.Fatalf("Failed to generate valid analysis: %v", err)
}

var result AnalysisResult
if err := json.Unmarshal([]byte(response), &result); err != nil {
    log.Fatalf("Failed to parse response: %v", err)
}

fmt.Printf("Analysis: %+v\n", result)

Prompt Optimizer

Use the PromptOptimizer to automatically refine and improve your prompts:

initialPrompt := gollm.NewPrompt("Write a short story about a robot learning to love.")
taskDescription := "Generate a compelling short story that explores the theme of artificial intelligence developing emotions."

optimizerInstance := optimizer.NewPromptOptimizer(
    llm,
    initialPrompt,
    taskDescription,
    optimizer.WithCustomMetrics(
        optimizer.Metric{Name: "Creativity", Description: "How original and imaginative the story is"},
        optimizer.Metric{Name: "Emotional Impact", Description: "How well the story evokes feelings in the reader"},
    ),
    optimizer.WithRatingSystem("numerical"),
    optimizer.WithThreshold(0.8),
    optimizer.WithVerbose(),
)

optimizedPrompt, err := optimizerInstance.OptimizePrompt(ctx)
if err != nil {
    log.Fatalf("Optimization failed: %v", err)
}

fmt.Printf("Optimized Prompt: %s\n", optimizedPrompt.Input)

Model Comparison

Compare responses from different LLM providers or models:

configs := []*gollm.Config{
    {
        Provider:  "openai",
        Model:     "gpt-4o-mini",
        APIKey:    os.Getenv("OPENAI_API_KEY"),
        MaxTokens: 500,
    },
    {
        Provider:  "anthropic",
        Model:     "claude-3-5-sonnet-20240620",
        APIKey:    os.Getenv("ANTHROPIC_API_KEY"),
        MaxTokens: 500,
    },
    {
        Provider:  "groq",
        Model:     "llama-3.1-70b-versatile",
        APIKey:    os.Getenv("GROQ_API_KEY"),
        MaxTokens: 500,
    },
}

promptText := "Tell me a joke about programming. Respond in JSON format with 'setup' and 'punchline' fields."

validateJoke := func(joke map[string]interface{}) error {
    if joke["setup"] == "" || joke["punchline"] == "" {
        return fmt.Errorf("joke must have both a setup and a punchline")
    }
    return nil
}

results, err := tools.CompareModels(context.Background(), promptText, validateJoke, configs...)
if err != nil {
    log.Fatalf("Error comparing models: %v", err)
}

fmt.Println(tools.AnalyzeComparisonResults(results))

Memory Retention

Enable memory to maintain context across multiple interactions:

llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-3.5-turbo"),
    gollm.SetAPIKey(os.Getenv("OPENAI_API_KEY")),
    gollm.SetMemory(4096), // Enable memory with a 4096 token limit
)
if err != nil {
    log.Fatalf("Failed to create LLM: %v", err)
}

ctx := context.Background()

// First interaction
prompt1 := gollm.NewPrompt("What's the capital of France?")
response1, err := llm.Generate(ctx, prompt1)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}
fmt.Printf("Response 1: %s\n", response1)

// Second interaction, referencing the first
prompt2 := gollm.NewPrompt("What's the population of that city?")
response2, err := llm.Generate(ctx, prompt2)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}
fmt.Printf("Response 2: %s\n", response2)

Structured Messages and Caching

When using memory with LLMs, gollm preserves each message as a separate structured object (enabled by default). This approach:

• Maintains conversation structure for better LLM understanding
• Enables API-level caching (especially with Anthropic's prompt caching)
• Reduces token costs by allowing providers to cache repeated context

Basic Usage (Automatic)

Structured messages are enabled by default when you use SetMemory():

llm, err := gollm.NewLLM(
    gollm.SetProvider("anthropic"),
    gollm.SetModel("claude-3-5-sonnet-20241022"),
    gollm.SetAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
    gollm.SetMemory(4096), // Structured messages enabled by default
)

Cache Control with Anthropic

For fine-grained control over caching, you can add messages with explicit cache control:

// Add a message with cache control
// "ephemeral" caching is recommended for most use cases
llm.AddStructuredMessage("user", "Here is a long document to analyze...", "ephemeral")

// Generate - subsequent requests with the same context will use cached tokens
response, err := llm.Generate(ctx, gollm.NewPrompt("Summarize the key points"))

Managing Memory

// Check if memory is enabled
if llm.HasMemory() {
    // Get current conversation history
    messages := llm.GetMemory()

    // Add a message manually
    llm.AddToMemory("user", "Remember this context")

    // Clear conversation history
    llm.ClearMemory()
}

Switching Between Modes

You can toggle between structured and flattened message modes:

// Use structured messages (default, recommended for caching)
llm.SetUseStructuredMessages(true)

// Or use flattened prompt (legacy mode, combines all messages into one)
llm.SetUseStructuredMessages(false)

See the examples/caching_structured/ directory for a complete working example.

Best Practices

1. Prompt Engineering:

   • Use NewPrompt() with options like WithContext(), WithDirectives(), and WithOutput() to create well-structured prompts.
   • Example:

     prompt := gollm.NewPrompt("Your main prompt here",
         gollm.WithContext("Provide relevant context"),
         gollm.WithDirectives("Be concise", "Use examples"),
         gollm.WithOutput("Specify expected output format"),
     )
     

2. Utilize Prompt Templates:

   • For consistent prompt generation, create and use `

Documentation

Overview

Package gollm provides a high-level interface for interacting with various Language Learning Models (LLMs). This file re-exports configuration types and functions from the config package to provide a clean, centralized API for configuring LLM interactions.

Package gollm provides a high-level interface for interacting with various Language Learning Models (LLMs). It supports multiple providers including OpenAI, Anthropic, Ollama, and others, with features like prompt optimization, caching, and structured output handling.

Package gollm provides a high-level interface for interacting with Language Learning Models (LLMs).

Package gollm provides prompt handling and manipulation functionality for Language Learning Models. This file contains type definitions, re-exports, and utility functions for working with prompts and their associated components like caching, templates, and message handling.

Package gollm provides streaming functionality for Language Learning Models. This file contains type definitions and re-exports for working with streaming responses.

Package gollm provides validation functionality for Language Learning Model interactions. This file contains utilities for validating structured data and generating JSON schemas, which are essential for ensuring proper data formats in LLM communications.

Index

• Constants
• Variables
• func CleanResponse(response string) string
• func DefaultValidate(s interface{}) error
• func GenerateJSONSchema(v interface{}) ([]byte, error)
• func Validate(s interface{}) error
• func ValidateWithCustomValidator(s interface{}, customValidator ValidatorFunc) error
• type CacheType
• type Config
• type ConfigOption
• type ContentPart
• type Function
• type ImageSource
• type ImageURL
• type LLM
  • func NewLLM(opts ...ConfigOption) (LLM, error)
• type LogLevel
• type Logger
• type MOA
  • func NewMOA(moaConfig MOAConfig, aggregatorOpts ...ConfigOption) (*MOA, error)
  • func (moa *MOA) Generate(ctx context.Context, input string) (string, error)
• type MOAConfig
• type MOALayer
• type MemoryMessage
• type MemoryOption
• type Prompt
• type PromptMessage
• type PromptOption
• type PromptTemplate
• type RetryStrategy
• type SchemaOption
• type StreamConfig
• type StreamOption
• type StreamToken
• type TokenStream
• type Tool
• type ToolCall
• type ToolResult
  • func NewToolError(toolCallID, errorMessage string) ToolResult
  • func NewToolResult(toolCallID, content string) ToolResult
• type ValidatorFunc

Constants

const (
	LogLevelOff   = utils.LogLevelOff   // Disables all logging
	LogLevelError = utils.LogLevelError // Logs only errors
	LogLevelWarn  = utils.LogLevelWarn  // Logs warnings and errors
	LogLevelInfo  = utils.LogLevelInfo  // Logs info, warnings, and errors
	LogLevelDebug = utils.LogLevelDebug // Logs all messages including debug
)

LogLevel constants define available logging verbosity levels

const (
	// CacheTypeEphemeral indicates that cached responses should only persist
	// for the duration of the program's execution.
	CacheTypeEphemeral = llm.CacheTypeEphemeral
)

Cache type constants define the available caching strategies.

Variables

var (
	// LoadConfig loads configuration from environment variables and returns a new Config instance.
	// It automatically detects and loads API keys from environment variables matching the pattern
	// *_API_KEY (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY).
	//
	// Example usage:
	//   cfg, err := LoadConfig()
	//   if err != nil {
	//       log.Fatal(err)
	//   }
	LoadConfig = config.LoadConfig

	// ApplyOptions applies a series of ConfigOption functions to a Config instance.
	// This enables fluent configuration updates using the builder pattern.
	//
	// Example usage:
	//   cfg := NewConfig()
	//   cfg = ApplyOptions(cfg, SetProvider("openai"), SetModel("gpt-3.5-turbo"))
	ApplyOptions = config.ApplyOptions
)

Re-export core configuration functions

var (
	// Provider configuration
	SetProvider       = config.SetProvider       // Sets the LLM provider (e.g., "openai", "anthropic")
	SetModel          = config.SetModel          // Sets the model name for the selected provider
	SetOllamaEndpoint = config.SetOllamaEndpoint // Sets the endpoint URL for Ollama local deployment
	SetVLLMEndpoint   = config.SetVLLMEndpoint   // Sets the endpoint URL for vLLM local deployment
	SetAPIKey         = config.SetAPIKey         // Sets the API key for the current provider

	// Generation parameters
	SetTemperature      = config.SetTemperature      // Controls randomness in generation (0.0-1.0)
	SetMaxTokens        = config.SetMaxTokens        // Sets maximum tokens to generate
	SetTopP             = config.SetTopP             // Controls nucleus sampling
	SetFrequencyPenalty = config.SetFrequencyPenalty // Penalizes frequent token usage
	SetPresencePenalty  = config.SetPresencePenalty  // Penalizes repeated tokens
	SetSeed             = config.SetSeed             // Sets random seed for reproducible generation

	// Advanced generation parameters
	SetMinP          = config.SetMinP          // Sets minimum probability threshold
	SetRepeatPenalty = config.SetRepeatPenalty // Controls repetition penalty
	SetRepeatLastN   = config.SetRepeatLastN   // Sets context window for repetition
	SetMirostat      = config.SetMirostat      // Enables Mirostat sampling
	SetMirostatEta   = config.SetMirostatEta   // Sets Mirostat learning rate
	SetMirostatTau   = config.SetMirostatTau   // Sets Mirostat target entropy
	SetTfsZ          = config.SetTfsZ          // Sets tail-free sampling parameter

	// Runtime configuration
	SetTimeout      = config.SetTimeout      // Sets request timeout duration
	SetMaxRetries   = config.SetMaxRetries   // Sets maximum retry attempts
	SetRetryDelay   = config.SetRetryDelay   // Sets delay between retries
	SetLogLevel     = config.SetLogLevel     // Sets logging verbosity
	SetExtraHeaders = config.SetExtraHeaders // Sets additional HTTP headers

	// Feature toggles
	SetEnableCaching = config.SetEnableCaching // Enables/disables response caching
	SetMemory        = config.SetMemory        // Configures conversation memory

	// Validation configuration
	SetCustomValidator = config.SetCustomValidator // Sets a custom validation function to override default validation

	// Logging
	SetLogger = config.SetLogger // Sets a custom logger implementation

	// Configuration creation
	NewConfig = config.NewConfig // Creates a new Config with default values

	// Logger utilities
	NewNopLogger = utils.NewNopLogger // Creates a no-op logger that discards all output
)

Re-export ConfigOption functions for configuration modification. Each function returns a ConfigOption that can be used with ApplyOptions to modify configuration settings.

var (
	// NewPrompt creates a new prompt instance with the given options.
	NewPrompt = llm.NewPrompt

	// CacheOption configures caching behavior for a prompt.
	CacheOption = llm.CacheOption

	// WithSystemPrompt adds a system-level prompt message.
	WithSystemPrompt = llm.WithSystemPrompt

	// WithMessage adds a single message to the prompt.
	WithMessage = llm.WithMessage

	// WithTools configures available tools for the prompt.
	WithTools = llm.WithTools

	// WithToolChoice specifies how tools should be selected.
	WithToolChoice = llm.WithToolChoice

	// WithMessages adds multiple messages to the prompt.
	WithMessages = llm.WithMessages

	// WithDirectives adds special instructions or constraints.
	WithDirectives = llm.WithDirectives

	// WithOutput configures the expected output format.
	WithOutput = llm.WithOutput

	// WithContext adds contextual information to the prompt.
	WithContext = llm.WithContext

	// WithMaxLength sets the maximum length for generated responses.
	WithMaxLength = llm.WithMaxLength

	// WithExamples adds example conversations or outputs.
	WithExamples = llm.WithExamples

	// WithExpandedStruct enables detailed structure expansion.
	WithExpandedStruct = llm.WithExpandedStruct

	// NewPromptTemplate creates a new template for generating prompts.
	NewPromptTemplate = llm.NewPromptTemplate

	// WithPromptOptions adds multiple prompt options at once.
	WithPromptOptions = llm.WithPromptOptions

	// WithJSONSchemaValidation enables JSON schema validation.
	WithJSONSchemaValidation = llm.WithJSONSchemaValidation

	// WithStream enables or disables streaming responses.
	WithStream = config.WithStream

	// WithImageURL adds an image by URL to the prompt for vision-capable models.
	// The detail parameter controls image processing quality: "auto", "low", or "high".
	WithImageURL = llm.WithImageURL

	// WithImageBase64 adds a base64-encoded image to the prompt for vision-capable models.
	// The mediaType should specify the image format (e.g., "image/png", "image/jpeg").
	WithImageBase64 = llm.WithImageBase64

	// WithImages adds multiple images to the prompt for vision-capable models.
	WithImages = llm.WithImages
)

The following variables are re-exported functions from the llm package. They provide the primary means of constructing and customizing prompts.

Functions

func CleanResponse

func CleanResponse(response string) string

CleanResponse processes and cleans up LLM responses by removing markdown formatting and extracting JSON content. It performs the following operations:

1. Removes markdown code block delimiters (```json)
2. Extracts JSON content between the first '{' and last '}'
3. Trims any remaining whitespace

This is particularly useful when working with LLMs that return formatted markdown or when you need to extract clean JSON from a response.

Parameters:

• response: The raw response string from the LLM

Returns:

• A cleaned string containing only the relevant content

func DefaultValidate

func DefaultValidate(s interface{}) error

DefaultValidate performs the standard struct validation using the go-playground/validator. This can be called from custom validators to fall back to default validation behavior.

Example usage in a custom validator:

gollm.SetCustomValidator(func(v interface{}) error {
    if config, ok := v.(*gollm.Config); ok && config.Provider == "google" {
        return nil // Skip validation for Google
    }
    return gollm.DefaultValidate(v) // Use default for others
})

func GenerateJSONSchema

func GenerateJSONSchema(v interface{}) ([]byte, error)

GenerateJSONSchema generates a JSON schema for the given struct. The schema is generated based on struct fields and their tags, providing a complete JSON Schema that can be used for validation or documentation.

The function analyzes struct fields and generates a schema that includes:

• Field types and formats
• Required fields
• Validation rules from struct tags
• Nested object structures
• Array specifications
• Custom validation rules
• Format constraints

Example usage:

type Message struct {
    Role     string   `json:"role" validate:"required,oneof=system user assistant"`
    Content  string   `json:"content" validate:"required,min=1"`
    Tokens   int      `json:"tokens,omitempty" validate:"min=0"`
    Tags     []string `json:"tags,omitempty" validate:"unique"`
}

type Conversation struct {
    ID       string    `json:"id" validate:"required,uuid"`
    Messages []Message `json:"messages" validate:"required,min=1"`
    Model    string    `json:"model" validate:"required,model"`
}

schema, err := GenerateJSONSchema(&Conversation{})

Parameters:

• v: The struct to generate schema for. Must be a pointer to a struct.

Returns:

• []byte: The generated JSON schema as a byte slice
• error: Any error encountered during schema generation

func Validate

func Validate(s interface{}) error

Validate checks if the given struct is valid according to its validation rules. It uses struct tags to define validation rules and performs comprehensive validation of the input structure.

The function supports various validation rules through struct tags, including:

• required: Field must be present and non-zero
• min/max: Numeric range validation
• len: Exact length requirement
• email: Email format validation
• url: URL format validation
• regex: Pattern matching
• contains/excludes: String content validation
• unique: Array unique items validation
• minItems/maxItems: Array length validation
• password: Password strength validation

Example usage:

type Config struct {
    Model     string `validate:"required,model"`
    MaxTokens int    `validate:"min=1,max=4096"`
    Email     string `validate:"required,email"`
    Password  string `validate:"required,password=strong"`
}

config := Config{
    Model:     "gpt-4",
    MaxTokens: 2048,
    Email:     "user@example.com",
    Password:  "SecureP@ss123",
}
err := Validate(&config)

Parameters:

• s: The struct to validate. Must be a pointer to a struct.

Returns:

• error: nil if validation passes, otherwise returns detailed validation errors

func ValidateWithCustomValidator

func ValidateWithCustomValidator(s interface{}, customValidator ValidatorFunc) error

ValidateWithCustomValidator checks if the given struct is valid, using a custom validator if provided. This allows for scoped custom validation without affecting other goroutines.

Parameters:

• s: The struct to validate. Must be a pointer to a struct.
• customValidator: Optional ValidatorFunc. If nil, uses default validation.

Returns:

• error: nil if validation passes, otherwise returns validation errors

Example:

err := gollm.ValidateWithCustomValidator(cfg, func(v interface{}) error {
    if config, ok := v.(*gollm.Config); ok && config.Provider == "google" {
        return nil // Skip validation for Google
    }
    return gollm.DefaultValidate(v)
})

Types

type CacheType

type CacheType = llm.CacheType

CacheType defines how prompts should be cached in the system. Different cache types affect the persistence and availability of cached responses.

type Config

type Config = config.Config

Config represents the complete configuration for LLM interactions. It includes settings for model selection, API endpoints, generation parameters, and runtime behavior. See config.Config for detailed field documentation.

Example usage:

cfg := NewConfig()
cfg = ApplyOptions(cfg, SetProvider("openai"), SetModel("gpt-3.5-turbo"))

type ConfigOption

type ConfigOption = config.ConfigOption

ConfigOption is a function type that modifies a Config instance. It enables a builder pattern for configuration, allowing for clean and flexible configuration updates.

Example usage:

cfg := NewConfig()
cfg = ApplyOptions(cfg, SetTemperature(0.7), SetMaxTokens(2048))

type ContentPart

type ContentPart = types.ContentPart

ContentPart represents a single part of multimodal content (text or image). Used for vision-capable models that can process both text and images.

type Function

type Function = utils.Function

Function defines a callable function that can be used by the LLM. It includes metadata like name, description, and parameter schemas.

type ImageSource

type ImageSource = types.ImageSource

ImageSource represents a base64-encoded image for vision models.

type ImageURL

type ImageURL = types.ImageURL

ImageURL represents an image specified by URL for vision models.

type LLM

type LLM interface {
	llm.LLM // Embed the base interface
	// GetPromptJSONSchema returns the JSON schema for prompt validation in byte format.
	// The schema can be customized using SchemaOption parameters.
	GetPromptJSONSchema(opts ...SchemaOption) ([]byte, error)
	// GetProvider returns the name of the current LLM provider (e.g., "openai", "anthropic").
	GetProvider() string
	// GetModel returns the name of the current model being used.
	GetModel() string
	// UpdateLogLevel changes the logging verbosity for both gollm and internal LLM operations.
	UpdateLogLevel(level LogLevel)
	// Debug logs a debug message with optional key-value pairs for detailed logging.
	Debug(msg string, keysAndValues ...interface{})
	// GetLogLevel returns the current logging verbosity level.
	GetLogLevel() LogLevel
	// SetOllamaEndpoint configures a custom endpoint for Ollama provider.
	// Returns an error if the current provider doesn't support endpoint configuration.
	SetOllamaEndpoint(endpoint string) error
	// SetSystemPrompt updates the system prompt with caching configuration.
	// The cacheType parameter determines how the prompt should be cached.
	SetSystemPrompt(prompt string, cacheType CacheType)

	// HasMemory returns true if this LLM instance has memory enabled.
	HasMemory() bool
	// ClearMemory removes all messages from the conversation history.
	// Does nothing if memory is not enabled.
	ClearMemory()
	// GetMemory returns the conversation history as a slice of messages.
	// Returns nil if memory is not enabled.
	GetMemory() []MemoryMessage
	// AddToMemory adds a message to the conversation history.
	// Does nothing if memory is not enabled.
	AddToMemory(role, content string)
	// AddStructuredMessage adds a message with cache control to the conversation history.
	// The cacheControl parameter specifies caching behavior (e.g., "ephemeral").
	// Does nothing if memory is not enabled.
	AddStructuredMessage(role, content, cacheControl string)
	// SetUseStructuredMessages configures whether to use structured messages.
	// When true (default), messages are passed as separate JSON objects to the provider.
	// When false, messages are flattened into a single prompt string.
	// Does nothing if memory is not enabled.
	SetUseStructuredMessages(use bool)

	// AddToolResult adds a tool execution result to the conversation history.
	// Use this after executing a tool to provide the result back to the LLM.
	// Does nothing if memory is not enabled.
	AddToolResult(toolCallID, result string)
	// AddToolError adds a tool execution error to the conversation history.
	// Use this when a tool fails to inform the LLM of the error.
	// Does nothing if memory is not enabled.
	AddToolError(toolCallID, errorMessage string)
	// AddAssistantMessageWithToolCalls adds an assistant message with tool calls.
	// This preserves tool calls in the conversation for multi-turn tool usage.
	// Does nothing if memory is not enabled.
	AddAssistantMessageWithToolCalls(content string, toolCalls []ToolCall)
}

LLM is the interface that wraps the basic LLM operations. It extends the base llm.LLM interface with additional functionality specific to gollm, providing a comprehensive set of methods for LLM interaction, configuration, and management.

func NewLLM

func NewLLM(opts ...ConfigOption) (LLM, error)

NewLLM creates a new LLM instance with the specified configuration options. It supports memory management, caching, and provider-specific optimizations. If memory options are provided, it creates an LLM instance with conversation memory.

The function performs the following setup: 1. Loads and applies configuration from both default and provided options 2. Initializes logging system with appropriate verbosity 3. Sets up provider-specific optimizations (e.g., Anthropic caching headers) 4. Creates and configures the base LLM instance 5. Optionally enables conversation memory if specified in config

Returns an error if: - Configuration loading fails - Provider initialization fails - Memory setup fails (if memory option is enabled)

type LogLevel

type LogLevel = utils.LogLevel

LogLevel defines the verbosity of logging output. Available levels are defined as constants: LogLevelOff through LogLevelDebug.

Example usage:

cfg := NewConfig()
cfg = ApplyOptions(cfg, SetLogLevel(LogLevelInfo))

type Logger

type Logger = utils.Logger

Logger is the interface for custom loggers. Implement this interface to use your own logging solution.

type MOA

type MOA struct {
	// Config holds the MOA system configuration
	Config MOAConfig

	// Layers contains the processing layers, each with one or more models
	Layers []MOALayer

	// Aggregator is the LLM instance used to synthesize outputs from multiple models
	Aggregator llm.LLM
}

MOA (Mixture of Agents) implements an ensemble learning system that combines multiple language models to produce higher quality outputs. It processes inputs through multiple layers and iterations, then aggregates the results using a designated aggregator model.

func NewMOA

func NewMOA(moaConfig MOAConfig, aggregatorOpts ...ConfigOption) (*MOA, error)

NewMOA creates a new Mixture of Agents instance with the specified configuration and aggregator options.

Parameters:

• moaConfig: Configuration for the MOA system
• aggregatorOpts: Configuration options for the aggregator model

Returns:

• A configured MOA instance
• Any error encountered during initialization

Example:

moa, err := NewMOA(MOAConfig{
    Iterations:   3,
    Models:      []ConfigOption{SetModel("gpt-4"), SetModel("claude-3")},
    MaxParallel: 2,
})

func (*MOA) Generate

func (moa *MOA) Generate(ctx context.Context, input string) (string, error)

Generate processes the input through the Mixture of Agents and returns the final output. It runs the input through multiple iterations of processing layers, then aggregates the results using the designated aggregator model.

Parameters:

• ctx: Context for cancellation and timeout control
• input: The text input to process

Returns:

• The final processed output
• Any error encountered during processing

The processing flow is: 1. Input is processed through each layer in sequence 2. Each layer's output becomes the input for the next layer 3. This process repeats for the specified number of iterations 4. Results from all iterations are aggregated into the final output

type MOAConfig

type MOAConfig struct {
	// Iterations specifies the number of refinement cycles to perform
	Iterations int

	// Models defines the set of language models to use in the mixture
	// Each model is configured using ConfigOption functions
	Models []ConfigOption

	// MaxParallel limits the number of concurrent model executions
	// If set to 0, all models run concurrently without limitation
	MaxParallel int

	// AgentTimeout specifies the maximum duration for each agent's execution
	// If set to 0, no timeout is applied
	AgentTimeout time.Duration
}

MOAConfig represents the configuration for the Mixture of Agents (MOA) system. MOA is an ensemble learning approach that combines multiple language models to produce higher quality outputs through iterative refinement and aggregation.

type MOALayer

type MOALayer struct {
	// Models contains the LLM instances that operate within this layer
	Models []llm.LLM
}

MOALayer represents a single layer in the Mixture of Agents architecture. Each layer contains one or more language models that process inputs in parallel.

type MemoryMessage

type MemoryMessage = llm.MemoryMessage

MemoryMessage represents a message stored in the LLM's conversation memory. These messages provide context for maintaining coherent conversations.

type MemoryOption

type MemoryOption = config.MemoryOption

MemoryOption configures the memory settings for conversation history. It controls how much context is retained between interactions.

Example usage:

cfg := NewConfig()
cfg = ApplyOptions(cfg, SetMemory(MemoryOption{MaxHistory: 10}))

type Prompt

type Prompt = llm.Prompt

Prompt represents a complete prompt structure that can be sent to an LLM. It includes system messages, user messages, and optional components like tools and cache settings.

type PromptMessage

type PromptMessage = llm.PromptMessage

PromptMessage represents a single message in a conversation with the LLM. It can be a system message, user message, or assistant message.

type PromptOption

type PromptOption = llm.PromptOption

PromptOption defines a function that can modify a prompt's configuration. These are used to customize prompt behavior in a flexible, chainable way.

type PromptTemplate

type PromptTemplate = llm.PromptTemplate

PromptTemplate defines a reusable template for generating prompts. Templates can include variables that are filled in at runtime.

type RetryStrategy

type RetryStrategy = llm.RetryStrategy

RetryStrategy defines the interface for handling stream interruptions.

type SchemaOption

type SchemaOption = llm.SchemaOption

SchemaOption defines options for JSON schema generation. These control how prompts are validated against schemas.

type StreamConfig

type StreamConfig = llm.StreamConfig

StreamConfig holds configuration options for streaming.

type StreamOption

type StreamOption = llm.StreamOption

StreamOption is a function type that modifies StreamConfig

type StreamToken

type StreamToken = llm.StreamToken

StreamToken represents a single token from a streaming response.

type TokenStream

type TokenStream = llm.TokenStream

TokenStream represents a stream of tokens from an LLM response. It follows Go's io.ReadCloser pattern but at the token level.

type Tool

type Tool = utils.Tool

Tool represents a tool that can be used by the LLM during generation. Tools are higher-level abstractions over functions that include usage policies.

type ToolCall

type ToolCall = llm.ToolCall

ToolCall represents a request from the LLM to use a specific tool. It includes the tool name and any arguments needed for execution.

type ToolResult

type ToolResult = types.ToolResult

ToolResult represents the result of executing a tool. Use NewToolResult or NewToolError to create instances.

func NewToolError

func NewToolError(toolCallID, errorMessage string) ToolResult

NewToolError creates an error tool result.

func NewToolResult

func NewToolResult(toolCallID, content string) ToolResult

NewToolResult creates a successful tool result.

type ValidatorFunc

type ValidatorFunc = llm.ValidatorFunc

ValidatorFunc is the signature for custom validation functions. Custom validators receive the value to validate and return an error if validation fails. Use gollm.DefaultValidate within custom validators to fall back to standard validation.

Example:

validator := gollm.ValidatorFunc(func(v interface{}) error {
    if config, ok := v.(*gollm.Config); ok && config.Provider == "google" {
        return nil // Skip validation for Google
    }
    return gollm.DefaultValidate(v)
})