Help us shape the future of Go! Take the 2026 Go Developer Survey

gollm

package module
v0.0.95 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 19, 2024 License: Apache-2.0 Imports: 16 Imported by: 62

README

gollm - Go Large Language Model

Gophers building a robot by Renee French

gollm is a Go package designed to help you build your own AI golems. Just as the mystical golem of legend was brought to life with sacred words, gollm empowers you to breathe life into your AI creations using the power of Large Language Models (LLMs). This package simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers to craft their own digital servants.

Documentation

Table of Contents

Key Features

  • Unified API for Multiple LLM Providers: Shape your golem's mind using various providers, including OpenAI, Anthropic Groq and Ollama. Seamlessly switch between models like GPT-4, GPT-4o-mini, Claude, and llama-3.1.
  • Easy Provider and Model Switching: Mold your golem's capabilities by configuring preferred providers and models with simple function calls.
  • Flexible Configuration Options: Customize your golem's essence using environment variables, code-based configuration, or configuration files to suit your project's needs.
  • Advanced Prompt Engineering: Craft sophisticated instructions to guide your golem's responses effectively.
  • PromptOptimizer: Automatically refine and improve your prompts for better results, with support for custom metrics and different rating systems.
  • Memory Retention: Maintain context across multiple interactions for more coherent conversations.
  • Structured Output and Validation: Ensure your golem's outputs are consistent and reliable with JSON schema generation and validation.
  • Provider Comparison Tools: Test your golem's performance across different LLM providers and models for the same task.
  • High-Level AI Functions: Empower your golem with pre-built functions like ChainOfThought for complex reasoning tasks.
  • Robust Error Handling and Retries: Build resilience into your golem with built-in retry mechanisms to handle API rate limits and transient errors.
  • Extensible Architecture: Easily expand your golem's capabilities by extending support for new LLM providers and features.

Real-World Applications

Your gollm-powered golems can handle a wide range of AI-powered tasks, including:

  • Content Creation Workflows: Generate research summaries, article ideas, and refined paragraphs for writing projects.
  • Complex Reasoning Tasks: Use the ChainOfThought function to break down and analyze complex problems step-by-step.
  • Structured Data Generation: Create and validate complex data structures with customizable JSON schemas.
  • Model Performance Analysis: Compare different models' performance for specific tasks to optimize your AI pipeline.
  • Prompt Optimization: Automatically improve prompts for various tasks, from creative writing to technical documentation.
  • Mixture of Agents: Combine responses from multiple LLM providers to create diverse and robust AI agents.

Installation

go get github.com/teilomillet/gollm

Quick Start

Basic Usage
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/teilomillet/gollm"
)

func main() {
	llm, err := gollm.NewLLM(
		gollm.SetProvider("openai"),
		gollm.SetModel("gpt-4o-mini"),
		gollm.SetMaxTokens(100),
		gollm.SetAPIKey("your-api-key-here"),
	)
	if err != nil {
		log.Fatalf("Failed to create LLM: %v", err)
	}

	ctx := context.Background()

	prompt := gollm.NewPrompt("Tell me a short joke about programming.")
	response, err := llm.Generate(ctx, prompt)
	if err != nil {
		log.Fatalf("Failed to generate text: %v", err)
	}
	fmt.Printf("Response: %s\n", response)
}

For more advanced usage, including research and content refinement, check out the examples directory.

Quick Reference

Here's a quick reference guide for the most commonly used functions and options in the gollm package:

LLM Creation and Configuration
llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-4"),
    gollm.SetAPIKey("your-api-key"),
    gollm.SetMaxTokens(100),
    gollm.SetTemperature(0.7),
    gollm.SetMemory(4096),
)
Prompt Creation
prompt := gollm.NewPrompt("Your prompt text here",
    gollm.WithContext("Additional context"),
    gollm.WithDirectives("Be concise", "Use examples"),
    gollm.WithOutput("Expected output format"),
    gollm.WithMaxLength(300),
)
Generate Response
response, err := llm.Generate(ctx, prompt)
Chain of Thought
response, err := gollm.ChainOfThought(ctx, llm, "Your question here")
Prompt Optimization
optimizer := gollm.NewPromptOptimizer(llm, initialPrompt, taskDescription,
    gollm.WithCustomMetrics(/* custom metrics */),
    gollm.WithRatingSystem("numerical"),
    gollm.WithThreshold(0.8),
)
optimizedPrompt, err := optimizer.OptimizePrompt(ctx)
Model Comparison
results, err := gollm.CompareModels(ctx, prompt, validateFunc, config1, config2, config3)

Advanced Usage

The gollm package offers a range of advanced features to enhance your AI applications:

  • Prompt Engineering
  • Pre-built Functions (e.g., Chain of Thought)
  • Working with Examples
  • Prompt Templates
  • Structured Output (JSON output validation)
  • Prompt Optimizer
  • Model Comparison
  • Memory Retention

Here are examples of how to use these advanced features:

Prompt Engineering

Create sophisticated prompts with multiple components:

prompt := gollm.NewPrompt("Explain the concept of recursion in programming.",
    gollm.WithContext("The audience is beginner programmers."),
    gollm.WithDirectives(
        "Use simple language and avoid jargon.",
        "Provide a practical example.",
        "Explain potential pitfalls and how to avoid them.",
    ),
    gollm.WithOutput("Structure your response with sections: Definition, Example, Pitfalls, Best Practices."),
    gollm.WithMaxLength(300),
)

response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to generate explanation: %v", err)
}

fmt.Printf("Explanation of Recursion:\n%s\n", response)
Pre-built Functions (Chain of Thought)

Use the ChainOfThought function for step-by-step reasoning:

question := "What is the result of 15 * 7 + 22?"
response, err := gollm.ChainOfThought(ctx, llm, question)
if err != nil {
    log.Fatalf("Failed to perform chain of thought: %v", err)
}
fmt.Printf("Chain of Thought:\n%s\n", response)
Working with Examples

Load examples directly from files:

examples, err := gollm.readExamplesFromFile("examples.txt")
if err != nil {
    log.Fatalf("Failed to read examples: %v", err)
}

prompt := gollm.NewPrompt("Generate a similar example:",
    gollm.WithExamples(examples...),
)

response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to generate example: %v", err)
}
fmt.Printf("Generated Example:\n%s\n", response)
Prompt Templates

Create reusable prompt templates for consistent prompt generation:

// Create a new prompt template
template := gollm.NewPromptTemplate(
    "AnalysisTemplate",
    "A template for analyzing topics",
    "Provide a comprehensive analysis of {{.Topic}}. Consider the following aspects:\n" +
    "1. Historical context\n" +
    "2. Current relevance\n" +
    "3. Future implications",
    gollm.WithPromptOptions(
        gollm.WithDirectives(
            "Use clear and concise language",
            "Provide specific examples where appropriate",
        ),
        gollm.WithOutput("Structure your analysis with clear headings for each aspect."),
    ),
)

// Use the template to create a prompt
data := map[string]interface{}{
    "Topic": "artificial intelligence in healthcare",
}
prompt, err := template.Execute(data)
if err != nil {
    log.Fatalf("Failed to execute template: %v", err)
}

// Generate a response using the created prompt
response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}

fmt.Printf("Analysis:\n%s\n", response)
Structured Output (JSON Output Validation)

Ensure your LLM outputs are in a valid JSON format:

prompt := gollm.NewPrompt("Analyze the pros and cons of remote work.",
    gollm.WithOutput("Respond in JSON format with 'topic', 'pros', 'cons', and 'conclusion' fields."),
)

response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())
if err != nil {
    log.Fatalf("Failed to generate valid analysis: %v", err)
}

var result AnalysisResult
if err := json.Unmarshal([]byte(response), &result); err != nil {
    log.Fatalf("Failed to parse response: %v", err)
}

fmt.Printf("Analysis: %+v\n", result)
Prompt Optimizer

Use the PromptOptimizer to automatically refine and improve your prompts:

initialPrompt := "Write a short story about a robot learning to love."
taskDescription := "Generate a compelling short story that explores the theme of artificial intelligence developing emotions."

optimizer := gollm.NewPromptOptimizer(llm, initialPrompt, taskDescription,
    gollm.WithCustomMetrics(
        gollm.Metric{Name: "Creativity", Description: "How original and imaginative the story is"},
        gollm.Metric{Name: "Emotional Impact", Description: "How well the story evokes feelings in the reader"},
    ),
    gollm.WithRatingSystem("numerical"),
    gollm.WithThreshold(0.8),
    gollm.WithVerbose(),
)

optimizedPrompt, err := optimizer.OptimizePrompt(ctx)
if err != nil {
    log.Fatalf("Optimization failed: %v", err)
}

fmt.Printf("Optimized Prompt: %s\n", optimizedPrompt)
Model Comparison

Compare responses from different LLM providers or models:

configs := []*gollm.Config{
    {Provider: "openai", Model: "gpt-4o-mini", APIKey: "your-openai-api-key"},
    {Provider: "anthropic", Model: "claude-3-5-sonnet-20240620", APIKey: "your-anthropic-api-key"},
    {Provider: "groq", Model: "llama-3.1-70b-versatile", APIKey: "your-groq-api-key"},
}

prompt := "Tell me a joke about programming. Respond in JSON format with 'setup' and 'punchline' fields."

results, err := gollm.CompareModels(context.Background(), prompt, validateJoke, configs...)
if err != nil {
    log.Fatalf("Error comparing models: %v", err)
}

fmt.Println(gollm.AnalyzeComparisonResults(results))
Memory Retention

Enable memory to maintain context across multiple interactions:

llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-3.5-turbo"),
    gollm.SetAPIKey(os.Getenv("OPENAI_API_KEY")),
    gollm.SetMemory(4096), // Enable memory with a 4096 token limit
)
if err != nil {
    log.Fatalf("Failed to create LLM: %v", err)
}

ctx := context.Background()

// First interaction
prompt1 := gollm.NewPrompt("What's the capital of France?")
response1, err := llm.Generate(ctx, prompt1)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}
fmt.Printf("Response 1: %s\n", response1)

// Second interaction, referencing the first
prompt2 := gollm.NewPrompt("What's the population of that city?")
response2, err := llm.Generate(ctx, prompt2)
if err != nil {
    log.Fatalf("Failed to generate response: %v", err)
}
fmt.Printf("Response 2: %s\n", response2)

Best Practices

  1. Prompt Engineering:

    • Use the NewPrompt() function with options like WithContext(), WithDirectives(), and WithOutput() to create well-structured prompts.
    • Example: 
      prompt := gollm.NewPrompt("Your main prompt here",
    gollm.WithContext("Provide relevant context"),
    gollm.WithDirectives("Be concise", "Use examples"),
    gollm.WithOutput("Specify expected output format"),
)

  2. Utilize Prompt Templates:

    • For consistent prompt generation, create and use PromptTemplate objects.
    • Example: 
      template := gollm.NewPromptTemplate(
    "CustomTemplate",
    "A template for custom prompts",
    "Generate a {{.Type}} about {{.Topic}}",
    gollm.WithPromptOptions(
        gollm.WithDirectives("Be creative", "Use vivid language"),
        gollm.WithOutput("Your {{.Type}}:"),
    ),
)

  3. Leverage Pre-built Functions:

    • Use provided functions like ChainOfThought() for complex reasoning tasks.
    • Example: 
      response, err := gollm.ChainOfThought(ctx, llm, "Your complex question here")

  4. Work with Examples:

    • Use the readExamplesFromFile() function to load examples from files for more consistent and varied outputs.
    • Example: 
      examples, err := gollm.readExamplesFromFile("examples.txt")
if err != nil {
    log.Fatalf("Failed to read examples: %v", err)
}

  5. Implement Structured Output:

    • Use the WithJSONSchemaValidation() option when generating responses to ensure valid JSON outputs.
    • Example: 
      response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())

  6. Optimize Prompts:

    • Utilize the PromptOptimizer to refine and improve your prompts automatically.
    • Example: 
      optimizer := gollm.NewPromptOptimizer(llm, initialPrompt, taskDescription,
    gollm.WithCustomMetrics(
        gollm.Metric{Name: "Relevance", Description: "How relevant the response is to the task"},
    ),
    gollm.WithRatingSystem("numerical"),
    gollm.WithThreshold(0.8),
)
optimizedPrompt, err := optimizer.OptimizePrompt(ctx)

  7. Compare Model Performances:

    • Use the CompareModels() function to evaluate different models or providers for specific tasks.
    • Example: 
      results, err := gollm.CompareModels(ctx, prompt, validateFunc, config1, config2, config3)

  8. Implement Memory for Contextual Interactions:

    • Enable memory retention for maintaining context across multiple interactions.
    • Example: 
      llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-3.5-turbo"),
    gollm.SetMemory(4096), // Enable memory with a 4096 token limit
)

  9. Error Handling and Retries:

    • Always check for errors returned by gollm functions.
    • Configure retry mechanisms to handle transient errors and rate limits.
    • Example: 
      llm, err := gollm.NewLLM(
    gollm.SetMaxRetries(3),
    gollm.SetRetryDelay(time.Second * 2),
)

  10. Secure API Key Handling:

    • Use environment variables or secure configuration management to handle API keys.
    • Example: 
      llm, err := gollm.NewLLM(
    gollm.SetAPIKey(os.Getenv("OPENAI_API_KEY")),
)

By following these best practices, you can make the most effective use of the gollm package, creating more robust, efficient, and maintainable AI-powered applications.

Examples and Tutorials

Check out our examples directory for more usage examples, including:

  • Basic usage
  • Different prompt types
  • Comparing providers
  • Advanced prompt templates
  • Prompt optimization
  • JSON output validation
  • Mixture of Agents

Project Status

gollm is actively maintained and under continuous development. We welcome contributions and feedback from the community.

Philosophy

gollm is built on a philosophy of pragmatic minimalism and forward-thinking simplicity:

  1. Build what's necessary: We add features and capabilities as they become needed, avoiding speculative development.

  2. Simplicity first: Every addition should be as simple and straightforward as possible while fulfilling its purpose.

  3. Future-compatible: While we don't build for hypothetical future needs, we carefully consider how current changes might impact future development.

  4. Readability counts: Code should be clear and self-explanatory. If it's not, we improve it or document it thoroughly.

  5. Modular design: Each component should do one thing well, allowing for easy understanding, testing, and future modification.

This philosophy guides our development process and helps us maintain a lean, efficient, and adaptable codebase. We encourage all contributors to keep these principles in mind when proposing changes or new features.

Contributing

We welcome contributions that align with our philosophy! Whether you're fixing a bug, improving documentation, or proposing new features, your efforts are appreciated.

To get started:

  1. Familiarize yourself with our philosophy and development approach.
  2. Check out our CONTRIBUTING.md for guidelines on how to contribute.
  3. Look through our issues for something that interests you.
  4. Fork the repository, make your changes, and submit a pull request.

Remember, the best contributions are those that adhere to our philosophy of pragmatic minimalism and readability. We encourage you to include examples and clear comments with your code.

If you have any questions or ideas, feel free to open an issue or join our community chat. We're always excited to discuss how we can improve gollm while staying true to our core principles.

Thank you for helping make gollm better!

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var QuestionAnswerTemplate = NewPromptTemplate(
	"QuestionAnswer",
	"Answer the given question",
	"Answer the following question:\n\n{{.Question}}",
	WithPromptOptions(
		WithDirectives("Provide a clear and concise answer"),
		WithOutput("Answer:"),
	),
)

Functions

func AnalyzeComparisonResults 

func AnalyzeComparisonResults[T any](results []ComparisonResult[T]) string

func ChainOfThought 

func ChainOfThought(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

ChainOfThought performs a chain of thought reasoning

func CleanResponse 

func CleanResponse(response string) string

CleanResponse removes markdown code block syntax and trims the JSON response

func ExtractStructuredData 

func ExtractStructuredData[T any](ctx context.Context, l LLM, text string, opts ...PromptOption) (*T, error)

ExtractStructuredData extracts structured data from text based on a given struct type

func GenerateJSONSchema 

func GenerateJSONSchema(v interface{}) ([]byte, error)

GenerateJSONSchema generates a JSON schema for the given struct

func OptimizePrompt added in v0.0.93 

func OptimizePrompt(ctx context.Context, llm LLM, config OptimizationConfig) (optimizedPrompt string, response string, err error)

OptimizePrompt optimizes the given prompt and generates a response

func QuestionAnswer 

func QuestionAnswer(ctx context.Context, l LLM, question string, opts ...PromptOption) (string, error)

QuestionAnswer performs question answering

func Summarize 

func Summarize(ctx context.Context, l LLM, text string, opts ...PromptOption) (string, error)

Summarize performs text summarization

func Validate 

func Validate(s interface{}) error

Validate checks if the given struct is valid according to its validation rules

Types

type BatchPromptOptimizer added in v0.0.93 

type BatchPromptOptimizer struct {
	LLM     LLM
	Verbose bool
}

BatchPromptOptimizer handles batch optimization of prompts

func NewBatchPromptOptimizer added in v0.0.93 

func NewBatchPromptOptimizer(llm LLM) *BatchPromptOptimizer

NewBatchPromptOptimizer creates a new BatchPromptOptimizer

func (*BatchPromptOptimizer) OptimizePrompts added in v0.0.93 

func (bpo *BatchPromptOptimizer) OptimizePrompts(ctx context.Context, examples []PromptExample) []OptimizationResult

OptimizePrompts optimizes a batch of prompts concurrently

type ComparisonResult 

type ComparisonResult[T any] struct {
	Provider string
	Model    string
	Response string
	Data     T
	Error    error
	Attempts int
}

func CompareModels 

func CompareModels[T any](ctx context.Context, prompt string, validateFunc ValidateFunc[T], configs ...*Config) ([]ComparisonResult[T], error)

CompareModels now uses a generic type T and accepts a validation function

type Config 

type Config struct {
	Provider         string
	Model            string
	OllamaEndpoint   string
	Temperature      float64
	MaxTokens        int
	TopP             float64
	FrequencyPenalty float64
	PresencePenalty  float64
	Timeout          time.Duration
	MaxRetries       int
	RetryDelay       time.Duration
	APIKey           string
	DebugLevel       LogLevel
	MemoryOption     *MemoryOption
	Seed             *int
	MinP             *float64
	RepeatPenalty    *float64
	RepeatLastN      *int
	Mirostat         *int
	MirostatEta      *float64
	MirostatTau      *float64
	TfsZ             *float64
}

Config holds the configuration for the LLM

func LoadConfig 

func LoadConfig() (*Config, error)

LoadConfig loads the configuration from environment variables

type ConfigOption 

type ConfigOption func(*Config)

ConfigOption is a function type for modifying Config

func SetAPIKey 

func SetAPIKey(apiKey string) ConfigOption

SetAPIKey sets the API key for the current provider in the Config

func SetDebugLevel 

func SetDebugLevel(level LogLevel) ConfigOption

SetDebugLevel sets the debug level in the Config

func SetFrequencyPenalty added in v0.0.95 

func SetFrequencyPenalty(penalty float64) ConfigOption

SetFrequencyPenalty sets the frequency penalty. It penalizes new tokens based on their frequency in the text so far. Higher values decrease the model's likelihood to repeat the same line verbatim.

func SetMaxRetries 

func SetMaxRetries(maxRetries int) ConfigOption

SetMaxRetries sets the maximum number of retries in the Config

func SetMaxTokens 

func SetMaxTokens(maxTokens int) ConfigOption

SetMaxTokens sets the max tokens in the Config

func SetMemory added in v0.0.8 

func SetMemory(maxTokens int) ConfigOption

SetMemory sets the memory option for the LLM

func SetMinP added in v0.0.95 

func SetMinP(minP float64) ConfigOption

SetMinP sets the minimum probability for a token to be considered, relative to the probability of the most likely token. It aims to ensure a balance of quality and variety. (Default: 0.05)

func SetMirostat added in v0.0.95 

func SetMirostat(mode int) ConfigOption

SetMirostat enables Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)

func SetMirostatEta added in v0.0.95 

func SetMirostatEta(eta float64) ConfigOption

SetMirostatEta influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)

func SetMirostatTau added in v0.0.95 

func SetMirostatTau(tau float64) ConfigOption

SetMirostatTau controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)

func SetModel 

func SetModel(model string) ConfigOption

SetModel sets the model in the Config

func SetOllamaEndpoint 

func SetOllamaEndpoint(endpoint string) ConfigOption

SetOllamaEndpoint sets the Ollama endpoint in the Config

func SetPresencePenalty added in v0.0.95 

func SetPresencePenalty(penalty float64) ConfigOption

SetPresencePenalty sets the presence penalty. It penalizes new tokens based on whether they appear in the text so far. Higher values increase the model's likelihood to talk about new topics.

func SetProvider 

func SetProvider(provider string) ConfigOption

SetProvider sets the provider in the Config

func SetRepeatLastN added in v0.0.95 

func SetRepeatLastN(n int) ConfigOption

SetRepeatLastN sets how far back the model should look to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)

func SetRepeatPenalty added in v0.0.95 

func SetRepeatPenalty(penalty float64) ConfigOption

SetRepeatPenalty sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)

func SetRetryDelay 

func SetRetryDelay(retryDelay time.Duration) ConfigOption

SetRetryDelay sets the delay between retries in the Config

func SetSeed added in v0.0.95 

func SetSeed(seed int) ConfigOption

SetSeed sets the random number seed for generation. Setting this to a specific number will make the model generate the same text for the same prompt. (Default: 0)

func SetTemperature 

func SetTemperature(temperature float64) ConfigOption

SetTemperature sets the temperature in the Config

func SetTfsZ added in v0.0.95 

func SetTfsZ(z float64) ConfigOption

SetTfsZ sets the value for tail free sampling, which is used to reduce the impact of less probable tokens. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. (Default: 1)

func SetTimeout 

func SetTimeout(timeout time.Duration) ConfigOption

SetTimeout sets the timeout in the Config

func SetTopP added in v0.0.95 

func SetTopP(topP float64) ConfigOption

SetTopP sets the top-p value. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)

type Example 

type Example struct {
	Content string `json:"content"`
}

type GenerateOption 

type GenerateOption func(*generateConfig)

GenerateOption is a function type for configuring generate options

func WithJSONSchemaValidation 

func WithJSONSchemaValidation() GenerateOption

WithJSONSchemaValidation returns a GenerateOption that enables JSON schema validation

type IterationCallback added in v0.0.9 

type IterationCallback func(iteration int, entry OptimizationEntry)

IterationCallback is a function type for the iteration callback

type LLM 

type LLM interface {
	// Generate produces a response given a context, prompt, and optional generate options
	Generate(ctx context.Context, prompt *Prompt, opts ...GenerateOption) (string, error)

	// SetOption sets an option for the LLM
	SetOption(key string, value interface{})

	// GetPromptJSONSchema returns the JSON schema for the prompt
	GetPromptJSONSchema(opts ...SchemaOption) ([]byte, error)

	// GetProvider returns the provider of the LLM
	GetProvider() string

	// GetModel returns the model of the LLM
	GetModel() string

	// UpdateDebugLevel updates the debug level of the LLM
	UpdateDebugLevel(level LogLevel)

	// Debug logs a debug message with optional key-value pairs
	Debug(msg string, keysAndValues ...interface{})

	// GetDebugLevel returns the current debug level of the LLM
	GetDebugLevel() LogLevel

	SetOllamaEndpoint(endpoint string) error
}

LLM is the interface that wraps the basic LLM operations

func NewLLM 

func NewLLM(opts ...ConfigOption) (LLM, error)

NewLLM creates a new LLM instance, potentially with memory if the option is set

type LogLevel 

type LogLevel int

LogLevel represents the level of logging 

const (
	LogLevelOff LogLevel = iota
	LogLevelError
	LogLevelWarn
	LogLevelInfo
	LogLevelDebug
)

type MOA added in v0.0.94 

type MOA struct {
	Config     MOAConfig
	Layers     []MOALayer
	Aggregator LLM
}

MOA represents the Mixture of Agents

func NewMOA added in v0.0.94 

func NewMOA(config MOAConfig, aggregatorConfig []ConfigOption) (*MOA, error)

NewMOA creates a new Mixture of Agents instance

func (*MOA) Generate added in v0.0.94 

func (moa *MOA) Generate(ctx context.Context, input string) (string, error)

Generate processes the input through the Mixture of Agents and returns the final output

type MOAConfig added in v0.0.94 

type MOAConfig struct {
	Iterations   int
	Models       [][]ConfigOption // Each inner slice represents a model's configuration
	MaxParallel  int              // Maximum number of parallel requests per layer (0 for sequential processing)
	AgentTimeout time.Duration    // Timeout for each agent's request (0 for no timeout)
}

MOAConfig represents the configuration for the Mixture of Agents

type MOALayer added in v0.0.94 

type MOALayer struct {
	Models []LLM
}

MOALayer represents a single layer in the Mixture of Agents

type MemoryOption added in v0.0.8 

type MemoryOption struct {
	MaxTokens int
}

type Metric added in v0.0.9 

type Metric = llm.Metric

Type aliases to bridge public and internal types

type OptimizationConfig added in v0.0.93 

type OptimizationConfig struct {
	Prompt       string
	Description  string
	Metrics      []Metric
	RatingSystem string
	Threshold    float64
	MaxRetries   int
	RetryDelay   time.Duration
}

OptimizationConfig holds the configuration for prompt optimization

func DefaultOptimizationConfig added in v0.0.93 

func DefaultOptimizationConfig() OptimizationConfig

DefaultOptimizationConfig returns a default configuration for prompt optimization

type OptimizationEntry added in v0.0.9 

type OptimizationEntry = llm.OptimizationEntry

type OptimizationResult added in v0.0.93 

type OptimizationResult struct {
	Name             string
	OriginalPrompt   string
	OptimizedPrompt  string
	GeneratedContent string
	Error            error
}

OptimizationResult represents the result of a single prompt optimization

type OptimizerOption added in v0.0.9 

type OptimizerOption func(*PromptOptimizer)

OptimizerOption is a function type for configuring the PromptOptimizer

func WithCustomMetrics added in v0.0.9 

func WithCustomMetrics(metrics ...Metric) OptimizerOption

WithCustomMetrics adds custom metrics to the PromptOptimizer

func WithIterationCallback added in v0.0.9 

func WithIterationCallback(callback IterationCallback) OptimizerOption

Modify the existing WithIterationCallback function

func WithIterations added in v0.0.91 

func WithIterations(iterations int) OptimizerOption

func WithMaxRetries added in v0.0.9 

func WithMaxRetries(maxRetries int) OptimizerOption

WithMaxRetries sets the maximum number of retries for the PromptOptimizer

func WithMemorySize added in v0.0.91 

func WithMemorySize(size int) OptimizerOption

WithMemorySize sets the memory size for the PromptOptimizer

func WithOptimizationGoal added in v0.0.9 

func WithOptimizationGoal(goal string) OptimizerOption

WithOptimizationGoal sets the optimization goal for the PromptOptimizer

func WithRatingSystem added in v0.0.9 

func WithRatingSystem(system string) OptimizerOption

WithRatingSystem sets the rating system for the PromptOptimizer

func WithRetryDelay added in v0.0.9 

func WithRetryDelay(delay time.Duration) OptimizerOption

WithRetryDelay sets the delay between retries for the PromptOptimizer

func WithThreshold added in v0.0.9 

func WithThreshold(threshold float64) OptimizerOption

WithThreshold sets the threshold for the PromptOptimizer

func WithVerbose added in v0.0.92 

func WithVerbose() OptimizerOption

type Prompt 

type Prompt struct {
	Input      string   `json:"input" jsonschema:"required,description=The main input text for the LLM" validate:"required"`
	Output     string   `json:"output,omitempty" jsonschema:"description=Specification for the expected output format"`
	Directives []string `json:"directives,omitempty" jsonschema:"description=List of directives to guide the LLM"`
	Context    string   `json:"context,omitempty" jsonschema:"description=Additional context for the LLM"`
	MaxLength  int      `` /* 128-byte string literal not displayed */
	Examples   []string `json:"examples,omitempty" jsonschema:"description=List of examples to guide the LLM"`
}

Prompt represents a structured prompt for an LLM

func NewPrompt 

func NewPrompt(input string, opts ...PromptOption) *Prompt

NewPrompt creates a new Prompt with the given input and applies any provided options

func (*Prompt) Apply 

func (p *Prompt) Apply(opts ...PromptOption)

Apply applies the given options to the Prompt

func (*Prompt) GenerateJSONSchema 

func (p *Prompt) GenerateJSONSchema(opts ...SchemaOption) ([]byte, error)

GenerateJSONSchema returns the JSON Schema for the Prompt struct

func (*Prompt) String 

func (p *Prompt) String() string

String returns the formatted prompt as a string

func (*Prompt) Validate 

func (p *Prompt) Validate() error

Validate checks if the Prompt is valid according to its validation rules

type PromptExample added in v0.0.93 

type PromptExample struct {
	Name        string
	Prompt      string
	Description string
	Metrics     []Metric
	Threshold   float64
}

PromptExample represents a single prompt to be optimized

type PromptOptimizer added in v0.0.9 

type PromptOptimizer struct {
	// contains filtered or unexported fields
}

PromptOptimizer is the public interface for the prompt optimization system

func NewPromptOptimizer added in v0.0.9 

func NewPromptOptimizer(l LLM, initialPrompt string, taskDesc string, opts ...OptimizerOption) *PromptOptimizer

NewPromptOptimizer creates a new PromptOptimizer

func (*PromptOptimizer) GetOptimizationHistory added in v0.0.9 

func (po *PromptOptimizer) GetOptimizationHistory() []OptimizationEntry

GetOptimizationHistory returns the history of optimization attempts

func (*PromptOptimizer) OptimizePrompt added in v0.0.9 

func (po *PromptOptimizer) OptimizePrompt(ctx context.Context) (string, error)

OptimizePrompt runs the optimization process

type PromptOption 

type PromptOption func(*Prompt)

PromptOption is a function type that modifies a Prompt

func WithContext 

func WithContext(context string) PromptOption

WithContext adds context to the Prompt

func WithDirectives 

func WithDirectives(directives ...string) PromptOption

WithDirectives adds directives to the Prompt

func WithExamples 

func WithExamples(examples ...string) PromptOption

WithExamples adds examples to the Prompt

func WithMaxLength 

func WithMaxLength(length int) PromptOption

WithMaxLength sets the maximum length for the output

func WithOutput 

func WithOutput(output string) PromptOption

WithOutput adds an output specification to the Prompt

type PromptTemplate 

type PromptTemplate struct {
	Name        string
	Description string
	Template    string
	Options     []PromptOption
}

PromptTemplate represents a template for generating prompts

func NewPromptTemplate 

func NewPromptTemplate(name, description, template string, opts ...PromptTemplateOption) *PromptTemplate

NewPromptTemplate creates a new PromptTemplate with the given name, description, and template

func (*PromptTemplate) Execute 

func (pt *PromptTemplate) Execute(data map[string]interface{}) (*Prompt, error)

Execute generates a Prompt from the PromptTemplate with the given data

type PromptTemplateOption 

type PromptTemplateOption func(*PromptTemplate)

PromptTemplateOption is a function type that modifies a PromptTemplate

func WithPromptOptions 

func WithPromptOptions(options ...PromptOption) PromptTemplateOption

WithPromptOptions adds PromptOptions to the PromptTemplate

type SchemaOption 

type SchemaOption func(*jsonschema.Reflector)

SchemaOption is a function type for schema generation options

func WithExpandedStruct 

func WithExpandedStruct(expanded bool) SchemaOption

WithExpandedStruct sets the ExpandedStruct option for schema generation

type ValidateFunc 

type ValidateFunc[T any] func(T) error

ValidateFunc is a function type for custom validation logic

Source Files

View all Source files

Directories

Path Synopsis
cmd
goal command
internal
llm

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.