contextwindow

package module
v0.1.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 25, 2025 License: Apache-2.0 Imports: 19 Imported by: 0

README

ContextWindow: a low-level Go agentic LLM session library

GoDoc

ContextWindow is a straightforward library for managing conversations with LLMs. It supports tool calls, token usage tracking, summary-based compression (with an arbitrary summary model), and automatically persists conversations in SQLite.

The impetus for this library is multi-context agents. Under the hood of an LLM agent, a "context window" is simply a list of strings. LLM APIs generally don't have a notion of "sessions". As you build up a conversation, you simply send all prompts and responses to the LLM each time (we do support previous_response_id). This means there's no drama in having 2, 10, or 100 different "conversations" in an agent, and ContextWindow tries to make that straightforward to express.

Simple usage

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/superfly/contextwindow"
	"github.com/openai/openai-go/v2/shared"
)

func main() {
	ctx := context.Background()

        // reads OPENAI_API_KEY
        model, err := NewOpenAIResponsesModel(shared.ResponsesModel4o)
	if err != nil {
		log.Fatalf("Failed to create model: %v", err)
	}

	// Initialize the ContextWindow
	// This creates a new conversation history, backed by an in-memory SQLite DB.
	// For persistence, provide a file path: contextwindow.New(model, summarizerModel, "/path/to/your.db")
	cw, err := contextwindow.New(nil, model, "")
	if err != nil {
		log.Fatalf("Failed to create context window: %v", err)
	}
	defer cw.Close()

        if err := cw.AddPrompt(ctx, "how's the weather over there?")
	if err := cw.AddPrompt(ctx, prompt); err != nil {
		log.Fatalf("Failed to add prompt: %v", err)
	}

	// Call the model to get a response
	response, err := cw.CallModel(ctx)
	if err != nil {
		log.Fatalf("Failed to call model: %v", err)
	}
	fmt.Printf("<- Model: %s\n", response)

Tool calling

An LLM tool or function call is just a set of special JSON blobs the LLM is trained to parse and emit instead of English text. In this library, they're simply function calls:

lsTool := contextwindow.NewTool("list_files", `
  This tool lists files in the specified directory.
`).AddStringParameter("directory", "Directory to list", true)

cw.AddTool(lsTool, contextwindow.ToolRunnerFunc(func context.Context,
                                                args json.RawMessage) (string, error) {
  var treq struct {
    Dir string `json:"directory"`
  }
  json.Unmarshal(args, &treq)
  // actually run ls, or pretend to
  return "here\nare\nsome\nfiles.exe\n", nil
})

Arguments to function calls are passed in JSON messages; tool responses are passed as string return values.

Attach different tools to different ContextWindow instances; that's kind of the point. If one ContextWindow does something sensitive (like reading a database) and another reads untrusted data (like incoming email addresses), check carefully before relaying information from the untrusted side to the trusted side (for instance: instruct the untrusted side to provide responses in a JSON message you can parse in your code before handing off).

Context Management and Stats

For building table views or managing multiple conversations, you can efficiently get statistics for any context without expensive operations:

// Get all contexts
contexts, err := cw.ListContexts()
if err != nil {
    log.Fatalf("Failed to list contexts: %v", err)
}

// Get efficient stats for each context (no table scans)
for _, context := range contexts {
    stats, err := cw.GetContextStats(context)
    if err != nil {
        log.Printf("Failed to get stats for %s: %v", context.Name, err)
        continue
    }
    
    fmt.Printf("Context: %s\n", context.Name)
    fmt.Printf("  Created: %s\n", context.StartTime.Format("2006-01-02 15:04"))
    fmt.Printf("  Live tokens: %d\n", stats.LiveTokens)
    fmt.Printf("  Records: %d total, %d live\n", stats.TotalRecords, stats.LiveRecords)
    if stats.LastActivity != nil {
        fmt.Printf("  Last activity: %s\n", stats.LastActivity.Format("2006-01-02 15:04"))
    } else {
        fmt.Printf("  Last activity: never\n")
    }
}

The GetContextStats method uses a single aggregation query with existing indexes, making it efficient even with many contexts and large conversation histories.

Maturity

This is alpha-quality code. Happy to get feedback or PRs or whatever. Publishing this more as a statement than as an endorsement of the code itself. :)

Documentation

Overview

Package contextwindow implements sqlite-backed context windows for LLM sessions, with opt-in LLM summarization for compaction, token count tracking, and a simple tool call abstraction.

Abbreviated usage 

          model, err := NewOpenAIResponsesModel(shared.ResponsesModel4o)
          if err != nil {
            log.Fatalf("Failed to create model: %v", err)
          }

	     cw, err := contextwindow.New(model, nil, "")
	     if err != nil {
            log.Fatalf("Failed to create context window: %v", err)
	     }
	     defer cw.Close()

	     if err := cw.AddPrompt(ctx, "how's the weather?"); err != nil {
            log.Fatalf("Failed to add prompt: %v", err)
          }

	     response, err := cw.CallModel(ctx)
          if err != nil {
            log.Fatalf("Failed to call model: %v", err)
	     }

	     fmt.Printf("response: %s\n", response)

System prompts

Use ContextWindow.SetSystemPrompt to provide a system prompt for each conversation.

Tool calling

Instruct LLMs to call tools locally with ContextWindow.AddTool (and NewTool). 

	    lsTool := contextwindow.NewTool("list_files", `
	        This tool lists files in the specified directory.
	    `).AddStringParameter("directory", "Directory to list", true)

	    cw.AddTool(lsTool, contextwindow.ToolRunnerFunc(func context.Context,
	                                                    args json.RawMessage) (string, error) {
	    var treq struct {
           Dir string `json:"directory"`
         }
	    json.Unmarshal(args, &treq)
	      // actually run ls, or pretend to
	      return "here\nare\nsome\nfiles.exe\n", nil
	    })

You can selectively enable and disable tools with ContextWindow.CallModelWithOpts.

Tool calls are very sensitive to the descriptions provided of the tool and arguments (the example up there is way too simple). Treat descriptions like part of the system prompt; tell the agent what to do.

Summarization

Models have context token limits (we use github.com/peterheb/gotoken/cl100kbase to track usage); both input and output tokens count towards the limit.

You can provide a summarizer model to automatically compact your context window: 

	    summarizerModel, err := openai.New(apiKey, "gpt-3.5-turbo")
	    if err != nil {
           log.Fatalf("Failed to create summarizer: %v", err)
	    }

	    cw, err := contextwindow.New(model, summarizerModel, "")

And then "compress" your context with [ContextWindow.SummarizeLiveContent].

Under the hood

A context window is just a list of strings, representing the history of a "conversation" with an LLM. The "live" strings in a conversation will be fed to the LLM on every call; we retain the "unalive" strings in records so that tool calls can fetch them later.

LLM conversations are stored in SQLite. If you don't care about persistant storage for your context, just specify ":memory:" as your database path.

Thread Safety

ContextWindow write operations (AddPrompt, SwitchContext, SetMaxTokens, etc.) require external coordination when used concurrently. However, you can use ContextWindow.Reader() to get a thread-safe read-only view: 

reader := cw.Reader()
go updateUI(reader)        // safe for concurrent use
go updateMetrics(reader)   // safe for concurrent use

// Meanwhile, main thread can safely modify state:
cw.SwitchContext("new-context")
cw.SetMaxTokens(8192)

ContextReader provides access to read operations like LiveRecords(), TokenUsage(), and context querying, all of which are safe for concurrent use.

Index

Constants

View Source 
const (
	ResponsesModelGPT5     shared.ResponsesModel = "gpt-5-2025-08-07"
	ResponsesModelGPT5Mini shared.ResponsesModel = "gpt-5-mini-2025-08-07"
	ResponsesModel4o       shared.ResponsesModel = "gpt-4o-2024-08-06"
	ResponsesModelO4Mini   shared.ResponsesModel = "o4-mini-2025-04-16"
)

Variables

This section is empty.

Functions

func CloneContext added in v0.1.8 

func CloneContext(db *sql.DB, sourceName, destName string) error

CloneContext creates a copy of the specified source context with a new name.

func DeleteContext 

func DeleteContext(db *sql.DB, contextID string) error

DeleteContext removes a context and all its records by ID.

func DeleteContextByName 

func DeleteContextByName(db *sql.DB, name string) error

DeleteContextByName removes a context and all its records by name.

func ExportContextJSON 

func ExportContextJSON(db *sql.DB, contextID string) ([]byte, error)

ExportContextJSON exports a context as JSON by ID.

func ExportContextJSONByName 

func ExportContextJSONByName(db *sql.DB, name string) ([]byte, error)

ExportContextJSONByName exports a context as JSON by name.

func HasContextTool 

func HasContextTool(db *sql.DB, contextID, toolName string) (bool, error)

HasContextTool checks if a specific tool is available in a context.

func InitializeSchema 

func InitializeSchema(db *sql.DB) error

InitializeSchema ensures the contexts and records tables and indexes exist. Also handles migrations by adding new columns to existing tables.

func ListContextToolNames 

func ListContextToolNames(db *sql.DB, contextID string) ([]string, error)

ListContextToolNames returns just the tool names for a specific context.

func NewContextDB 

func NewContextDB(dbpath string) (*sql.DB, error)

NewContextDB opens a database to store context windows in (pass ":memory:" as the name for a transient database), and migrates its schema. You'll need to do this before creating context windows.

func RemoveContextTool 

func RemoveContextTool(db *sql.DB, contextID, toolName string) error

RemoveContextTool removes a tool from a specific context.

func SetContextServerSideThreading 

func SetContextServerSideThreading(db *sql.DB, contextID string, useServerSideThreading bool) error

SetContextServerSideThreading enables or disables server-side threading for a context.

func UpdateContextLastResponseID 

func UpdateContextLastResponseID(db *sql.DB, contextID, responseID string) error

UpdateContextLastResponseID updates the last response ID for a context.

Types

type CallModelOpts 

type CallModelOpts struct {
	DisableTools bool
}

CallModelOpts contains options for model calls.

type CallOptsCapable 

type CallOptsCapable interface {
	CallWithOpts(ctx context.Context, inputs []Record, opts CallModelOpts) (events []Record, tokensUsed int, err error)
	CallWithThreadingAndOpts(
		ctx context.Context,
		useServerSideThreading bool,
		lastResponseID *string,
		inputs []Record,
		opts CallModelOpts,
	) (events []Record, responseID *string, tokensUsed int, err error)
}

CallOptsCapable is an optional interface that models can implement to support call options like disabling tools.

type Context 

type Context struct {
	ID                     string    `json:"id"`
	Name                   string    `json:"name"`
	StartTime              time.Time `json:"start_time"`
	UseServerSideThreading bool      `json:"use_server_side_threading"`
	LastResponseID         *string   `json:"last_response_id,omitempty"`
}

Context represents a named context window with metadata.

func CreateContext 

func CreateContext(db *sql.DB, name string) (Context, error)

CreateContext creates a new context with the given name. Name must not be empty and must be unique.

func CreateContextWithThreading 

func CreateContextWithThreading(db *sql.DB, name string, useServerSideThreading bool) (Context, error)

CreateContextWithThreading creates a new context with threading mode specified.

func GetContext 

func GetContext(db *sql.DB, contextID string) (Context, error)

GetContext retrieves a context by ID.

func GetContextByName 

func GetContextByName(db *sql.DB, name string) (Context, error)

GetContextByName retrieves a context by name.

func ListContexts 

func ListContexts(db *sql.DB) ([]Context, error)

ListContexts returns all contexts ordered by start time.

type ContextExport 

type ContextExport struct {
	Context Context       `json:"context"`
	Records []Record      `json:"records"`
	Tools   []ContextTool `json:"tools"`
}

ContextExport represents a complete context with all its records.

func ExportContext 

func ExportContext(db *sql.DB, contextID string) (ContextExport, error)

ExportContext extracts a complete context with all its records by ID.

func ExportContextByName 

func ExportContextByName(db *sql.DB, name string) (ContextExport, error)

ExportContextByName extracts a complete context with all its records by name.

type ContextReader added in v0.1.6 

type ContextReader struct {
	// contains filtered or unexported fields
}

ContextReader provides thread-safe read access to context window data. It's a thin proxy that forwards calls to safe read methods on the underlying ContextWindow.

All methods on ContextReader are safe for concurrent use by multiple goroutines, even while the original ContextWindow is being modified.

func (*ContextReader) ExportContext added in v0.1.6 

func (cr *ContextReader) ExportContext(name string) (ContextExport, error)

ExportContext extracts a complete context with all its records.

func (*ContextReader) ExportContextJSON added in v0.1.6 

func (cr *ContextReader) ExportContextJSON(name string) ([]byte, error)

ExportContextJSON exports a context as JSON bytes.

func (*ContextReader) GetContext added in v0.1.6 

func (cr *ContextReader) GetContext(name string) (Context, error)

GetContext retrieves context metadata by name.

func (*ContextReader) GetContextStats added in v0.1.6 

func (cr *ContextReader) GetContextStats(context Context) (ContextStats, error)

GetContextStats retrieves efficient statistics for any context.

func (*ContextReader) GetCurrentContext added in v0.1.6 

func (cr *ContextReader) GetCurrentContext() string

GetCurrentContext returns the name of the current context.

func (*ContextReader) GetCurrentContextInfo added in v0.1.6 

func (cr *ContextReader) GetCurrentContextInfo() (Context, error)

GetCurrentContextInfo returns the current context metadata.

func (*ContextReader) ListContexts added in v0.1.6 

func (cr *ContextReader) ListContexts() ([]Context, error)

ListContexts returns all available context windows.

func (*ContextReader) LiveRecords added in v0.1.6 

func (cr *ContextReader) LiveRecords() ([]Record, error)

LiveRecords retrieves all "live" records from the context.

func (*ContextReader) LiveTokens added in v0.1.6 

func (cr *ContextReader) LiveTokens() (int, error)

LiveTokens estimates the total number of tokens in all "live" messages.

func (*ContextReader) MaxTokens added in v0.1.6 

func (cr *ContextReader) MaxTokens() int

MaxTokens returns the maximum number of tokens allowed in the context window.

func (*ContextReader) TokenUsage added in v0.1.6 

func (cr *ContextReader) TokenUsage() (TokenUsage, error)

TokenUsage returns current token usage metrics optimized for UI display.

func (*ContextReader) TotalTokens added in v0.1.6 

func (cr *ContextReader) TotalTokens() int

TotalTokens returns the cumulative token count across all model calls.

type ContextStats added in v0.1.4 

type ContextStats struct {
	LiveTokens   int        // tokens in live records
	TotalRecords int        // total number of records
	LiveRecords  int        // number of live records
	LastActivity *time.Time // timestamp of most recent record, nil if no records
}

ContextStats provides metrics for a specific context without expensive computation.

type ContextTool 

type ContextTool struct {
	ID        int64     `json:"id"`
	ContextID string    `json:"context_id"`
	ToolName  string    `json:"tool_name"`
	CreatedAt time.Time `json:"created_at"`
}

ContextTool represents a tool available in a specific context.

func AddContextTool 

func AddContextTool(db *sql.DB, contextID, toolName string) (ContextTool, error)

AddContextTool adds a tool name to a specific context.

func ListContextTools 

func ListContextTools(db *sql.DB, contextID string) ([]ContextTool, error)

ListContextTools returns all tools for a specific context.

type ContextWindow 

type ContextWindow struct {
	// contains filtered or unexported fields
}

ContextWindow holds our LLM context manager state.

func NewContextWindow 

func NewContextWindow(
	db *sql.DB,
	model Model,
	contextName string,
) (*ContextWindow, error)

NewContextWindow initializes a ContextWindow with an existing database connection and a specific context. If the context doesn't exist, it will be created. The caller is responsible for closing the database. You can provide an empty context name, in which case we'll generate a UUID for it.

func NewContextWindowWithThreading 

func NewContextWindowWithThreading(
	db *sql.DB,
	model Model,
	contextName string,
	useServerSideThreading bool,
) (*ContextWindow, error)

NewContextWindowWithThreading creates a context window with specified threading mode.

func (*ContextWindow) AcceptSummary 

func (cw *ContextWindow) AcceptSummary(
	result *SummaryResult,
) error

func (*ContextWindow) AcceptSummaryInContext 

func (cw *ContextWindow) AcceptSummaryInContext(
	result *SummaryResult,
	contextName string,
) error

func (*ContextWindow) AddMiddleware 

func (cw *ContextWindow) AddMiddleware(m Middleware)

AddMiddleware registers middleware to hook into tool call events.

func (*ContextWindow) AddPrompt 

func (cw *ContextWindow) AddPrompt(text string) error

AddPrompt logs a user prompt to the current context.

func (*ContextWindow) AddTool 

func (cw *ContextWindow) AddTool(tool *ToolBuilder, runner ToolRunner) error

func (*ContextWindow) AddToolCall 

func (cw *ContextWindow) AddToolCall(name, args string) error

AddToolCall logs a tool invocation to the current context.

func (*ContextWindow) AddToolFromJSON 

func (cw *ContextWindow) AddToolFromJSON(name string, jsonDefinition interface{}, runner ToolRunner) error

func (*ContextWindow) AddToolOutput 

func (cw *ContextWindow) AddToolOutput(output string) error

AddToolOutput logs a tool's output to the current context.

func (*ContextWindow) CallModel 

func (cw *ContextWindow) CallModel(ctx context.Context) (string, error)

CallModel drives an LLM. It composes live messages, invokes cw.model.Call, logs the response, updates token count, and triggers compaction.

func (*ContextWindow) CallModelWithOpts 

func (cw *ContextWindow) CallModelWithOpts(ctx context.Context, opts CallModelOpts) (string, error)

CallModelWithOpts drives an LLM with options. It composes live messages, invokes cw.model.Call, logs the response, updates token count, and triggers compaction.

func (*ContextWindow) Clone added in v0.1.7 

func (cw *ContextWindow) Clone(destName string) error

Clone creates a copy of the current context with a new name.

func (*ContextWindow) Close 

func (cw *ContextWindow) Close() error

Close closes the database connection. Only call this if you opened the database using NewContextWindow or NewContextWindowWithContext. If you used NewContextWindowWithDB, you should close the database yourself.

func (*ContextWindow) CreateContext 

func (cw *ContextWindow) CreateContext(name string) error

CreateContext creates a new named context window.

func (*ContextWindow) DeleteContext 

func (cw *ContextWindow) DeleteContext(name string) error

DeleteContext removes a context and all its records.

func (*ContextWindow) ExecuteTool 

func (cw *ContextWindow) ExecuteTool(ctx context.Context, name string, args json.RawMessage) (string, error)

ExecuteTool implements ToolExecutor interface.

func (*ContextWindow) ExportContext 

func (cw *ContextWindow) ExportContext(name string) (ContextExport, error)

ExportContext extracts a complete context with all its records.

func (*ContextWindow) ExportContextJSON 

func (cw *ContextWindow) ExportContextJSON(name string) ([]byte, error)

ExportContextJSON exports a context as JSON bytes.

func (*ContextWindow) GetContext 

func (cw *ContextWindow) GetContext(name string) (Context, error)

GetContext retrieves context metadata by name.

func (*ContextWindow) GetContextStats added in v0.1.4 

func (cw *ContextWindow) GetContextStats(context Context) (ContextStats, error)

GetContextStats retrieves efficient statistics for any context. All metrics are computed using database aggregations with existing indexes.

func (*ContextWindow) GetCurrentContext 

func (cw *ContextWindow) GetCurrentContext() string

func (*ContextWindow) GetCurrentContextInfo 

func (cw *ContextWindow) GetCurrentContextInfo() (Context, error)

GetCurrentContextInfo returns the current context metadata.

func (*ContextWindow) GetRegisteredTools 

func (cw *ContextWindow) GetRegisteredTools() []ToolDefinition

GetRegisteredTools returns all registered tool definitions.

func (*ContextWindow) GetTool 

func (cw *ContextWindow) GetTool(name string) (ToolRunner, bool)

GetTool retrieves a registered tool runner by name.

func (*ContextWindow) HasTool 

func (cw *ContextWindow) HasTool(name string) (bool, error)

HasTool checks if a tool name is available in this context.

func (*ContextWindow) IsServerSideThreadingEnabled 

func (cw *ContextWindow) IsServerSideThreadingEnabled() (bool, error)

IsServerSideThreadingEnabled returns whether server-side threading is enabled.

func (*ContextWindow) ListContexts 

func (cw *ContextWindow) ListContexts() ([]Context, error)

ListContexts returns all available context windows.

func (*ContextWindow) ListTools 

func (cw *ContextWindow) ListTools() ([]string, error)

ListTools returns the names of all tools available in this context.

func (*ContextWindow) LiveRecords 

func (cw *ContextWindow) LiveRecords() ([]Record, error)

LiveRecords retrieves all "live" records from the context. This is an important function, since this is usually what you want to call to get what's currently meaningful in your context --- it's what gets sent to the LLM.

func (*ContextWindow) LiveTokens 

func (cw *ContextWindow) LiveTokens() (int, error)

LiveTokens estimates the total number of tokens in all "live" messages in the context. Depending on your model, at some threshold of tokens you'll want either to summarize, or to start a new context window.

func (*ContextWindow) MaxTokens 

func (cw *ContextWindow) MaxTokens() int

MaxTokens returns the maximum number of tokens allowed in the context window.

func (*ContextWindow) Reader added in v0.1.6 

func (cw *ContextWindow) Reader() *ContextReader

Reader returns a thread-safe read-only view of this ContextWindow. The returned ContextReader is safe for concurrent use by multiple goroutines, even while this ContextWindow is being modified.

func (*ContextWindow) RegisterTool 

func (cw *ContextWindow) RegisterTool(name string, definition interface{}, runner ToolRunner) error

RegisterTool registers a tool with this ContextWindow instance and stores the tool name as a hint in the database.

func (*ContextWindow) RejectSummary 

func (cw *ContextWindow) RejectSummary(
	result *SummaryResult,
)

func (*ContextWindow) SetMaxTokens 

func (cw *ContextWindow) SetMaxTokens(max int)

SetMaxTokens updates the maximum number of tokens allowed in the context window.

func (*ContextWindow) SetRecordLiveStateByRange added in v0.1.8 

func (cw *ContextWindow) SetRecordLiveStateByRange(startIndex, endIndex int, live bool) error

SetRecordLiveStateByRange updates the live status of records in the specified range. Indices are based on the current LiveRecords() slice, with both start and end inclusive. This allows selective marking of context elements as active (live=true) or inactive (live=false) based on their position in the conversation.

Examples: 

SetRecordLiveStateByRange(2, 4, false) // marks records at indices 2, 3, 4 as dead
SetRecordLiveStateByRange(5, 5, false) // marks only record at index 5 as dead

func (*ContextWindow) SetServerSideThreading 

func (cw *ContextWindow) SetServerSideThreading(enabled bool) error

SetServerSideThreading enables or disables server-side threading for the current context.

func (*ContextWindow) SetSummarizer 

func (cw *ContextWindow) SetSummarizer(summarizer Summarizer)

func (*ContextWindow) SetSummarizerPrompt 

func (cw *ContextWindow) SetSummarizerPrompt(prompt string)

func (*ContextWindow) SetSystemPrompt 

func (cw *ContextWindow) SetSystemPrompt(text string) error

SetSystemPrompt sets the system prompt for the current context.

func (*ContextWindow) SummarizeLiveContext 

func (cw *ContextWindow) SummarizeLiveContext(ctx context.Context) (*SummaryResult, error)

func (*ContextWindow) SummarizeLiveContextInContext 

func (cw *ContextWindow) SummarizeLiveContextInContext(
	ctx context.Context,
	contextName string,
) (*SummaryResult, error)

func (*ContextWindow) SwitchContext added in v0.1.3 

func (cw *ContextWindow) SwitchContext(name string) error

SwitchContext switches the ContextWindow to operate on a different existing context.

func (*ContextWindow) TokenUsage 

func (cw *ContextWindow) TokenUsage() (TokenUsage, error)

TokenUsage returns current token usage metrics optimized for UI display.

func (*ContextWindow) TotalTokens 

func (cw *ContextWindow) TotalTokens() int

type Metrics 

type Metrics struct {
	// contains filtered or unexported fields
}

Metrics tracks token usage across model calls.

func (*Metrics) Add 

func (m *Metrics) Add(n int)

func (*Metrics) Total 

func (m *Metrics) Total() int

type Middleware 

type Middleware interface {
	// OnToolCall is invoked when a tool is about to be called.
	OnToolCall(ctx context.Context, name, args string)
	// OnToolResult is invoked when a tool call completes.
	OnToolResult(ctx context.Context, name, result string, err error)
}

Middleware allows hooking into tool call lifecycle events.

type MiddlewareCapable 

type MiddlewareCapable interface {
	SetMiddleware([]Middleware)
}

MiddlewareCapable is an optional interface that models can implement to receive middleware updates.

type Model 

type Model interface {
	// Call sends messages, returns model reply and token usage.
	Call(ctx context.Context, inputs []Record) (events []Record, tokensUsed int, err error)
}

Model abstracts out an LLM client library.

type OpenAIModel 

type OpenAIModel struct {
	// contains filtered or unexported fields
}

func NewOpenAIModel 

func NewOpenAIModel(model shared.ChatModel) (*OpenAIModel, error)

func (*OpenAIModel) Call 

func (o *OpenAIModel) Call(
	ctx context.Context,
	inputs []Record,
) ([]Record, int, error)

func (*OpenAIModel) CallWithOpts 

func (o *OpenAIModel) CallWithOpts(
	ctx context.Context,
	inputs []Record,
	opts CallModelOpts,
) ([]Record, int, error)

func (*OpenAIModel) CallWithThreading 

func (o *OpenAIModel) CallWithThreading(
	ctx context.Context,
	useServerSideThreading bool,
	lastResponseID *string,
	inputs []Record,
) ([]Record, *string, int, error)

CallWithThreading implements ServerSideThreadingCapable interface

func (*OpenAIModel) CallWithThreadingAndOpts 

func (o *OpenAIModel) CallWithThreadingAndOpts(
	ctx context.Context,
	useServerSideThreading bool,
	lastResponseID *string,
	inputs []Record,
	opts CallModelOpts,
) ([]Record, *string, int, error)

CallWithThreadingAndOpts implements CallOptsCapable interface

func (*OpenAIModel) MaxTokens 

func (o *OpenAIModel) MaxTokens() int

func (*OpenAIModel) SetMiddleware 

func (o *OpenAIModel) SetMiddleware(middleware []Middleware)

SetMiddleware sets the middleware for the OpenAI model

func (*OpenAIModel) SetToolExecutor 

func (o *OpenAIModel) SetToolExecutor(executor ToolExecutor)

SetToolExecutor sets the tool executor for the OpenAI model

type OpenAIResponsesModel 

type OpenAIResponsesModel struct {
	// contains filtered or unexported fields
}

func NewOpenAIResponsesModel 

func NewOpenAIResponsesModel(model shared.ResponsesModel) (*OpenAIResponsesModel, error)

func (*OpenAIResponsesModel) Call 

func (o *OpenAIResponsesModel) Call(
	ctx context.Context,
	inputs []Record,
) ([]Record, int, error)

func (*OpenAIResponsesModel) CallWithOpts 

func (o *OpenAIResponsesModel) CallWithOpts(
	ctx context.Context,
	inputs []Record,
	opts CallModelOpts,
) ([]Record, int, error)

func (*OpenAIResponsesModel) CallWithThreading 

func (o *OpenAIResponsesModel) CallWithThreading(
	ctx context.Context,
	useServerSideThreading bool,
	lastResponseID *string,
	inputs []Record,
) ([]Record, *string, int, error)

func (*OpenAIResponsesModel) CallWithThreadingAndOpts 

func (o *OpenAIResponsesModel) CallWithThreadingAndOpts(
	ctx context.Context,
	useServerSideThreading bool,
	lastResponseID *string,
	inputs []Record,
	opts CallModelOpts,
) ([]Record, *string, int, error)

func (*OpenAIResponsesModel) MaxTokens 

func (o *OpenAIResponsesModel) MaxTokens() int

func (*OpenAIResponsesModel) SetMiddleware 

func (o *OpenAIResponsesModel) SetMiddleware(middleware []Middleware)

func (*OpenAIResponsesModel) SetToolExecutor 

func (o *OpenAIResponsesModel) SetToolExecutor(executor ToolExecutor)

type Parameter 

type Parameter struct {
	Name        string
	Type        ParameterType
	Description string
	Required    bool
	Items       *Parameter
	Properties  map[string]*Parameter
}

type ParameterType 

type ParameterType string
const (
	ParameterTypeString  ParameterType = "string"
	ParameterTypeNumber  ParameterType = "number"
	ParameterTypeBoolean ParameterType = "boolean"
	ParameterTypeArray   ParameterType = "array"
	ParameterTypeObject  ParameterType = "object"
)

type Record 

type Record struct {
	ID         int64      `json:"id"`
	Timestamp  time.Time  `json:"timestamp"`
	Source     RecordType `json:"source"`
	Content    string     `json:"content"`
	Live       bool       `json:"live"`
	EstTokens  int        `json:"est_tokens"`
	ContextID  string     `json:"context_id"`
	ResponseID *string    `json:"response_id,omitempty"`
}

Record is one row in context history.

func InsertRecord 

func InsertRecord(
	db *sql.DB,
	contextID string,
	source RecordType,
	content string,
	live bool,
) (Record, error)

InsertRecord inserts a new record in the specified context.

func InsertRecordWithResponseID 

func InsertRecordWithResponseID(
	db *sql.DB,
	contextID string,
	source RecordType,
	content string,
	live bool,
	responseID *string,
) (Record, error)

InsertRecordWithResponseID inserts a new record with optional response ID.

func ListLiveRecords 

func ListLiveRecords(db *sql.DB, contextID string) ([]Record, error)

ListLiveRecords returns all live records in a context in timestamp order.

func ListRecordsInContext 

func ListRecordsInContext(db *sql.DB, contextID string) ([]Record, error)

ListRecordsInContext returns all records in a context in timestamp order.

type RecordType 

type RecordType int

RecordType distinguishes entry kinds. 

const (
	Prompt RecordType = iota
	ModelResp
	ToolCall
	ToolOutput
	SystemPrompt
)

type ServerSideThreadingCapable 

type ServerSideThreadingCapable interface {
	// CallWithThreading calls the model with optional server-side threading.
	CallWithThreading(
		ctx context.Context,
		useServerSideThreading bool,
		lastResponseID *string,
		inputs []Record,
	) (events []Record, responseID *string, tokensUsed int, err error)
}

ServerSideThreadingCapable is an optional interface for models that support server-side threading.

type Summarizer 

type Summarizer interface {
	Model
}

type SummaryResult 

type SummaryResult struct {
	Summary      string
	Replaced     []Record
	OrigCount    int
	SummaryCount int
}

type TokenReporter 

type TokenReporter interface {
	TotalTokens() int
	LiveTokens() (int, error)
	TokenUsage() (TokenUsage, error)
}

type TokenUsage 

type TokenUsage struct {
	Live    int     // tokens currently in context window
	Total   int     // cumulative tokens used across all calls
	Max     int     // maximum tokens allowed in context window
	Percent float64 // live/max as percentage (0.0-1.0)
}

TokenUsage provides a snapshot of current token usage for UI display.

type ToolBuilder 

type ToolBuilder struct {
	// contains filtered or unexported fields
}

func NewTool 

func NewTool(name, description string) *ToolBuilder

func (*ToolBuilder) AddArrayParameter 

func (tb *ToolBuilder) AddArrayParameter(name, description string, required bool, itemType ParameterType) *ToolBuilder

func (*ToolBuilder) AddBooleanParameter 

func (tb *ToolBuilder) AddBooleanParameter(name, description string, required bool) *ToolBuilder

func (*ToolBuilder) AddNumberParameter 

func (tb *ToolBuilder) AddNumberParameter(name, description string, required bool) *ToolBuilder

func (*ToolBuilder) AddObjectParameter 

func (tb *ToolBuilder) AddObjectParameter(name, description string, required bool, properties map[string]*Parameter) *ToolBuilder

func (*ToolBuilder) AddStringParameter 

func (tb *ToolBuilder) AddStringParameter(name, description string, required bool) *ToolBuilder

func (*ToolBuilder) ToOpenAI 

func (tb *ToolBuilder) ToOpenAI() openai.FunctionDefinitionParam

type ToolCapable 

type ToolCapable interface {
	SetToolExecutor(ToolExecutor)
}

ToolCapable is an optional interface that models can implement to receive a ToolExecutor for handling tool calls.

type ToolDefinition 

type ToolDefinition struct {
	Name       string      `json:"name"`
	Definition interface{} `json:"definition"` // Model-specific tool definition (e.g., OpenAI FunctionDefinitionParam)
}

ToolDefinition represents a tool that can be called by the model.

type ToolExecutor 

type ToolExecutor interface {
	ExecuteTool(ctx context.Context, name string, args json.RawMessage) (string, error)
	GetRegisteredTools() []ToolDefinition
}

ToolExecutor can execute tools by name and provide access to tool definitions.

type ToolRunner 

type ToolRunner interface {
	Run(ctx context.Context, args json.RawMessage) (string, error)
}

ToolRunner defines the interface for executing a tool.

type ToolRunnerFunc 

type ToolRunnerFunc func(ctx context.Context, args json.RawMessage) (string, error)

ToolRunnerFunc allows functions to implement ToolRunner.

func (ToolRunnerFunc) Run 

func (f ToolRunnerFunc) Run(ctx context.Context, args json.RawMessage) (string, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.