jpf

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2025 License: MIT Imports: 12 Imported by: 7

README

Go Report Card Go Ref

jpf: A Lightweight Framework for AI-Powered Applications

jpf is a Go library for building lightweight AI-powered applications. It provides essential building blocks, including model construction, embedding generation, and robust LLM interaction interfaces, enabling you to craft custom solutions without the bloat.

Features

  • Flexible Orchestration: Design iterative workflows for reasoning, task automation, or application backends.
  • Retry and Feedback Handling: Resilient mechanisms for retrying tasks and incorporating feedback into interactions.
  • Embedding Utilities: Generate and manipulate vector embeddings for tasks like similarity, search, or clustering.
  • Customizable Models: Seamlessly integrate LLMs, including reasoning chains and hybrid models.
  • Token Usage Tracking: Stay informed of API token consumption for cost-effective development.
  • Easy-to-use Caching: Reduce the calls made to models by composing a caching layer onto an existing model.

Installation

Install jpf in your Go project via:

go get github.com/JoshPattman/jpf

Usage

Example code is in the examples subdirectory. However, a brief overview of the components is as follows:

  • Model: An interface defining a model that can create a message given a set of other messages. This encompasses both normal and reasoning models. Models can also be wrapped by other models to achieve retry logic, hybrid reasoning, and more.
  • Function: An interface that defines a stateless typed function that performs one LLM call, and includes logic for formatting and parsing the text responses.
  • RetryFunction: An extension of the above, but including logic to generate feedback for the LLM upon a failed parse, allowing the LLM call to be run in a loop until valid.
  • Embedder: A string to vector embbedding interface.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Contributing

Contributions are welcome! Open an issue or submit a pull request on GitHub.

FAQ

  • Will streaming (token-by-token) ever be supported?
    • No. This framework is designed to be more of a back-end tool, and character streaming would add extra complexity that most applications of this package would not benefit from (in my opinion).
  • Are there any pre-built functions?
    • The aim of this package is to create the framework, not the functionality. If you have any ideas of useful functions, feel free to put them on an issue, and if enough arise, I can make a new repo for these.
  • Where are the agents?
    • I removed the agent interface recently as I think it was far too restrictuve. I would like to instead get the core building blocks ironed out before moving on to coming up with an agent interface.

Author

Developed by Josh Pattman. Learn more at GitHub.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrInvalidResponse = errors.New("llm produced an invalid response")
)

Functions

func CosineSimilarity 

func CosineSimilarity(vec1, vec2 []float64) (float64, error)

CosineSimilarity takes the cosine similarity between two vectors.

func HashMessages added in v0.6.0 

func HashMessages(msgs []Message) string

func LogWithJson added in v0.6.0 

func LogWithJson(lmp ModelLoggingInfo, dst io.Writer) error

func NewRawStringMessageEncoder added in v0.6.0 

func NewRawStringMessageEncoder(systemPrompt string) *rawStringMessageEncoder

NewRawStringMessageEncoder creates a MessageEncoder that encodes a system prompt and user input as raw string messages.

func NewRawStringResponseDecoder added in v0.6.0 

func NewRawStringResponseDecoder() *rawStringResponseDecoder

NewRawStringResponseDecoder creates a ResponseDecoder that returns the response as a raw string without modification.

Types

type CachedModelBuilder added in v0.6.0 

type CachedModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildCachedModel added in v0.6.0 

func BuildCachedModel(model Model, cache ModelResponseCache) *CachedModelBuilder

func (*CachedModelBuilder) Validate added in v0.6.0 

func (b *CachedModelBuilder) Validate() (Model, error)

type ConcurrentLimitedModelBuilder added in v0.6.0 

type ConcurrentLimitedModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildConcurrentLimitedModel added in v0.6.0 

func BuildConcurrentLimitedModel(model Model) *ConcurrentLimitedModelBuilder

Builds a model that has a maximum number of concurrent uses at once. The default number of uses is 1. There is no certainty about the order of calls (i.e. a later call made to this may be processed before an earlier one).

func (*ConcurrentLimitedModelBuilder) Validate added in v0.6.0 

func (m *ConcurrentLimitedModelBuilder) Validate() (Model, error)

func (*ConcurrentLimitedModelBuilder) WithUses added in v0.6.0 

func (m *ConcurrentLimitedModelBuilder) WithUses(n int) *ConcurrentLimitedModelBuilder

Sets the number on concurrent uses, must be >= 1.

type Embedder 

type Embedder interface {
	Embed(text string) ([]float64, error)
}

Embedder defines an object that is capable of embedding a string into a vector.

type FakeReasoningModelBuilder added in v0.5.0 

type FakeReasoningModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildFakeReasoningModel added in v0.5.0 

func BuildFakeReasoningModel(reasoner Model, answerer Model) *FakeReasoningModelBuilder

func (*FakeReasoningModelBuilder) Validate added in v0.5.0 

func (b *FakeReasoningModelBuilder) Validate() (Model, error)

func (*FakeReasoningModelBuilder) WithReasoningPrompt added in v0.5.0 

func (b *FakeReasoningModelBuilder) WithReasoningPrompt(prompt string) *FakeReasoningModelBuilder

type FeedbackGenerator added in v0.6.0 

type FeedbackGenerator interface {
	FormatFeedback(Message, error) string
}

A FeedbackGenerator can take an error and convert it to a pice of text feedback to send to the LLM.

func NewRawMessageFeedbackGenerator added in v0.6.0 

func NewRawMessageFeedbackGenerator() FeedbackGenerator

NewRawMessageFeedbackGenerator creates a FeedbackGenerator that formats feedback by returning the error message as a string.

type LoggingModelBuilder added in v0.6.0 

type LoggingModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildLoggingModel added in v0.6.0 

func BuildLoggingModel(dst io.Writer, model Model) *LoggingModelBuilder

func (*LoggingModelBuilder) Validate added in v0.6.0 

func (lmb *LoggingModelBuilder) Validate() (Model, error)

func (*LoggingModelBuilder) WithLogFunc added in v0.6.0 

func (lmb *LoggingModelBuilder) WithLogFunc(logFunc func(ModelLoggingInfo, io.Writer) error) *LoggingModelBuilder

type MapFunc added in v0.6.0 

type MapFunc[T, U any] interface {
	Call(T) (U, Usage, error)
}

func NewFeedbackMapFunc added in v0.6.0 

func NewFeedbackMapFunc[T, U any](
	enc MessageEncoder[T],
	pars ResponseDecoder[U],
	fed FeedbackGenerator,
	model Model,
	feedbackRole Role,
	maxRetries int,
) MapFunc[T, U]

Creates a map func that will keep adding to the conversation with feedback when errors are detected. It will only ever add to the conversation if the error returned from fed is a ErrInvalidResponse (using errors.Is)

func NewOneShotMapFunc added in v0.6.0 

func NewOneShotMapFunc[T, U any](enc MessageEncoder[T], pars ResponseDecoder[U], model Model) MapFunc[T, U]

type Message 

type Message struct {
	Role    Role
	Content string
}

Message defines a text message to/from an LLM.

type MessageEncoder added in v0.6.0 

type MessageEncoder[T any] interface {
	BuildInputMessages(T) ([]Message, error)
}

A MessageEncoder encodes a structured pice of data into a set of messages for an LLM.

type Model 

type Model interface {
	// Tokens specifies how many tokens are allowed to be sent.
	Tokens() (int, int)
	// Responds to a set of input messages, with a set of auxilliary messages and a final message.
	// There may be no auxilliary messages, or things like tool calls, function calls, and reasoning may go in the auxilliary messages,
	Respond([]Message) ([]Message, Message, Usage, error)
}

Model defines an interface to an LLM.

type ModelLoggingInfo added in v0.6.0 

type ModelLoggingInfo struct {
	// contains filtered or unexported fields
}

type ModelResponseCache added in v0.6.0 

type ModelResponseCache interface {
	GetCachedResponse([]Message) (bool, []Message, Message, error)
	SetCachedResponse(inputs []Message, aux []Message, out Message) error
}

func NewInMemoryCache added in v0.6.0 

func NewInMemoryCache() ModelResponseCache

type OpenAIEmbedderBuilder added in v0.5.0 

type OpenAIEmbedderBuilder struct {
	// contains filtered or unexported fields
}

func BuildOpenAIEmbedder added in v0.5.0 

func BuildOpenAIEmbedder(key, model string) *OpenAIEmbedderBuilder

func (*OpenAIEmbedderBuilder) Validate added in v0.5.0 

func (b *OpenAIEmbedderBuilder) Validate() (Embedder, error)

func (*OpenAIEmbedderBuilder) WithURL added in v0.5.0 

func (b *OpenAIEmbedderBuilder) WithURL(url string) *OpenAIEmbedderBuilder

type OpenAIModelBuilder added in v0.5.0 

type OpenAIModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildOpenAIModel added in v0.5.0 

func BuildOpenAIModel(key, modelName string, isReasoning bool) *OpenAIModelBuilder

func (*OpenAIModelBuilder) Validate added in v0.5.0 

func (b *OpenAIModelBuilder) Validate() (Model, error)

func (*OpenAIModelBuilder) WithHeader added in v0.5.0 

func (b *OpenAIModelBuilder) WithHeader(key, val string) *OpenAIModelBuilder

func (*OpenAIModelBuilder) WithReasoningEffort added in v0.5.0 

func (b *OpenAIModelBuilder) WithReasoningEffort(re ReasoningEffort) *OpenAIModelBuilder

func (*OpenAIModelBuilder) WithTemperature added in v0.5.0 

func (b *OpenAIModelBuilder) WithTemperature(temp float64) *OpenAIModelBuilder

func (*OpenAIModelBuilder) WithTokens added in v0.5.0 

func (b *OpenAIModelBuilder) WithTokens(input, output int) *OpenAIModelBuilder

func (*OpenAIModelBuilder) WithURL added in v0.5.0 

func (b *OpenAIModelBuilder) WithURL(url string) *OpenAIModelBuilder

type ReasoningEffort 

type ReasoningEffort uint8

ReasoningEffort defines how hard a reasoning model should think. 

const (
	LowReasoning ReasoningEffort = iota
	MediumReasoning
	HighReasoning
)

type ResponseDecoder added in v0.6.0 

type ResponseDecoder[T any] interface {
	ParseResponseText(string) (T, error)
}

a ResponseDecoder converts an LLM response into a structured peice of data. When the LLM responsee was invalid, it should return ErrInvalidResponse (or an error joined on that).

type RetryModelBuilder added in v0.5.0 

type RetryModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildRetryModel added in v0.5.0 

func BuildRetryModel(model Model) *RetryModelBuilder

func (*RetryModelBuilder) Validate added in v0.5.0 

func (b *RetryModelBuilder) Validate() (Model, error)

func (*RetryModelBuilder) WithDelay added in v0.5.0 

func (b *RetryModelBuilder) WithDelay(delay time.Duration) *RetryModelBuilder

func (*RetryModelBuilder) WithMaxRetries added in v0.5.0 

func (b *RetryModelBuilder) WithMaxRetries(maxRetries int) *RetryModelBuilder

type Role 

type Role uint8

Role is an enum specifying a role for a message. It is not 1:1 with openai roles (i.e. there is a reasoning role here). 

const (
	SystemRole Role = iota
	UserRole
	AssistantRole
	ReasoningRole
)

func (Role) String added in v0.6.0 

func (r Role) String() string

type SystemReasonModelBuilder added in v0.6.0 

type SystemReasonModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildSystemReasonModel added in v0.5.0 

func BuildSystemReasonModel(model Model) *SystemReasonModelBuilder

func (*SystemReasonModelBuilder) Validate added in v0.6.0 

func (b *SystemReasonModelBuilder) Validate() (Model, error)

func (*SystemReasonModelBuilder) WithPrefix added in v0.6.0 

func (b *SystemReasonModelBuilder) WithPrefix(prefix string) *SystemReasonModelBuilder

type Usage 

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage defines how many tokens were used when making calls to LLMs.

func (Usage) Add 

func (u Usage) Add(u2 Usage) Usage

type UsageCounter added in v0.6.0 

type UsageCounter struct {
	// contains filtered or unexported fields
}

Counts up the sum usage. Is completely concurrent-safe.

func NewUsageCounter added in v0.6.0 

func NewUsageCounter() *UsageCounter

Create a zero usage counter.

func (*UsageCounter) Add added in v0.6.0 

func (u *UsageCounter) Add(usage Usage)

Add the given usage to the counter.

func (*UsageCounter) Get added in v0.6.0 

func (u *UsageCounter) Get() Usage

Get the current usage in the counter.

type UsageCountingModelBuilder added in v0.6.0 

type UsageCountingModelBuilder struct {
	// contains filtered or unexported fields
}

func BuildUsageCountingModel added in v0.6.0 

func BuildUsageCountingModel(model Model, counter *UsageCounter) *UsageCountingModelBuilder

Builds a model that adds all usage of the child model to the counter.

func (*UsageCountingModelBuilder) Validate added in v0.6.0 

func (m *UsageCountingModelBuilder) Validate() (Model, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.