promptengine

README

Prompt Engine

A production-oriented prompt compiler for Go that helps you assemble bounded, low-allocation prompts for chatbots, agents, and LLM-backed services.

What this library currently provides

The package is built around a small but useful set of primitives:

• a bounded sliding window of role/content messages
• character and token budget enforcement
• optional summarization when a conversation grows past the configured budget
• pluggable token counting through a tokenizer interface
• prompt rendering with {{var}} and {{messages}} placeholders
• context-aware rendering for cancelable request flows
• provider execution hooks for sending compiled prompts to an external model
• pooled message blocks and reusable byte buffers to keep allocation pressure low

Quick start

package main

import (
    "fmt"

    pe "github.com/mastershashi/go-patterns/pkg/prompt-engine"
)

func main() {
    engine := pe.NewEngine(pe.Config{
        MaxChars:    64 * 1024,
        MaxMessages: 64,
        MaxTokens:   8192,
    })

    _ = engine.AddSystem("You are a helpful assistant")
    _ = engine.AddUser("Translate this to {{language}}: {{text}}")

    buf := make([]byte, 0, 1024)
    rendered := engine.CompileTemplate(buf, "{{messages}}", map[string]string{
        "language": "English",
        "text":     "hello world",
    })

    fmt.Println(string(rendered))
}

Real-world use cases

This library is a good fit when you need a lightweight prompt assembly layer inside a larger AI application:

• chat assistants with long-running conversations
• agent runtimes that must keep context bounded while preserving recent turns
• RAG systems that combine retrieved documents with the latest chat history
• tool-using workflows where prompts are assembled repeatedly under tight latency budgets
• backend services that need predictable memory behavior and low GC churn

Example: summarize older context before sending to a provider

type wordTokenizer struct{}
func (wordTokenizer) CountTokens(text string) int { return len(strings.Fields(text)) }

type compactSummarizer struct{}
func (compactSummarizer) Summarize(messages []pe.Message) string {
    return "summary: earlier context retained"
}

engine := pe.NewEngine(pe.Config{
    MaxChars:    64 * 1024,
    MaxMessages: 16,
    MaxTokens:   96,
    Tokenizer:   wordTokenizer{},
    Summarizer:  compactSummarizer{},
})

_ = engine.AddSystem("You are a travel planner")
_ = engine.AddUser("Plan a weekend trip to Berlin")
_ = engine.AddAssistant("Focus on museums and food")

prompt := string(engine.CompileTemplate(nil, "{{messages}}", nil))
_ = prompt

Limitations and non-goals

This package is intentionally focused on prompt assembly rather than being a full AI framework.

Current limitations include:

• it does not persist conversation state or provide durable memory
• it does not implement retries, streaming, or provider-specific request handling
• token counting is approximate unless a custom tokenizer is supplied
• summarization is a hook you provide; the package does not ship a learned summarizer
• it is best suited for the “prompt construction” layer, not full orchestration, tool routing, or agent planning

Design highlights

• fixed-capacity ring window with no array-copy churn during eviction
• pooled message blocks to reduce GC pressure
• atomic character and token budget tracking
• reusable output buffers for hot-path rendering
• context-aware compile flow for cancelable execution
• provider hooks for integrating with external model execution backends

Performance guidance

For the best results:

• reuse the same output byte slice across requests
• keep budgets aligned with your model and workload
• use larger budgets for long-running sessions and smaller budgets for latency-sensitive services
• treat this package as the prompt assembly layer, not as a complete agent framework

Documentation

Overview

// Package promptengine provides a low-allocation prompt runtime for building // bounded, context-aware LLM prompts in Go. // // The package is designed for high-throughput services that need predictable // memory behavior, reusable buffers, and a compact sliding-window memory model // for system/user/assistant prompt history.

Index

• type Config
• type Engine
  • func NewEngine(cfg Config) *Engine
  • func (e *Engine) AddAssistant(content string) error
  • func (e *Engine) AddSystem(content string) error
  • func (e *Engine) AddUser(content string) error
  • func (e *Engine) CharCount() int
  • func (e *Engine) Compile(dst []byte, vars map[string]string) []byte
  • func (e *Engine) CompileTemplate(dst []byte, template string, vars map[string]string) []byte
  • func (e *Engine) CompileTemplateWithContext(ctx context.Context, dst []byte, template string, vars map[string]string) []byte
  • func (e *Engine) ExecuteWithProvider(ctx context.Context, provider Provider, template string, ...) (string, error)
  • func (e *Engine) Len() int
  • func (e *Engine) Push(role, content string) error
  • func (e *Engine) Reset()
  • func (e *Engine) Snapshot() []Message
  • func (e *Engine) TokenCount() int
• type Message
• type Provider
• type ProviderFunc
  • func (f ProviderFunc) Generate(ctx context.Context, prompt string) (string, error)
• type Summarizer
• type Tokenizer
• type WhitespaceTokenizer
  • func (WhitespaceTokenizer) CountTokens(text string) int

Types

type Config

type Config struct {
	MaxChars    int
	MaxMessages int
	MaxTokens   int
	Tokenizer   Tokenizer
	Summarizer  Summarizer
}

Config controls the sliding window budgets used by the engine.

MaxChars limits the total accumulated character length of the retained messages. MaxMessages limits the number of retained messages. MaxTokens provides an additional token-based guard for prompt size.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine compiles prompt templates with a reusable message window.

It is designed for low-allocation prompt construction in high-throughput Go services. The engine keeps a bounded sliding window of messages and prunes old entries when configured budgets are exceeded.

func NewEngine

func NewEngine(cfg Config) *Engine

NewEngine creates a prompt engine with bounded memory usage.

If Config values are left at zero, the engine uses conservative production defaults intended for a reusable prompt runtime.

func (*Engine) AddAssistant

func (e *Engine) AddAssistant(content string) error

AddAssistant appends an assistant message using the helper API.

func (*Engine) AddSystem

func (e *Engine) AddSystem(content string) error

AddSystem appends a system message using the helper API.

func (*Engine) AddUser

func (e *Engine) AddUser(content string) error

AddUser appends a user message using the helper API.

func (*Engine) CharCount

func (e *Engine) CharCount() int

CharCount returns the current character budget usage.

func (*Engine) Compile

func (e *Engine) Compile(dst []byte, vars map[string]string) []byte

Compile renders the current message window into dst using the provided vars.

func (*Engine) CompileTemplate

func (e *Engine) CompileTemplate(dst []byte, template string, vars map[string]string) []byte

CompileTemplate renders the current message window into dst using the supplied vars.

The function supports `{{var}}` replacements and a special `{{messages}}` placeholder. Callers should reuse a byte slice across requests when they want to minimize allocations on the hot path.

func (*Engine) CompileTemplateWithContext

func (e *Engine) CompileTemplateWithContext(ctx context.Context, dst []byte, template string, vars map[string]string) []byte

CompileTemplateWithContext renders the current message window into dst using the supplied vars and context.

If the context is canceled before rendering begins, the method returns an empty output slice immediately.

func (*Engine) ExecuteWithProvider

func (e *Engine) ExecuteWithProvider(ctx context.Context, provider Provider, template string, vars map[string]string) (string, error)

ExecuteWithProvider renders the current prompt and sends it to the supplied provider.

func (*Engine) Len

func (e *Engine) Len() int

Len returns the number of active messages in the sliding window.

func (*Engine) Push

func (e *Engine) Push(role, content string) error

Push stores a new role/content pair in the sliding window.

The engine normalizes the role and prunes the oldest messages if the configured character or token budgets would be exceeded. Empty content is ignored so callers can safely add blank messages.

func (*Engine) Reset

func (e *Engine) Reset()

Reset clears the sliding window and returns pooled message blocks.

It resets the internal window state and returns the retained blocks to the pool so future use can reuse them without additional allocations.

func (*Engine) Snapshot

func (e *Engine) Snapshot() []Message

Snapshot returns a copy of the active messages in oldest-to-newest order.

The returned slice preserves the current order of the message window, from oldest to newest.

func (*Engine) TokenCount

func (e *Engine) TokenCount() int

TokenCount returns the current estimated token budget usage.

type Message

type Message struct {
	Role    string
	Content string
}

Message represents a single role/content block in the prompt window.

type Provider

type Provider interface {
	Generate(ctx context.Context, prompt string) (string, error)
}

Provider executes a compiled prompt against an external model provider.

type ProviderFunc

type ProviderFunc func(ctx context.Context, prompt string) (string, error)

ProviderFunc adapts a function to the Provider interface.

func (ProviderFunc) Generate

func (f ProviderFunc) Generate(ctx context.Context, prompt string) (string, error)

Generate executes the provider function.

type Summarizer

type Summarizer interface {
	Summarize(messages []Message) string
}

Summarizer converts a set of messages into a compact summary string.

type Tokenizer

type Tokenizer interface {
	CountTokens(text string) int
}

Tokenizer counts tokens for a given input string.

type WhitespaceTokenizer

type WhitespaceTokenizer struct{}

WhitespaceTokenizer is a simple tokenizer based on whitespace-delimited tokens.

func (WhitespaceTokenizer) CountTokens

func (WhitespaceTokenizer) CountTokens(text string) int

CountTokens counts whitespace-delimited tokens.