ork

README

 ██████╗ ██████╗ ██╗  ██╗
██╔═══██╗██╔══██╗██║ ██╔╝
██║   ██║██████╔╝█████╔╝
██║   ██║██╔══██╗██╔═██╗
╚██████╔╝██║  ██║██║  ██╗
 ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝

Ork is a tiny and deterministic orchestration runtime for LLMs.

Notice: ork is currently being used as an orchestration runtime for a conversational voice and text platform. It has been extracted from the internal monorepo and open-sourced for external use.

Installation

go get codeberg.org/ju4n97/ork

Usage

See examples in the samples directory.

Core concepts

Blueprints & flows

A Blueprint is a collection of registered capabilities your agent has. A Flow is a specific capability (which acts as a "Tool" to the LLM). When an LLM asks to use a tool, Ork executes the corresponding Flow.

Nodes

Flows are made of nodes. A node does one specific task.

• node.HTTP: Makes API calls, templates payloads, handles retries, and dumps JSON responses into your state.
• node.Message: Compiles Go templates injected with your variables and emits text.
• node.End: Forcefully terminates a conversation early based on logic.

State management (Input, Args, Vars)

• Messages: Read-only conversation history.
• Args: Tool arguments generated by the LLM (e.g., extracting "location": "Tokyo" from the user prompt).
• Vars: Your application's internal state or "blackboard" (e.g., API keys, database results, user profiles). Nodes can read from and write to this state dynamically.

Contributing

As this was recently open-sourced from an internal monorepo, feel free to open issues for bugs or feature requests.

License

MIT.

Documentation

Index

• Variables
• type Blueprint
  • func NewBlueprint(flows ...*Flow) (*Blueprint, error)
• type Context
  • func (c *Context) Emit(content string) error
  • func (c *Context) Halt(reason string)
• type Event
• type EventChunk
• type EventComplete
• type EventContent
• type EventError
• type EventFlowEnd
• type EventFlowStart
• type EventNodeEnd
• type EventNodeStart
• type Flow
  • func NewFlow(name string, opts ...FlowOption) (*Flow, error)
• type FlowOption
  • func WithDescription(desc string) FlowOption
  • func WithEdge(from, to string) FlowOption
  • func WithNode(id string, n Node) FlowOption
  • func WithSchema(schema map[string]any) FlowOption
• type GenerateRequest
• type GenerateResponse
• type Hooks
• type Input
• type Interceptor
• type LLM
• type Message
• type MessageRole
• type Node
• type NodeInitializer
• type Option
  • func WithHooks(h Hooks) Option
  • func WithInterceptor(i Interceptor) Option
  • func WithMaxRounds(n int) Option
• type Runner
  • func NewRunner(llm LLM, opts ...Option) *Runner
  • func (r *Runner) Run(ctx context.Context, blueprint *Blueprint, input Input) iter.Seq[Event]
  • func (r *Runner) RunFlow(ctx context.Context, flow *Flow, input Input) iter.Seq[Event]
• type Tool
• type ToolCall

Variables

var (
	// ErrMaxRoundsExceeded is returned when a run exceeds the configured
	// maximum number of LLM round-trips without producing a final response.
	ErrMaxRoundsExceeded = errors.New("ork: max rounds exceeded")

	// ErrCycleDetected is returned by NewFlow when the node graph contains a cycle.
	ErrCycleDetected = errors.New("ork: cycle detected in node graph")

	// ErrDuplicateFlow is returned by NewBlueprint when two flows share the same name.
	ErrDuplicateFlow = errors.New("ork: duplicate flow name")

	// ErrUnknownFlow is returned when the LLM invokes a flow not present in the blueprint.
	ErrUnknownFlow = errors.New("ork: unknown flow")
)

Types

type Blueprint

type Blueprint struct {
	// contains filtered or unexported fields
}

Blueprint groups one or more flows. The LLM uses each flow's name and description to decide which to invoke for a given conversation turn. All flows are compiled and validated at construction time.

func NewBlueprint

func NewBlueprint(flows ...*Flow) (*Blueprint, error)

NewBlueprint creates a Blueprint from the provided flows. Returns an error if any flow has graph issues (cycles, missing nodes) or if two flows share the same name.

type Context

type Context struct {
	// Ctx is the execution context for cancellations and timeouts.
	Ctx context.Context
	// Args contains trigger arguments provided by the user or the LLM (Read-only).
	Args map[string]any
	// Vars is the shared blackboard seeded by Input.Vars and mutated by nodes (Read/Write).
	Vars map[string]any
	// contains filtered or unexported fields
}

Context provides read-write state and actions for a single node execution.

func (*Context) Emit

func (c *Context) Emit(content string) error

Emit sends content immediately to the event channel for real-time streaming.

func (*Context) Halt

func (c *Context) Halt(reason string)

Halt terminates the flow and conversation early.

type Event

type Event interface {
	// contains filtered or unexported methods
}

Event is the interface implemented by all runtime events emitted by Run and RunFlow. Use a type switch to handle specific event types.

type EventChunk

type EventChunk struct {
	Chunk string
}

EventChunk is emitted for each token chunk streamed from the LLM.

type EventComplete

type EventComplete struct {
	// Content is the LLM's final text response, if the run ended with
	// a plain generation rather than a flow termination.
	Content string

	// Outcome is set when the run ended via node.End.
	Outcome string

	// Vars contains the final mutated blackboard state.
	Vars map[string]any
}

EventComplete is emitted when the run finishes successfully. StoreWrites is the merged result of all NodeOutput.StoreWrites produced during the run.

type EventContent

type EventContent struct {
	Content string
}

EventContent is emitted when a node produces static rendered text, such as a resolved message template.

type EventError

type EventError struct {
	Err error
}

EventError is emitted when an unrecoverable error occurs. The channel closes immediately after this event.

type EventFlowEnd

type EventFlowEnd struct {
	FlowName   string
	ToolCallID string         // correlates with EventFlowStart
	Result     map[string]any // the vars produced by the flow
}

EventFlowEnd is emitted when a flow finishes executing successfully, before its result is fed back to the LLM.

type EventFlowStart

type EventFlowStart struct {
	FlowName   string
	ToolCallID string // correlates with EventFlowEnd
	Args       map[string]any
}

EventFlowStart is emitted when the LLM selects a flow and its execution begins.

type EventNodeEnd

type EventNodeEnd struct {
	NodeID string
}

EventNodeEnd is emitted immediately after a node finishes without error.

type EventNodeStart

type EventNodeStart struct {
	NodeID string
}

EventNodeStart is emitted immediately before a node begins executing.

type Flow

type Flow struct {
	// contains filtered or unexported fields
}

Flow is a compiled, highly-optimized array of nodes in topological order.

func NewFlow

func NewFlow(name string, opts ...FlowOption) (*Flow, error)

NewFlow creates a new Flow with the provided options.

type FlowOption

type FlowOption func(*flowBuilder) error

FlowOption configures a Flow.

func WithDescription

func WithDescription(desc string) FlowOption

WithDescription sets the description of the flow.

func WithEdge

func WithEdge(from, to string) FlowOption

WithEdge adds an edge to the flow.

func WithNode

func WithNode(id string, n Node) FlowOption

WithNode adds a node to the flow.

func WithSchema

func WithSchema(schema map[string]any) FlowOption

WithSchema sets the schema of the flow.

type GenerateRequest

type GenerateRequest struct {
	// Messages is the full conversation window.
	Messages []Message

	// Tools is the set of flows exposed to the LLM as callable functions.
	// Ork constructs this from the blueprint automatically.
	Tools []Tool

	// OnChunk is called for each streamed token chunk. Nil means no streaming.
	// Returning an error cancels the generation.
	OnChunk func(chunk string) error
}

GenerateRequest is the input to a single LLM generation call.

type GenerateResponse

type GenerateResponse struct {
	// Message is the assistant's response always populated.
	// Set Message.ToolCalls when the model invokes one or more flows.
	Message Message
}

GenerateResponse is the output of a single LLM generation call.

type Hooks

type Hooks struct {
	// BeforeNode fires immediately before a node begins executing.
	BeforeNode func(ctx *Context, nodeID string)

	// AfterNode fires immediately after a node finishes without error.
	AfterNode func(ctx *Context, nodeID string)

	// OnError fires when a node returns an error.
	OnError func(ctx *Context, nodeID string, err error)
}

Hooks fire around every node execution. They must not block.

type Input

type Input struct {
	// Messages is the full conversation history including the current turn.
	// Ork treats this as read-only and never appends or modifies the slice.
	Messages []Message

	// Args provides manual trigger arguments for RunFlow.
	// Ignored by Run (the LLM generates them instead).
	Args map[string]any

	// Vars seeds the flow's blackboard. Use this to pass application state
	// like API keys, user IDs, or database state.
	Vars map[string]any
}

Input is the data provided by the caller for a single run.

type Interceptor

type Interceptor interface {
	// Before is called with the full message window before each LLM call.
	// The returned slice replaces the original.
	Before(ctx context.Context, messages []Message) ([]Message, error)

	// After is called with the LLM response before Ork processes it.
	// The returned message replaces the original.
	After(ctx context.Context, response Message) (Message, error)
}

Interceptor wraps every LLM generation call. It can be used for PII redaction, content filtering, prompt enrichment, or policy enforcement. Returning an error from either method aborts the run.

type LLM

type LLM interface {
	// Generate sends a request to the model and returns its response.
	// If req.OnChunk is non-nil, it is called with each token as it streams.
	Generate(ctx context.Context, req *GenerateRequest) (*GenerateResponse, error)
}

LLM is the interface Ork requires to interact with a language model. Implement this to integrate any provider.

type Message

type Message struct {
	Role    MessageRole
	Content string

	// ToolCalls is set on assistant messages when the LLM invokes flows.
	ToolCalls []ToolCall

	// ToolCallID links a tool result message to its originating call.
	ToolCallID string
}

Message is a single entry in a conversation.

type MessageRole

type MessageRole string

MessageRole is the author of a message.

const (
	MessageUser      MessageRole = "user"
	MessageAssistant MessageRole = "assistant"
	MessageTool      MessageRole = "tool"
	MessageSystem    MessageRole = "system"
)

type Node

type Node interface {
	// Execute performs the node's work. It can mutate ctx.Vars and ctx.Writes directly.
	Execute(orkCtx *Context) error
}

Node is the interface every executable unit must implement.

type NodeInitializer

type NodeInitializer interface {
	// Initialize sets up the node for execution.
	Initialize() error
}

NodeInitializer is used to set up a node for execution.

type Option

type Option func(*Runner)

Option configures a Runner.

func WithHooks

func WithHooks(h Hooks) Option

WithHooks configures a Runner with the provided hooks.

func WithInterceptor

func WithInterceptor(i Interceptor) Option

WithInterceptor configures a Runner with the provided interceptor.

func WithMaxRounds

func WithMaxRounds(n int) Option

WithMaxRounds configures a Runner with the provided maxRounds.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner executes blueprints and flows. It holds execution infrastructure and is safe for concurrent use. Create one per application, not per request.

func NewRunner

func NewRunner(llm LLM, opts ...Option) *Runner

NewRunner creates a Runner with the provided options.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, blueprint *Blueprint, input Input) iter.Seq[Event]

Run executes a blueprint returning an iterator. It executes synchronously and yields events to the caller, guaranteeing zero leaked goroutines.

func (*Runner) RunFlow

func (r *Runner) RunFlow(ctx context.Context, flow *Flow, input Input) iter.Seq[Event]

RunFlow executes a single flow directly.

type Tool

type Tool struct {
	Name        string
	Description string
	// Schema is a JSON Schema object describing the tool's input parameters.
	Schema map[string]any
}

Tool describes a flow as a callable function to the LLM.

type ToolCall

type ToolCall struct {
	// ID is an opaque identifier used to correlate tool results.
	ID   string
	Name string
	// Arguments are the tool's input parameters populated by the LLM.
	Arguments map[string]any
}

ToolCall is a single tool invocation returned by the LLM.