proxy

Documentation

Overview

Package proxy provides a ChatModel adapter that forwards LLM calls to a remote proxy server. The wire format ("frames") is bandwidth-optimized: frames carry only deltas, and the client reconstructs the full streaming message incrementally.

Index

• type Frame
• type FrameType
• type Model
  • func New(fn StreamFn) *Model
  • func (p *Model) Generate(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, ...) (*agentcore.LLMResponse, error)
  • func (p *Model) GenerateStream(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, ...) (<-chan agentcore.StreamEvent, error)
  • func (p *Model) SupportsTools() bool
• type StreamFn

Types

type Frame

type Frame struct {
	Type       FrameType            `json:"type"`
	Delta      string               `json:"delta,omitempty"`
	ToolCallID string               `json:"tool_call_id,omitempty"`
	ToolName   string               `json:"tool_name,omitempty"`
	StopReason agentcore.StopReason `json:"stop_reason,omitempty"`
	Usage      *agentcore.Usage     `json:"usage,omitempty"`
	Err        error                `json:"-"`
}

Frame is a single bandwidth-optimized event from a remote proxy server.

type FrameType

type FrameType string

FrameType identifies a proxy stream frame.

const (
	FrameTextDelta     FrameType = "text_delta"
	FrameThinkingDelta FrameType = "thinking_delta"
	FrameToolCallStart FrameType = "toolcall_start"
	FrameToolCallDelta FrameType = "toolcall_delta"
	FrameDone          FrameType = "done"
	FrameError         FrameType = "error"
)

type Model

type Model struct {
	// contains filtered or unexported fields
}

Model implements agentcore.ChatModel by forwarding to a proxy stream function. It reconstructs streaming events from incoming frames.

Usage:

m := proxy.New(myStreamFn)
agent := agentcore.NewAgent(agentcore.WithModel(m))

func New

func New(fn StreamFn) *Model

New creates a Model that delegates to the given proxy stream function.

func (*Model) Generate

func (p *Model) Generate(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (*agentcore.LLMResponse, error)

Generate collects the full streamed response synchronously.

func (*Model) GenerateStream

func (p *Model) GenerateStream(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (<-chan agentcore.StreamEvent, error)

GenerateStream converts proxy frames into standard StreamEvents.

func (*Model) SupportsTools

func (p *Model) SupportsTools() bool

SupportsTools reports that the proxy can handle tool calls.

type StreamFn

type StreamFn func(ctx context.Context, req *agentcore.LLMRequest) (<-chan Frame, error)

StreamFn makes an LLM call through a remote proxy and returns a channel of bandwidth-optimized frames.