llmutil

Documentation

Index

• func Complete(ctx context.Context, client gochatcore.Client, prompt string) (string, error)
• func CompleteStream(ctx context.Context, client gochatcore.Client, prompt string) (<-chan string, error)
• type CircuitBreakerClient
  • func NewCircuitBreakerClient(client gochatcore.Client, breaker *circuitbreaker.CircuitBreaker) *CircuitBreakerClient
  • func (c *CircuitBreakerClient) Chat(ctx context.Context, messages []gochatcore.Message, opts ...gochatcore.Option) (*gochatcore.Response, error)
  • func (c *CircuitBreakerClient) ChatStream(ctx context.Context, messages []gochatcore.Message, opts ...gochatcore.Option) (*gochatcore.Stream, error)

Functions

func Complete

func Complete(ctx context.Context, client gochatcore.Client, prompt string) (string, error)

Complete is a helper to adapt gochat's Chat method to the simpler Complete signature

func CompleteStream

func CompleteStream(ctx context.Context, client gochatcore.Client, prompt string) (<-chan string, error)

CompleteStream is a helper to adapt gochat's ChatStream to return a channel of strings

Types

type CircuitBreakerClient

type CircuitBreakerClient struct {
	// contains filtered or unexported fields
}

CircuitBreakerClient wraps an LLM Client with circuit breaker protection

func NewCircuitBreakerClient

func NewCircuitBreakerClient(client gochatcore.Client, breaker *circuitbreaker.CircuitBreaker) *CircuitBreakerClient

NewCircuitBreakerClient creates a new circuit breaker client

func (*CircuitBreakerClient) Chat

func (c *CircuitBreakerClient) Chat(ctx context.Context, messages []gochatcore.Message, opts ...gochatcore.Option) (*gochatcore.Response, error)

Chat generates a completion with circuit breaker protection

func (*CircuitBreakerClient) ChatStream

func (c *CircuitBreakerClient) ChatStream(ctx context.Context, messages []gochatcore.Message, opts ...gochatcore.Option) (*gochatcore.Stream, error)

ChatStream generates a completion stream with circuit breaker protection