llmops

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 1, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package llmops provides a unified interface for LLM observability platforms. It abstracts common functionality across providers like Comet Opik, Arize Phoenix, Langfuse, Lunary, and others.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// Configuration errors
	ErrMissingAPIKey    = errors.New("llmops: missing API key")
	ErrMissingEndpoint  = errors.New("llmops: missing endpoint URL")
	ErrMissingWorkspace = errors.New("llmops: missing workspace")
	ErrMissingProject   = errors.New("llmops: missing project name")

	// State errors
	ErrTracingDisabled = errors.New("llmops: tracing is disabled")
	ErrNoActiveTrace   = errors.New("llmops: no active trace in context")
	ErrNoActiveSpan    = errors.New("llmops: no active span in context")
	ErrAlreadyEnded    = errors.New("llmops: trace or span already ended")

	// Not found errors
	ErrTraceNotFound      = errors.New("llmops: trace not found")
	ErrSpanNotFound       = errors.New("llmops: span not found")
	ErrDatasetNotFound    = errors.New("llmops: dataset not found")
	ErrPromptNotFound     = errors.New("llmops: prompt not found")
	ErrProjectNotFound    = errors.New("llmops: project not found")
	ErrExperimentNotFound = errors.New("llmops: experiment not found")

	// Validation errors
	ErrInvalidInput    = errors.New("llmops: invalid input")
	ErrInvalidSpanType = errors.New("llmops: invalid span type")
	ErrInvalidMetric   = errors.New("llmops: invalid metric")

	// Provider errors
	ErrProviderNotFound       = errors.New("llmops: provider not found")
	ErrNotImplemented         = errors.New("llmops: not implemented")
	ErrCapabilityNotSupported = errors.New("llmops: capability not supported")
)

Sentinel errors for common conditions.

Functions

func IsDisabled 

func IsDisabled(err error) bool

IsDisabled returns true if tracing is disabled.

func IsNotFound 

func IsNotFound(err error) bool

IsNotFound returns true if the error indicates a resource was not found.

func IsNotImplemented 

func IsNotImplemented(err error) bool

IsNotImplemented returns true if the operation is not implemented.

func IsRateLimited 

func IsRateLimited(err error) bool

IsRateLimited returns true if the error indicates rate limiting.

func IsUnauthorized 

func IsUnauthorized(err error) bool

IsUnauthorized returns true if the error indicates an authentication failure.

func Providers 

func Providers() []string

Providers returns a sorted list of the names of the registered providers.

func Register 

func Register(name string, factory ProviderFactory)

Register makes a provider available by the provided name. If Register is called twice with the same name or if factory is nil, it panics.

func RegisterInfo 

func RegisterInfo(info ProviderInfo)

RegisterInfo registers metadata about a provider. This is optional and used for discovery/documentation.

func Unregister 

func Unregister(name string)

Unregister removes a provider from the registry. This is primarily useful for testing.

func UnregisterAll 

func UnregisterAll()

UnregisterAll removes all providers from the registry. This is primarily useful for testing.

func WrapCapabilityNotSupported 

func WrapCapabilityNotSupported(provider string, cap Capability) error

WrapCapabilityNotSupported wraps a capability in a not supported error.

func WrapNotImplemented 

func WrapNotImplemented(provider, operation string) error

WrapNotImplemented wraps an operation name in a not implemented error.

Types

type APIError 

type APIError struct {
	StatusCode int
	Message    string
	Details    map[string]any
	Provider   string
	Err        error
}

APIError represents an error from a provider API.

func NewAPIError 

func NewAPIError(provider string, statusCode int, message string, err error) *APIError

NewAPIError creates a new API error.

func (*APIError) Error 

func (e *APIError) Error() string

func (*APIError) Unwrap 

func (e *APIError) Unwrap() error

type Annotation 

type Annotation struct {
	ID          string         `json:"id,omitempty"`
	SpanID      string         `json:"span_id,omitempty"`  // Set for span annotations
	TraceID     string         `json:"trace_id,omitempty"` // Set for trace annotations
	Name        string         `json:"name"`               // Annotation name/type
	Score       float64        `json:"score,omitempty"`    // Numeric score (0-1)
	Label       string         `json:"label,omitempty"`    // Categorical label
	Explanation string         `json:"explanation,omitempty"`
	Source      AnnotatorKind  `json:"source,omitempty"` // Who created: human, llm, code
	Metadata    map[string]any `json:"metadata,omitempty"`
	CreatedAt   time.Time      `json:"created_at,omitempty"`
	UpdatedAt   time.Time      `json:"updated_at,omitempty"`
}

Annotation represents an annotation on a span or trace.

type AnnotationManager 

type AnnotationManager interface {
	// CreateAnnotation creates an annotation on a span or trace.
	// Either SpanID or TraceID must be set in the annotation.
	CreateAnnotation(ctx context.Context, annotation Annotation) error

	// ListAnnotations lists annotations for spans or traces.
	// Provide either spanIDs or traceIDs (not both).
	ListAnnotations(ctx context.Context, opts ListAnnotationsOptions) ([]*Annotation, error)
}

AnnotationManager handles annotations on spans and traces.

type AnnotatorKind 

type AnnotatorKind string

AnnotatorKind indicates who created the annotation. 

const (
	AnnotatorKindHuman AnnotatorKind = "human"
	AnnotatorKindLLM   AnnotatorKind = "llm"
	AnnotatorKindCode  AnnotatorKind = "code"
)

type Capability 

type Capability string

Capability represents a specific feature a provider may support. 

const (
	CapabilityTracing      Capability = "tracing"
	CapabilityEvaluation   Capability = "evaluation"
	CapabilityPrompts      Capability = "prompts"
	CapabilityDatasets     Capability = "datasets"
	CapabilityExperiments  Capability = "experiments"
	CapabilityAnnotations  Capability = "annotations"
	CapabilityStreaming    Capability = "streaming"
	CapabilityDistributed  Capability = "distributed_tracing"
	CapabilityCostTracking Capability = "cost_tracking"
	CapabilityOTel         Capability = "opentelemetry"
)

type CapabilityChecker 

type CapabilityChecker interface {
	// HasCapability checks if the provider supports a given capability.
	HasCapability(cap Capability) bool

	// Capabilities returns all supported capabilities.
	Capabilities() []Capability
}

CapabilityChecker allows querying provider capabilities.

type ClientOption 

type ClientOption func(*ClientOptions)

ClientOption configures client/provider creation.

func WithAPIKey 

func WithAPIKey(key string) ClientOption

WithAPIKey sets the API key for authentication.

func WithDebug 

func WithDebug(debug bool) ClientOption

WithDebug enables debug logging.

func WithDisabled 

func WithDisabled(disabled bool) ClientOption

WithDisabled disables tracing (useful for testing).

func WithEndpoint 

func WithEndpoint(endpoint string) ClientOption

WithEndpoint sets the API endpoint URL.

func WithHTTPClient 

func WithHTTPClient(client *http.Client) ClientOption

WithHTTPClient sets a custom HTTP client.

func WithLogger 

func WithLogger(logger *slog.Logger) ClientOption

WithLogger sets an slog.Logger for local trace event logging. When set, trace and span events are logged to this logger in addition to being sent to the observability platform.

func WithProjectName 

func WithProjectName(project string) ClientOption

WithProjectName sets the default project name.

func WithTimeout 

func WithTimeout(timeout time.Duration) ClientOption

WithTimeout sets the request timeout.

func WithWorkspace 

func WithWorkspace(workspace string) ClientOption

WithWorkspace sets the workspace name.

type ClientOptions 

type ClientOptions struct {
	APIKey      string //nolint:gosec // G117: APIKey is intentionally stored for provider authentication
	Endpoint    string
	Workspace   string
	ProjectName string
	HTTPClient  *http.Client
	Timeout     time.Duration
	Disabled    bool
	Debug       bool
	Logger      *slog.Logger // Optional: log trace events to slog
}

ClientOptions holds client configuration.

func ApplyClientOptions 

func ApplyClientOptions(opts ...ClientOption) *ClientOptions

ApplyClientOptions applies options to a ClientOptions struct.

type Dataset 

type Dataset struct {
	ID          string         `json:"id"`
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	Tags        []string       `json:"tags,omitempty"`
	Metadata    map[string]any `json:"metadata,omitempty"`
	ItemCount   int            `json:"item_count,omitempty"`
	CreatedAt   time.Time      `json:"created_at"`
	UpdatedAt   time.Time      `json:"updated_at"`
}

Dataset represents a test dataset for evaluation.

type DatasetItem 

type DatasetItem struct {
	ID       string         `json:"id,omitempty"`
	Input    any            `json:"input,omitempty"`
	Expected any            `json:"expected,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"`
	Tags     []string       `json:"tags,omitempty"`

	// Optional references for items created from traces
	TraceID string `json:"trace_id,omitempty"`
	SpanID  string `json:"span_id,omitempty"`
}

DatasetItem represents a single item in a dataset.

type DatasetManager 

type DatasetManager interface {
	// CreateDataset creates a new dataset for evaluation.
	CreateDataset(ctx context.Context, name string, opts ...DatasetOption) (*Dataset, error)

	// GetDataset retrieves a dataset by name.
	GetDataset(ctx context.Context, name string) (*Dataset, error)

	// GetDatasetByID retrieves a dataset by ID.
	GetDatasetByID(ctx context.Context, id string) (*Dataset, error)

	// AddDatasetItems adds items to a dataset.
	AddDatasetItems(ctx context.Context, datasetName string, items []DatasetItem) error

	// ListDatasets lists available datasets.
	ListDatasets(ctx context.Context, opts ...ListOption) ([]*Dataset, error)

	// DeleteDataset deletes a dataset by ID.
	DeleteDataset(ctx context.Context, datasetID string) error
}

DatasetManager handles test dataset management.

type DatasetOption 

type DatasetOption func(*DatasetOptions)

DatasetOption configures dataset creation.

func WithDatasetDescription 

func WithDatasetDescription(desc string) DatasetOption

WithDatasetDescription sets the dataset description.

func WithDatasetTags 

func WithDatasetTags(tags ...string) DatasetOption

WithDatasetTags sets the dataset tags.

type DatasetOptions 

type DatasetOptions struct {
	Description string
	Tags        []string
	Metadata    map[string]any
}

DatasetOptions holds dataset configuration.

type EndOption 

type EndOption func(*EndOptions)

EndOption configures trace/span ending.

func WithEndError 

func WithEndError(err error) EndOption

WithEndError records an error when ending.

func WithEndMetadata 

func WithEndMetadata(metadata map[string]any) EndOption

WithEndMetadata sets additional metadata when ending.

func WithEndOutput 

func WithEndOutput(output any) EndOption

WithEndOutput sets the final output when ending.

type EndOptions 

type EndOptions struct {
	Output   any
	Metadata map[string]any
	Error    error
}

EndOptions holds end configuration.

type EvalInput 

type EvalInput struct {
	Input    any            `json:"input,omitempty"`    // User query/prompt
	Output   any            `json:"output,omitempty"`   // LLM response
	Expected any            `json:"expected,omitempty"` // Ground truth/expected output
	Context  []string       `json:"context,omitempty"`  // Retrieved context (for RAG)
	Metadata map[string]any `json:"metadata,omitempty"` // Additional metadata

	// Optional references
	TraceID string `json:"trace_id,omitempty"`
	SpanID  string `json:"span_id,omitempty"`
}

EvalInput represents input for evaluation.

type EvalResult 

type EvalResult struct {
	Scores   []MetricScore  `json:"scores"`
	Metadata map[string]any `json:"metadata,omitempty"`
	Duration time.Duration  `json:"duration,omitempty"`
}

EvalResult contains evaluation results.

type Evaluator 

type Evaluator interface {
	// Evaluate runs evaluation metrics on the given input.
	Evaluate(ctx context.Context, input EvalInput, metrics ...Metric) (*EvalResult, error)

	// AddFeedbackScore adds a feedback score to a trace or span.
	AddFeedbackScore(ctx context.Context, opts FeedbackScoreOpts) error
}

Evaluator handles evaluation and scoring of LLM outputs.

type Experiment 

type Experiment struct {
	ID          string         `json:"id"`
	Name        string         `json:"name"`
	DatasetID   string         `json:"dataset_id,omitempty"`
	DatasetName string         `json:"dataset_name,omitempty"`
	Status      string         `json:"status,omitempty"` // running, completed, cancelled
	Metadata    map[string]any `json:"metadata,omitempty"`
	CreatedAt   time.Time      `json:"created_at"`
	UpdatedAt   time.Time      `json:"updated_at"`
}

Experiment represents an evaluation experiment.

type ExperimentItem 

type ExperimentItem struct {
	ID            string         `json:"id,omitempty"`
	ExperimentID  string         `json:"experiment_id"`
	DatasetItemID string         `json:"dataset_item_id,omitempty"`
	Input         any            `json:"input,omitempty"`
	Output        any            `json:"output,omitempty"`
	Expected      any            `json:"expected,omitempty"`
	Scores        []MetricScore  `json:"scores,omitempty"`
	Metadata      map[string]any `json:"metadata,omitempty"`
	TraceID       string         `json:"trace_id,omitempty"`
}

ExperimentItem represents a single evaluation result in an experiment.

type FeedbackOption 

type FeedbackOption func(*FeedbackOptions)

FeedbackOption configures feedback score creation.

func WithFeedbackCategory 

func WithFeedbackCategory(category string) FeedbackOption

WithFeedbackCategory sets the category for the score.

func WithFeedbackReason 

func WithFeedbackReason(reason string) FeedbackOption

WithFeedbackReason sets the reason for the score.

func WithFeedbackSource 

func WithFeedbackSource(source string) FeedbackOption

WithFeedbackSource sets the source of the score.

type FeedbackOptions 

type FeedbackOptions struct {
	Reason   string
	Category string
	Source   string
}

FeedbackOptions holds feedback configuration.

type FeedbackScore 

type FeedbackScore struct {
	Name     string  `json:"name"`
	Score    float64 `json:"score"`
	Reason   string  `json:"reason,omitempty"`
	Category string  `json:"category,omitempty"`
	Source   string  `json:"source,omitempty"` // e.g., "user", "llm", "heuristic"
}

FeedbackScore represents a score given to a trace or span.

type FeedbackScoreOpts 

type FeedbackScoreOpts struct {
	TraceID  string  `json:"trace_id,omitempty"`
	SpanID   string  `json:"span_id,omitempty"`
	Name     string  `json:"name"`
	Score    float64 `json:"score"`
	Reason   string  `json:"reason,omitempty"`
	Category string  `json:"category,omitempty"`
	Source   string  `json:"source,omitempty"`
}

FeedbackScoreOpts configures feedback score creation.

type ListAnnotationsOptions 

type ListAnnotationsOptions struct {
	SpanIDs  []string // List annotations for these span IDs
	TraceIDs []string // List annotations for these trace IDs
}

ListAnnotationsOptions configures annotation listing.

type ListOption 

type ListOption func(*ListOptions)

ListOption configures list operations.

func WithFilter 

func WithFilter(filter map[string]any) ListOption

WithFilter sets filter criteria.

func WithLimit 

func WithLimit(limit int) ListOption

WithLimit sets the maximum number of results.

func WithOffset 

func WithOffset(offset int) ListOption

WithOffset sets the offset for pagination.

func WithOrderBy 

func WithOrderBy(field string) ListOption

WithOrderBy sets the ordering field.

type ListOptions 

type ListOptions struct {
	Limit   int
	Offset  int
	OrderBy string
	Filter  map[string]any
}

ListOptions holds list configuration.

func ApplyListOptions 

func ApplyListOptions(opts ...ListOption) *ListOptions

ApplyListOptions applies options to a ListOptions struct.

type Metric 

type Metric interface {
	// Name returns the metric name.
	Name() string

	// Evaluate computes the metric score for the given input.
	Evaluate(input EvalInput) (MetricScore, error)
}

Metric defines an evaluation metric.

type MetricScore 

type MetricScore struct {
	Name     string  `json:"name"`
	Score    float64 `json:"score"`
	Reason   string  `json:"reason,omitempty"`
	Metadata any     `json:"metadata,omitempty"`
	Error    string  `json:"error,omitempty"`
}

MetricScore represents a single metric evaluation result.

type Project 

type Project struct {
	ID          string         `json:"id"`
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	Metadata    map[string]any `json:"metadata,omitempty"`
	CreatedAt   time.Time      `json:"created_at"`
	UpdatedAt   time.Time      `json:"updated_at"`
}

Project represents a project or workspace.

type ProjectManager 

type ProjectManager interface {
	// CreateProject creates a new project.
	CreateProject(ctx context.Context, name string, opts ...ProjectOption) (*Project, error)

	// GetProject retrieves a project by name.
	GetProject(ctx context.Context, name string) (*Project, error)

	// ListProjects lists available projects.
	ListProjects(ctx context.Context, opts ...ListOption) ([]*Project, error)

	// SetProject sets the current project for subsequent operations.
	SetProject(ctx context.Context, name string) error
}

ProjectManager handles project and workspace management.

type ProjectOption 

type ProjectOption func(*ProjectOptions)

ProjectOption configures project creation.

func WithProjectDescription 

func WithProjectDescription(desc string) ProjectOption

WithProjectDescription sets the project description.

type ProjectOptions 

type ProjectOptions struct {
	Description string
	Metadata    map[string]any
}

ProjectOptions holds project configuration.

type Prompt 

type Prompt struct {
	ID            string         `json:"id"`
	Name          string         `json:"name"`
	Template      string         `json:"template"`
	Description   string         `json:"description,omitempty"`
	Version       string         `json:"version,omitempty"`
	Tags          []string       `json:"tags,omitempty"`
	Metadata      map[string]any `json:"metadata,omitempty"`
	ModelName     string         `json:"model_name,omitempty"`     // LLM model name
	ModelProvider string         `json:"model_provider,omitempty"` // LLM provider
	CreatedAt     time.Time      `json:"created_at"`
	UpdatedAt     time.Time      `json:"updated_at"`
}

Prompt represents a prompt template.

func (*Prompt) Render 

func (p *Prompt) Render(vars map[string]any) (string, error)

Render renders the prompt template with the given variables.

type PromptManager 

type PromptManager interface {
	// CreatePrompt creates a new prompt template.
	CreatePrompt(ctx context.Context, name string, template string, opts ...PromptOption) (*Prompt, error)

	// GetPrompt retrieves a prompt by name, optionally at a specific version or tag.
	// The version parameter can be:
	//   - Empty/omitted: returns the latest version
	//   - A tag name (e.g., "production", "staging"): returns the version with that tag
	//   - A version ID: returns that specific version (if provider supports)
	//
	// Tag-based versioning allows deployment patterns like:
	//   prompt, _ := provider.GetPrompt(ctx, "my-prompt", "production")
	GetPrompt(ctx context.Context, name string, version ...string) (*Prompt, error)

	// ListPrompts lists available prompts.
	ListPrompts(ctx context.Context, opts ...ListOption) ([]*Prompt, error)
}

PromptManager handles prompt template management and versioning.

type PromptOption 

type PromptOption func(*PromptOptions)

PromptOption configures prompt creation.

func WithPromptDescription 

func WithPromptDescription(desc string) PromptOption

WithPromptDescription sets the prompt description.

func WithPromptModel 

func WithPromptModel(model string) PromptOption

WithPromptModel sets the LLM model for the prompt.

func WithPromptProvider 

func WithPromptProvider(provider string) PromptOption

WithPromptProvider sets the LLM provider for the prompt.

func WithPromptTags 

func WithPromptTags(tags ...string) PromptOption

WithPromptTags sets the prompt tags.

type PromptOptions 

type PromptOptions struct {
	Description   string
	Tags          []string
	Metadata      map[string]any
	ModelName     string // LLM model name (e.g., "gpt-4", "claude-3")
	ModelProvider string // LLM provider (e.g., "openai", "anthropic")
}

PromptOptions holds prompt configuration.

type Provider 

type Provider interface {
	Tracer
	Evaluator
	PromptManager
	DatasetManager
	ProjectManager
	AnnotationManager
	io.Closer

	// Name returns the provider name (e.g., "opik", "langfuse", "phoenix")
	Name() string
}

Provider is the main interface that all LLM observability backends implement. It composes specialized interfaces for different capabilities.

func MustOpen 

func MustOpen(name string, opts ...ClientOption) Provider

MustOpen is like Open but panics on error.

func Open 

func Open(name string, opts ...ClientOption) (Provider, error)

Open opens a provider specified by its name.

Most users will use a specific provider package import like: 

import _ "github.com/plexusone/omniobserve/llmops/opik"

And then open it with: 

provider, err := llmops.Open("opik", llmops.WithAPIKey("..."))

type ProviderFactory 

type ProviderFactory func(opts ...ClientOption) (Provider, error)

ProviderFactory creates a new Provider instance with the given options.

type ProviderInfo 

type ProviderInfo struct {
	Name         string
	Description  string
	Website      string
	OpenSource   bool
	SelfHosted   bool
	Capabilities []Capability
}

ProviderInfo contains metadata about a registered provider.

func AllProviderInfo 

func AllProviderInfo() []ProviderInfo

AllProviderInfo returns metadata for all registered providers.

func GetProviderInfo 

func GetProviderInfo(name string) (ProviderInfo, bool)

GetProviderInfo returns metadata about a registered provider.

type Span 

type Span interface {
	// ID returns the unique identifier for this span.
	ID() string

	// TraceID returns the parent trace ID.
	TraceID() string

	// ParentSpanID returns the parent span ID, if this is a nested span.
	ParentSpanID() string

	// Name returns the span name.
	Name() string

	// Type returns the span type (general, llm, tool, guardrail).
	Type() SpanType

	// StartSpan creates a child span within this span.
	StartSpan(ctx context.Context, name string, opts ...SpanOption) (context.Context, Span, error)

	// SetInput sets the span input data.
	SetInput(input any) error

	// SetOutput sets the span output data.
	SetOutput(output any) error

	// SetMetadata sets additional metadata on the span.
	SetMetadata(metadata map[string]any) error

	// SetModel sets the LLM model name (e.g., "gpt-4", "claude-3-opus").
	SetModel(model string) error

	// SetProvider sets the LLM provider name (e.g., "openai", "anthropic").
	SetProvider(provider string) error

	// SetUsage sets token usage information.
	SetUsage(usage TokenUsage) error

	// AddTag adds a tag to the span.
	AddTag(tag string) error

	// AddFeedbackScore adds a feedback score to this span.
	AddFeedbackScore(ctx context.Context, name string, score float64, opts ...FeedbackOption) error

	// End completes the span with optional final output and metadata.
	End(opts ...EndOption) error

	// EndTime returns when the span ended, if it has ended.
	EndTime() *time.Time

	// Duration returns the span duration, or time since start if not ended.
	Duration() time.Duration
}

Span represents a unit of work within a trace, such as an LLM call.

type SpanInfo 

type SpanInfo struct {
	ID           string          `json:"id"`
	TraceID      string          `json:"trace_id"`
	ParentSpanID string          `json:"parent_span_id,omitempty"`
	Name         string          `json:"name"`
	Type         SpanType        `json:"type"`
	StartTime    time.Time       `json:"start_time"`
	EndTime      *time.Time      `json:"end_time,omitempty"`
	Input        any             `json:"input,omitempty"`
	Output       any             `json:"output,omitempty"`
	Metadata     map[string]any  `json:"metadata,omitempty"`
	Model        string          `json:"model,omitempty"`
	Provider     string          `json:"provider,omitempty"`
	Usage        *TokenUsage     `json:"usage,omitempty"`
	Tags         []string        `json:"tags,omitempty"`
	Feedback     []FeedbackScore `json:"feedback,omitempty"`
}

SpanInfo provides read-only information about a span.

type SpanOption 

type SpanOption func(*SpanOptions)

SpanOption configures span creation.

func WithModel 

func WithModel(model string) SpanOption

WithModel sets the LLM model name.

func WithParentSpan 

func WithParentSpan(parentSpanID string) SpanOption

WithParentSpan sets the parent span ID.

func WithProvider 

func WithProvider(provider string) SpanOption

WithProvider sets the LLM provider name.

func WithSpanInput 

func WithSpanInput(input any) SpanOption

WithSpanInput sets the initial input for the span.

func WithSpanMetadata 

func WithSpanMetadata(metadata map[string]any) SpanOption

WithSpanMetadata sets initial metadata for the span.

func WithSpanOutput 

func WithSpanOutput(output any) SpanOption

WithSpanOutput sets the initial output for the span.

func WithSpanTags 

func WithSpanTags(tags ...string) SpanOption

WithSpanTags sets initial tags for the span.

func WithSpanType 

func WithSpanType(t SpanType) SpanOption

WithSpanType sets the span type.

func WithTokenCost 

func WithTokenCost(promptCost, completionCost float64, currency string) SpanOption

WithTokenCost sets token cost information.

func WithTokenUsage 

func WithTokenUsage(prompt, completion int) SpanOption

WithTokenUsage sets token usage information.

type SpanOptions 

type SpanOptions struct {
	Type         SpanType
	Input        any
	Output       any
	Metadata     map[string]any
	Tags         []string
	Model        string
	Provider     string
	Usage        *TokenUsage
	ParentSpanID string
}

SpanOptions holds span configuration.

func ApplySpanOptions 

func ApplySpanOptions(opts ...SpanOption) *SpanOptions

ApplySpanOptions applies options to a SpanOptions struct.

type SpanType 

type SpanType string

SpanType categorizes spans by their function. 

const (
	SpanTypeGeneral   SpanType = "general"
	SpanTypeLLM       SpanType = "llm"
	SpanTypeTool      SpanType = "tool"
	SpanTypeGuardrail SpanType = "guardrail"
	SpanTypeRetrieval SpanType = "retrieval"
	SpanTypeAgent     SpanType = "agent"
	SpanTypeChain     SpanType = "chain"
)

type StreamAccumulator 

type StreamAccumulator struct {
	Chunks       []StreamChunk `json:"chunks"`
	TotalContent string        `json:"total_content"`
	TotalTokens  int           `json:"total_tokens"`
	ChunkCount   int           `json:"chunk_count"`
	StartTime    time.Time     `json:"start_time"`
	FirstChunkAt *time.Time    `json:"first_chunk_at,omitempty"`
	FinishReason string        `json:"finish_reason,omitempty"`
}

StreamAccumulator accumulates streaming chunks.

func NewStreamAccumulator 

func NewStreamAccumulator() *StreamAccumulator

NewStreamAccumulator creates a new stream accumulator.

func (*StreamAccumulator) AddChunk 

func (a *StreamAccumulator) AddChunk(chunk StreamChunk)

AddChunk adds a chunk to the accumulator.

func (*StreamAccumulator) TimeToFirstChunk 

func (a *StreamAccumulator) TimeToFirstChunk() time.Duration

TimeToFirstChunk returns the duration until the first chunk was received.

func (*StreamAccumulator) TotalDuration 

func (a *StreamAccumulator) TotalDuration() time.Duration

TotalDuration returns the total streaming duration.

type StreamChunk 

type StreamChunk struct {
	Content      string         `json:"content,omitempty"`
	TokenCount   int            `json:"token_count,omitempty"`
	Index        int            `json:"index,omitempty"`
	FinishReason string         `json:"finish_reason,omitempty"`
	Metadata     map[string]any `json:"metadata,omitempty"`
}

StreamChunk represents a chunk from streaming LLM output.

type TokenUsage 

type TokenUsage struct {
	PromptTokens     int `json:"prompt_tokens,omitempty"`
	CompletionTokens int `json:"completion_tokens,omitempty"`
	TotalTokens      int `json:"total_tokens,omitempty"`

	// Cost tracking (optional, provider-dependent)
	PromptCost     float64 `json:"prompt_cost,omitempty"`
	CompletionCost float64 `json:"completion_cost,omitempty"`
	TotalCost      float64 `json:"total_cost,omitempty"`
	Currency       string  `json:"currency,omitempty"` // e.g., "USD"
}

TokenUsage represents token consumption for an LLM call.

type Trace 

type Trace interface {
	// ID returns the unique identifier for this trace.
	ID() string

	// Name returns the trace name.
	Name() string

	// StartSpan creates a child span within this trace.
	StartSpan(ctx context.Context, name string, opts ...SpanOption) (context.Context, Span, error)

	// SetInput sets the trace input data.
	SetInput(input any) error

	// SetOutput sets the trace output data.
	SetOutput(output any) error

	// SetMetadata sets additional metadata on the trace.
	SetMetadata(metadata map[string]any) error

	// AddTag adds a tag to the trace.
	AddTag(tag string) error

	// AddFeedbackScore adds a feedback score to this trace.
	AddFeedbackScore(ctx context.Context, name string, score float64, opts ...FeedbackOption) error

	// End completes the trace with optional final output and metadata.
	End(opts ...EndOption) error

	// EndTime returns when the trace ended, if it has ended.
	EndTime() *time.Time

	// Duration returns the trace duration, or time since start if not ended.
	Duration() time.Duration
}

Trace represents an execution trace for an LLM operation or workflow.

type TraceInfo 

type TraceInfo struct {
	ID        string          `json:"id"`
	Name      string          `json:"name"`
	ProjectID string          `json:"project_id,omitempty"`
	StartTime time.Time       `json:"start_time"`
	EndTime   *time.Time      `json:"end_time,omitempty"`
	Input     any             `json:"input,omitempty"`
	Output    any             `json:"output,omitempty"`
	Metadata  map[string]any  `json:"metadata,omitempty"`
	Tags      []string        `json:"tags,omitempty"`
	Feedback  []FeedbackScore `json:"feedback,omitempty"`
}

TraceInfo provides read-only information about a trace.

type TraceOption 

type TraceOption func(*TraceOptions)

TraceOption configures trace creation.

func WithThreadID 

func WithThreadID(threadID string) TraceOption

WithThreadID sets the thread ID for conversation tracking.

func WithTraceInput 

func WithTraceInput(input any) TraceOption

WithTraceInput sets the initial input for the trace.

func WithTraceMetadata 

func WithTraceMetadata(metadata map[string]any) TraceOption

WithTraceMetadata sets initial metadata for the trace.

func WithTraceOutput 

func WithTraceOutput(output any) TraceOption

WithTraceOutput sets the initial output for the trace.

func WithTraceProject 

func WithTraceProject(project string) TraceOption

WithTraceProject sets the project for the trace.

func WithTraceTags 

func WithTraceTags(tags ...string) TraceOption

WithTraceTags sets initial tags for the trace.

type TraceOptions 

type TraceOptions struct {
	ProjectName string
	Input       any
	Output      any
	Metadata    map[string]any
	Tags        []string
	ThreadID    string // For conversation threading
}

TraceOptions holds trace configuration.

func ApplyTraceOptions 

func ApplyTraceOptions(opts ...TraceOption) *TraceOptions

ApplyTraceOptions applies options to a TraceOptions struct.

type Tracer 

type Tracer interface {
	// StartTrace begins a new trace, optionally attaching it to the context.
	StartTrace(ctx context.Context, name string, opts ...TraceOption) (context.Context, Trace, error)

	// StartSpan begins a new span within the current trace context.
	// If no trace exists in context, implementations may create one or return an error.
	StartSpan(ctx context.Context, name string, opts ...SpanOption) (context.Context, Span, error)

	// TraceFromContext retrieves the current trace from context, if any.
	TraceFromContext(ctx context.Context) (Trace, bool)

	// SpanFromContext retrieves the current span from context, if any.
	SpanFromContext(ctx context.Context) (Span, bool)
}

Tracer handles trace and span operations for LLM call tracking.

Source Files

View all Source files

Directories

Path Synopsis
Package langfuse provides a Langfuse adapter for the llmops abstraction.
 Package langfuse provides a Langfuse adapter for the llmops abstraction.
Package metrics provides evaluation metrics for LLM observability.
 Package metrics provides evaluation metrics for LLM observability.
Package slog provides an llmops.Provider that logs trace events to slog.
 Package slog provides an llmops.Provider that logs trace events to slog.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.