otel

Documentation

Overview

Package otel adapts litellm's execution Hook into OpenTelemetry generation spans. Spans carry gen_ai.* semantic-convention attributes, so any OTLP backend (Langfuse, Phoenix, Jaeger, …) renders each LLM call as a generation — without litellm itself depending on OpenTelemetry.

Index

• type OTelHook
  • func New(tracer trace.Tracer, opts ...Option) *OTelHook
  • func (h *OTelHook) AfterResponse(ctx context.Context, meta litellm.CallMeta, resp *litellm.Response, err error)
  • func (h *OTelHook) BeforeRequest(ctx context.Context, meta litellm.CallMeta, req *litellm.Request)
  • func (h *OTelHook) OnStreamEnd(ctx context.Context, meta litellm.CallMeta, err error)
  • func (h *OTelHook) OnStreamEvent(ctx context.Context, meta litellm.CallMeta, event litellm.Event)
  • func (h *OTelHook) OnWarning(ctx context.Context, meta litellm.CallMeta, warning litellm.Warning)
• type Option
  • func WithCaptureContent(capture bool) Option
  • func WithSpanAttributes(fn func(ctx context.Context) []attribute.KeyValue) Option

Types

type OTelHook

type OTelHook struct {
	// contains filtered or unexported fields
}

OTelHook implements litellm.Hook, emitting one OpenTelemetry span per LLM call. litellm invokes hooks synchronously and without panic isolation, so every method recovers internally — observability must never break the call.

func New

func New(tracer trace.Tracer, opts ...Option) *OTelHook

New returns an OTelHook that emits one generation span per LLM call on the given tracer. Register it with litellm.WithHook.

func (*OTelHook) AfterResponse

func (h *OTelHook) AfterResponse(ctx context.Context, meta litellm.CallMeta, resp *litellm.Response, err error)

AfterResponse finishes non-streaming spans. For streaming calls it fires when the stream is established (resp == nil) — the span is finished in OnStreamEvent on the final DoneEvent; only a stream-setup error is handled here.

func (*OTelHook) BeforeRequest

func (h *OTelHook) BeforeRequest(ctx context.Context, meta litellm.CallMeta, req *litellm.Request)

BeforeRequest opens the span and records request-side attributes.

func (*OTelHook) OnStreamEnd

func (h *OTelHook) OnStreamEnd(ctx context.Context, meta litellm.CallMeta, err error)

OnStreamEnd finalizes a streaming span that did not complete through a final Done chunk — the stream aborted (provider error, context cancel; err != nil) or the caller closed it early (err == nil). When the stream completed normally the span was already ended in OnStreamEvent and removed from the map, so this is a no-op for the happy path. Whatever output streamed before termination is still flushed, so partial generations remain visible in the trace.

func (*OTelHook) OnStreamEvent

func (h *OTelHook) OnStreamEvent(ctx context.Context, meta litellm.CallMeta, event litellm.Event)

OnStreamEvent records stream metadata and finishes the span on DoneEvent.

func (*OTelHook) OnWarning

func (h *OTelHook) OnWarning(ctx context.Context, meta litellm.CallMeta, warning litellm.Warning)

type Option

type Option func(*OTelHook)

Option configures an OTelHook.

func WithCaptureContent

func WithCaptureContent(capture bool) Option

WithCaptureContent controls whether input and output messages are recorded on the span. Content capture is disabled by default because messages may contain sensitive information.

func WithSpanAttributes

func WithSpanAttributes(fn func(ctx context.Context) []attribute.KeyValue) Option

WithSpanAttributes registers a resolver invoked once per call in BeforeRequest; the attributes it returns are added to that call's generation span. Use it to attach trace-level metadata the gen_ai.* conventions don't cover — e.g. a session or user id for backends (such as Langfuse) that group generations by those keys. The resolver receives the call's context, so it may read values propagated via context or OTel baggage. Returning nil (or an empty slice) adds nothing for that call.