telemetry

package
v0.0.0-...-7daaf31 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 20, 2026 License: AGPL-3.0 Imports: 29 Imported by: 0

README

Telemetry Package

This package handles telemetry data storage and retrieval using ClickHouse for high-performance analytics queries.

ClickHouse Queries

Unlike PostgreSQL queries in other packages, ClickHouse queries are not auto-generated by sqlc since sqlc doesn't support ClickHouse. However, we follow sqlc conventions for consistency.

Query Files
  • queries.sql.go: Manual Go implementations of queries following sqlc patterns

Note: We do NOT maintain a separate queries.sql file. All queries are defined directly in queries.sql.go.

Adding a New Query

Implement directly in queries.sql.go:

  1. Define the query as a const with sqlc-style comment header
  2. Create a params struct (if needed) right before the function
  3. Implement the function following existing patterns
Example Pattern

In queries.sql.go:

const listItems = `-- name: ListItems :many
select id, name from items where project_id = ? limit ?
`

type ListItemsParams struct {
    ProjectID string
    Limit     int
}

func (q *Queries) ListItems(ctx context.Context, arg ListItemsParams) ([]Item, error) {
    rows, err := q.conn.Query(ctx, listItems, arg.ProjectID, arg.Limit)
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var items []Item
    for rows.Next() {
        var i Item
        if err = rows.ScanStruct(&i); err != nil {
            return nil, fmt.Errorf("error scanning row: %w", err)
        }
        items = append(items, i)
    }

    return items, rows.Err()
}

Pagination

Service Layer Pagination (limit + 1 pattern)

Pagination logic lives in the service layer (impl.go), not the repo layer. The repo returns raw results, and the service handles cursor computation.

  1. Client requests N items per page
  2. Service queries repo with N+1 items
  3. If N+1 items returned → compute nextCursor from item N, trim to N items
  4. If ≤N items returned → nextCursor = nil, return all items
  5. Cursor is the UUID of the last returned item
ClickHouse-Specific Patterns
Empty String Cursor Sentinel

Use empty string to indicate "no cursor" (first page). This avoids complex nil UUID checks:

AND (
    ? = '' OR
    IF(
        ? = 'asc',
        (time_unix_nano, toUUID(id)) > (SELECT time_unix_nano, toUUID(id) FROM table WHERE id = ? LIMIT 1),
        (time_unix_nano, toUUID(id)) < (SELECT time_unix_nano, toUUID(id) FROM table WHERE id = ? LIMIT 1)
    )
)

When calling from Go:

cursor := ""  // First page
cursor := "some-uuid"  // Subsequent pages
Optional Filters

Use the pattern (? = '' or field = ?) to make filters optional:

and (? = '' or deployment_id = ?)

Pass empty string when filter is not needed, or pass the value twice when filtering:

arg.DeploymentID, arg.DeploymentID  // pass twice for the pattern above
UUIDv7 Timestamp Extraction

Use UUIDv7ToDateTime(toUUID(?)) to extract the embedded timestamp from UUIDv7 for cursor-based pagination:

timestamp > UUIDv7ToDateTime(toUUID(?))

Testing

Tests use testcontainers to spin up a real ClickHouse instance. See list_tool_logs_test.go for examples.

Key testing patterns:

  • Use testenv.Launch() in TestMain to set up infrastructure
  • Create helper functions for inserting test data
  • Use table-driven tests with descriptive names
  • Add time.Sleep(100 * time.Millisecond) after inserts to allow ClickHouse to make data available

Data Models

See models.go for struct definitions with ClickHouse field tags (ch:"field_name").

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Attach 

func Attach(mux goahttp.Muxer, service *Service)

func EmitHTTPRequestLog 

func EmitHTTPRequestLog(
	ctx context.Context,
	logger *slog.Logger,
	provider ToolMetricsProvider,
	toolName string,
	request repo.ToolHTTPRequest,
)

EmitHTTPRequestLog logs the provided HTTP request using the tool metrics provider. Errors are reported through the supplied logger. Logging happens asynchronously to avoid blocking the caller and the request struct is copied to prevent data races.

Types

type PosthogClient 

type PosthogClient interface {
	CaptureEvent(ctx context.Context, eventName string, distinctID string, eventProperties map[string]interface{}) error
}

PosthogClient defines the interface for capturing events in PostHog.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

func NewService 

func NewService(logger *slog.Logger, db *pgxpool.Pool, sessions *sessions.Manager, chatSessions *chatsessions.Manager, tcm ToolMetricsProvider, features *productfeatures.Client, posthogClient PosthogClient) *Service

func (*Service) APIKeyAuth 

func (s *Service) APIKeyAuth(ctx context.Context, key string, schema *security.APIKeyScheme) (context.Context, error)

func (*Service) CaptureEvent 

func (s *Service) CaptureEvent(ctx context.Context, payload *telem_gen.CaptureEventPayload) (res *telem_gen.CaptureEventResult, err error)

CaptureEvent captures a telemetry event and forwards it to PostHog.

func (*Service) JWTAuth 

func (s *Service) JWTAuth(ctx context.Context, token string, schema *security.JWTScheme) (context.Context, error)

func (*Service) ListLogs 

func (s *Service) ListLogs(ctx context.Context, payload *gen.ListLogsPayload) (res *gen.ListToolLogResponse, err error)

func (*Service) ListToolExecutionLogs 

func (s *Service) ListToolExecutionLogs(ctx context.Context, payload *gen.ListToolExecutionLogsPayload) (res *gen.ListToolExecutionLogsResult, err error)

func (*Service) SearchLogs 

func (s *Service) SearchLogs(ctx context.Context, payload *telem_gen.SearchLogsPayload) (res *telem_gen.SearchLogsResult, err error)

SearchLogs retrieves unified telemetry logs with pagination.

func (*Service) SearchToolCalls 

func (s *Service) SearchToolCalls(ctx context.Context, payload *telem_gen.SearchToolCallsPayload) (res *telem_gen.SearchToolCallsResult, err error)

SearchToolCalls retrieves tool call summaries with pagination.

type StubToolMetricsClient 

type StubToolMetricsClient struct{}

func (*StubToolMetricsClient) InsertTelemetryLog 

func (n *StubToolMetricsClient) InsertTelemetryLog(_ context.Context, _ repo.InsertTelemetryLogParams) error

func (*StubToolMetricsClient) ListHTTPRequests 

func (n *StubToolMetricsClient) ListHTTPRequests(_ context.Context, _ repo.ListToolLogsOptions) (*repo.ListResult, error)

func (*StubToolMetricsClient) ListLogsForTrace 

func (n *StubToolMetricsClient) ListLogsForTrace(_ context.Context, _ repo.ListLogsForTraceParams) ([]repo.TelemetryLog, error)

func (*StubToolMetricsClient) ListTelemetryLogs 

func (n *StubToolMetricsClient) ListTelemetryLogs(_ context.Context, _ repo.ListTelemetryLogsParams) ([]repo.TelemetryLog, error)

func (*StubToolMetricsClient) ListToolLogs 

func (n *StubToolMetricsClient) ListToolLogs(_ context.Context, _ repo.ListToolLogsParams) (*repo.ToolLogsListResult, error)

func (*StubToolMetricsClient) ListTraces 

func (n *StubToolMetricsClient) ListTraces(_ context.Context, _ repo.ListTracesParams) ([]repo.TraceSummary, error)

func (*StubToolMetricsClient) LogHTTPRequest 

func (n *StubToolMetricsClient) LogHTTPRequest(_ context.Context, _ repo.ToolHTTPRequest) error

type ToolCallLogRoundTripper 

type ToolCallLogRoundTripper struct {
	// contains filtered or unexported fields
}

ToolCallLogRoundTripper wraps an http.RoundTripper and logs HTTP requests to ClickHouse

func NewToolCallLogRoundTripper 

func NewToolCallLogRoundTripper(rt http.RoundTripper, logger *slog.Logger, tracer trace.Tracer, toolInfo *ToolInfo, toolLogger ToolCallLogger) *ToolCallLogRoundTripper

NewToolCallLogRoundTripper creates a new RoundTripper that logs HTTP requests to ClickHouse

func (*ToolCallLogRoundTripper) RoundTrip 

func (h *ToolCallLogRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper interface

type ToolCallLogger 

type ToolCallLogger interface {
	Enabled() bool
	Emit(ctx context.Context, logger *slog.Logger)
	RecordDurationMs(durationMs float64)
	RecordHTTPServerURL(url string)
	RecordHTTPMethod(method string)
	RecordHTTPRoute(route string)
	RecordStatusCode(code int)
	RecordUserAgent(agent string)
	RecordRequestHeaders(headers map[string]string, isSensitive bool)
	RecordResponseHeaders(headers map[string]string)
	RecordRequestBodyBytes(bytes int64)
	RecordResponseBodyBytes(bytes int64)
}

ToolCallLogger represents a logging strategy for tool HTTP requests. Implementations may be backed by a real ToolHTTPRequest or behave as no-ops.

func NewNoopToolCallLogger 

func NewNoopToolCallLogger() ToolCallLogger

NewNoopToolCallLogger creates a ToolCallLogger that drops all logging information.

func NewToolCallLogger 

func NewToolCallLogger(
	ctx context.Context,
	provider ToolMetricsProvider,
	featuresClient *productfeatures.Client,
	organizationID string,
	info ToolInfo,
	toolName string,
	toolType repo.ToolType,
) (ToolCallLogger, error)

NewToolCallLogger returns a ToolCallLogger that records tool calls when logging is enabled for the organization; otherwise it returns a no-op logger. When an error occurs while preparing the logger, a no-op logger is returned alongside the error.

type ToolInfo 

type ToolInfo struct {
	ID             string
	Urn            string
	Name           string
	ProjectID      string
	DeploymentID   string
	OrganizationID string
}

ToolInfo represents the minimal tool information needed for logging

type ToolMetricsProvider 

type ToolMetricsProvider interface {
	// List tool call logs
	ListHTTPRequests(ctx context.Context, opts repo.ListToolLogsOptions) (*repo.ListResult, error)
	// List structured tool logs
	ListToolLogs(ctx context.Context, params repo.ListToolLogsParams) (*repo.ToolLogsListResult, error)
	// List unified telemetry logs (new OTel-based table)
	ListTelemetryLogs(ctx context.Context, params repo.ListTelemetryLogsParams) ([]repo.TelemetryLog, error)
	// List trace summaries for distributed tracing
	ListTraces(ctx context.Context, params repo.ListTracesParams) ([]repo.TraceSummary, error)
	// List all logs for a specific trace ID
	ListLogsForTrace(ctx context.Context, params repo.ListLogsForTraceParams) ([]repo.TelemetryLog, error)
	// Log tool call request/response
	LogHTTPRequest(context.Context, repo.ToolHTTPRequest) error
	// Insert telemetry log
	InsertTelemetryLog(ctx context.Context, params repo.InsertTelemetryLogParams) error
}

ToolMetricsProvider defines the interface for tool metrics operations.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.