metrics

package
v0.6.1-0...-7e0b49e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 27, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package metrics implements Iceberg's Metrics Reporting API for iceberg-go.

A Reporter is a pluggable sink that receives a MetricsReport — a ScanReport after scan planning or a CommitReport after a commit — describing what the client did (files and manifests considered, scanned and skipped; bytes read; commit attempts and durations). Reporting gives operators a standard way to aggregate these otherwise-invisible client-side metrics across many clients.

Reporting is strictly opt-in: with no reporter configured the instrumented code paths do no work. Reporters must never block or fail the scan/commit they observe — see Reporter for the contract.

This package provides the contract and the built-in reporters (NopReporter, LoggingReporter, InMemoryReporter, Combine, and CombineWithLogger). The concrete report types and the scan/commit instrumentation are layered on top in later work.

Index

Constants

View Source 
const (
	ReporterNameNop     = "nop"
	ReporterNameLogging = "logging"
)

Built-in reporter names usable as the value of ReporterImplKey.

View Source 
const ReporterImplKey = "metrics-reporter-impl"

ReporterImplKey is the catalog/table property that selects a registered reporter by name (e.g. "logging"). It is the Go analogue of Java's metrics-reporter-impl; Go uses a name→factory registry rather than reflection over a class name.

View Source 
const (
	TimeUnitNanoseconds = "nanoseconds"
)

TimeUnit values for a TimerResult, matching Java's lowercased java.util.concurrent.TimeUnit names. iceberg-go records durations in nanoseconds.

Variables

This section is empty.

Functions

func Deregister 

func Deregister(name string)

Deregister removes a previously registered reporter factory. It is a no-op if name is not registered. This exists primarily so tests can register a factory and undo it via t.Cleanup, keeping the process-global registry re-runnable under go test -count=N.

The built-in "nop" and "logging" factories cannot be removed: since Register panics on a duplicate name, nothing could put them back for the rest of the process, and a stray Deregister("nop") would permanently break metrics-reporter-impl=nop everywhere. Deregister silently ignores those names.

func IsNop 

func IsNop(r Reporter) bool

IsNop reports whether r is known to discard every report — a bare NopReporter or a Combine of only nop reporters. A false result does not guarantee the report is consumed; it only means r is not a recognized no-op.

func Register 

func Register(name string, factory Factory)

Register makes a reporter factory available under name. It panics if name is empty or already registered, mirroring database/sql.Register — registration is expected to happen once, from package init.

Types

type CachedReporter 

type CachedReporter struct {
	// contains filtered or unexported fields
}

CachedReporter builds a Reporter from catalog properties once and caches it, so a catalog holds a single reporter for its lifetime — matching Java's per-catalog MetricsReporter — rather than constructing a fresh one on every table load or commit. That per-operation construction is what makes a stateful reporter (an HTTP-backed one holding a shared client or a background dispatch worker) leak: a new one per load with no owner to close it.

The zero value is ready to use and safe for concurrent use. Close releases the built reporter, giving a catalog a single place to clean up at shutdown.

func (*CachedReporter) Close 

func (c *CachedReporter) Close() error

Close closes the built reporter, if one was ever built, and is a no-op otherwise (including when Get was never called or returned an error). Close is idempotent: it nils the cached reporter after closing, so a second Close does not re-close an underlying reporter that is not obliged to tolerate it, and a post-Close Get hands back NopReporter rather than a released reporter.

A factory returning a typed nil (e.g. (*Custom)(nil)) yields a non-nil Reporter interface that passes the guard below, so Close still invokes its Close — Factory implementations must return a usable reporter or an error, never a typed-nil reporter. Close is expected at catalog shutdown, after operations have quiesced; it is safe for concurrent use, but a Get racing a Close has no defined ordering.

func (*CachedReporter) Get 

func (c *CachedReporter) Get(props map[string]string) (Reporter, error)

Get returns the cached reporter, building it from props on the first call via FromProperties. The first call's result — reporter and error — is cached and returned to every later caller; props supplied on subsequent calls is ignored, because a catalog's reporter configuration does not change over its lifetime.

After Close, Get returns NopReporter with no error: the built reporter has been released, so handing it back would violate the "no Report after Close" contract, and returning nil would force every caller to nil-check.

type CommitMetricsResult 

type CommitMetricsResult struct {
	TotalDuration *TimerResult   `json:"total-duration,omitempty"`
	Attempts      *CounterResult `json:"attempts,omitempty"`

	AddedDataFiles   *CounterResult `json:"added-data-files,omitempty"`
	RemovedDataFiles *CounterResult `json:"removed-data-files,omitempty"`
	TotalDataFiles   *CounterResult `json:"total-data-files,omitempty"`

	AddedDeleteFiles   *CounterResult `json:"added-delete-files,omitempty"`
	RemovedDeleteFiles *CounterResult `json:"removed-delete-files,omitempty"`
	TotalDeleteFiles   *CounterResult `json:"total-delete-files,omitempty"`

	AddedEqualityDeleteFiles   *CounterResult `json:"added-equality-delete-files,omitempty"`
	RemovedEqualityDeleteFiles *CounterResult `json:"removed-equality-delete-files,omitempty"`

	AddedPositionalDeleteFiles   *CounterResult `json:"added-positional-delete-files,omitempty"`
	RemovedPositionalDeleteFiles *CounterResult `json:"removed-positional-delete-files,omitempty"`

	AddedDVs   *CounterResult `json:"added-dvs,omitempty"`
	RemovedDVs *CounterResult `json:"removed-dvs,omitempty"`

	AddedRecords   *CounterResult `json:"added-records,omitempty"`
	RemovedRecords *CounterResult `json:"removed-records,omitempty"`
	TotalRecords   *CounterResult `json:"total-records,omitempty"`

	AddedFilesSizeBytes   *CounterResult `json:"added-files-size-bytes,omitempty"`
	RemovedFilesSizeBytes *CounterResult `json:"removed-files-size-bytes,omitempty"`
	TotalFilesSizeBytes   *CounterResult `json:"total-files-size-bytes,omitempty"`

	AddedPositionalDeletes   *CounterResult `json:"added-positional-deletes,omitempty"`
	RemovedPositionalDeletes *CounterResult `json:"removed-positional-deletes,omitempty"`
	TotalPositionalDeletes   *CounterResult `json:"total-positional-deletes,omitempty"`

	AddedEqualityDeletes   *CounterResult `json:"added-equality-deletes,omitempty"`
	RemovedEqualityDeletes *CounterResult `json:"removed-equality-deletes,omitempty"`
	TotalEqualityDeletes   *CounterResult `json:"total-equality-deletes,omitempty"`

	ManifestsCreated         *CounterResult `json:"manifests-created,omitempty"`
	ManifestsReplaced        *CounterResult `json:"manifests-replaced,omitempty"`
	ManifestsKept            *CounterResult `json:"manifests-kept,omitempty"`
	ManifestEntriesProcessed *CounterResult `json:"manifest-entries-processed,omitempty"`
}

CommitMetricsResult is the serializable set of commit metrics. Field names and units match Java's CommitMetricsResult / CommitMetrics so the wire format is identical across implementations. Unset metrics are omitted.

The counter values are derived from the snapshot summary; note that several of Java's commit-report metric names differ from iceberg-go's snapshot summary keys (e.g. added-files-size-bytes vs the summary's added-files-size), and the commit instrumentation is responsible for emitting them under these Java names.

type CommitReport 

type CommitReport struct {
	TableName      string              `json:"table-name"`
	SnapshotID     int64               `json:"snapshot-id"`
	SequenceNumber int64               `json:"sequence-number"`
	Operation      string              `json:"operation"`
	Metrics        CommitMetricsResult `json:"metrics"`
	Metadata       map[string]string   `json:"metadata,omitempty"`
}

CommitReport is emitted after a commit completes. It maps to the spec's CommitReport schema.

type CounterResult 

type CounterResult struct {
	Unit  Unit  `json:"unit"`
	Value int64 `json:"value"`
}

CounterResult is the serializable snapshot of a single counter. It maps to the spec's CounterResult schema.

func NewCounterResult 

func NewCounterResult(unit Unit, value int64) *CounterResult

NewCounterResult returns a CounterResult with the given unit and value.

type Factory 

type Factory func(props map[string]string) (Reporter, error)

Factory builds a Reporter from configuration properties. The same property map that selected the reporter is passed in, so a factory may read its own configuration keys.

A factory must return either a usable Reporter or a non-nil error, never a typed-nil reporter (e.g. (*Custom)(nil)): that is a non-nil interface value, so it passes interface nil checks and its Report/Close would nil-deref where a caller reasonably assumed a nil interface meant "no reporter".

type InMemoryReporter 

type InMemoryReporter struct {
	// contains filtered or unexported fields
}

InMemoryReporter is a Reporter that retains every report it receives. It is primarily intended for tests and inspection. It is safe for concurrent use.

func (*InMemoryReporter) Close 

func (r *InMemoryReporter) Close() error

Close implements Reporter. InMemoryReporter retains reports in memory and holds no external resources, so Close is a no-op; retained reports remain readable via Reports.

func (*InMemoryReporter) Report 

func (r *InMemoryReporter) Report(_ context.Context, report MetricsReport)

Report appends report to the retained set.

func (*InMemoryReporter) Reports 

func (r *InMemoryReporter) Reports() []MetricsReport

Reports returns a copy of the reports received so far, in arrival order.

func (*InMemoryReporter) Reset 

func (r *InMemoryReporter) Reset()

Reset discards all retained reports.

type LoggingReporter 

type LoggingReporter struct {
	// contains filtered or unexported fields
}

LoggingReporter is a Reporter that logs each report via an slog.Logger. It is a convenient default for development and debugging.

func NewLoggingReporter 

func NewLoggingReporter(logger *slog.Logger) *LoggingReporter

NewLoggingReporter returns a LoggingReporter that logs to logger. If logger is nil, slog.Default is resolved at each Report call, so a later slog.SetDefault is honored rather than snapshotted at construction.

func (*LoggingReporter) Close 

func (r *LoggingReporter) Close() error

Close implements Reporter. A LoggingReporter does not own the logger it writes to, so there is nothing to release; it is a no-op.

func (*LoggingReporter) Report 

func (r *LoggingReporter) Report(ctx context.Context, report MetricsReport)

Report logs report at info level.

type MetricsReport 

type MetricsReport any

MetricsReport is the marker interface implemented by the concrete report types (ScanReport and CommitReport). It is intentionally empty, mirroring the open MetricsReport interface in the Java and Python Iceberg implementations, so downstream operators can implement it for their own report wrappers.

The set of report types is controlled by this package for now, but that is a convention rather than a structural guarantee: the interface is deliberately not sealed. That openness is the irreversible choice — sealing it later would be the breaking change — so it is worth being precise about what it does and does not buy. Custom report types are honored only by the in-process reporters (LoggingReporter, InMemoryReporter, and the Combine fan-out). The REST reporter (once it lands) can serialize only the discriminators it knows (scan-report and commit-report), so a third-party MetricsReport carries no discriminator and is silently dropped by that reporter — a spec-compliant catalog would reject it outright. Do not build a custom report expecting it to reach a catalog endpoint.

The name mirrors the MetricsReport type in the Java and Python Iceberg implementations, easing ports across the ecosystem and disambiguating the type from the Reporter.Report method that consumes it.

type NopReporter 

type NopReporter struct{}

NopReporter is a Reporter that discards every report. It is the default when no reporter is configured, so that instrumentation is free unless a user opts in. The zero value is ready to use.

func (NopReporter) Close 

func (NopReporter) Close() error

Close implements Reporter. NopReporter holds no resources, so it is a no-op.

func (NopReporter) Report 

func (NopReporter) Report(context.Context, MetricsReport)

Report implements Reporter and does nothing.

type ReportMetricsRequest 

type ReportMetricsRequest struct {
	Report MetricsReport
}

ReportMetricsRequest is the body POSTed to the catalog metrics endpoint (POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics). It maps to the spec's ReportMetricsRequest: a ScanReport or CommitReport flattened alongside a "report-type" discriminator (the report's fields are siblings of report-type, not nested).

func NewReportMetricsRequest 

func NewReportMetricsRequest(report MetricsReport) ReportMetricsRequest

NewReportMetricsRequest wraps a report for transport.

func (ReportMetricsRequest) MarshalJSON 

func (r ReportMetricsRequest) MarshalJSON() ([]byte, error)

MarshalJSON writes report-type plus the report's own fields at the top level.

func (*ReportMetricsRequest) UnmarshalJSON 

func (r *ReportMetricsRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON reads the report-type discriminator and decodes the matching concrete report.

type Reporter 

type Reporter interface {
	Report(ctx context.Context, report MetricsReport)
	io.Closer
}

Reporter is a pluggable sink for metrics reports.

Report is invoked inline at the scan/commit completion point, so implementations must return promptly and must never block or fail the operation being observed: network-backed reporters should dispatch the actual send on a background worker, and any error must be handled internally (logged and swallowed), never propagated back to the caller. Report must be safe for concurrent use by multiple goroutines.

Reporter embeds io.Closer to mirror Java's Closeable MetricsReporter: a stateful reporter (e.g. an HTTP-backed one holding a shared client or a background dispatch worker) releases those resources in Close. Close is called once, by the owner of the reporter, when the reporter is no longer needed; Report must not be called after Close returns. Stateless reporters return nil. Close lives on the interface deliberately — adding it after this surface stabilizes would be a breaking change for external implementers.

func Combine 

func Combine(reporters ...Reporter) Reporter

Combine returns a Reporter that forwards each report to all of the given reporters in order. nil reporters are skipped. A panic in one reporter must not prevent the others from receiving the report, so each call is isolated; in keeping with the Reporter contract a misbehaving reporter never affects the observed operation. A recovered panic is logged (with the reporter type) at warn level via slog.Default so a broken reporter is not silently swallowed; use CombineWithLogger to direct that log elsewhere.

As a convenience, Combine with no non-nil reporters returns NopReporter. Otherwise every reporter — even a lone one — is wrapped so it receives the per-reporter panic isolation the contract advertises. Wrapping a single reporter also closes a typed-nil hole: a concrete nil pointer (e.g. (*LoggingReporter)(nil)) is a non-nil interface, so it passes the nil filter; returning it unwrapped would let its eventual Report nil-deref escape with no recover to catch it.

func CombineWithLogger 

func CombineWithLogger(logger *slog.Logger, reporters ...Reporter) Reporter

CombineWithLogger is Combine with an explicit logger for recovered reporter panics. A nil logger resolves slog.Default at each Report call, matching Combine, so a later slog.SetDefault is honored rather than snapshotted at construction. Like Combine, with no non-nil reporters it returns NopReporter and the logger is unused.

func FromProperties 

func FromProperties(props map[string]string) (Reporter, error)

FromProperties builds the reporter named by props[ReporterImplKey]. An absent or empty name yields NopReporter (reporting is opt-in), and an unrecognized name is an error so misconfiguration surfaces rather than silently disabling metrics.

This nop default is an intentional divergence from Java, where an absent metrics-reporter-impl defaults to LoggingMetricsReporter. Callers migrating from Java that want scan/commit reports by default must set ReporterImplKey to ReporterNameLogging explicitly.

type ScanMetricsResult 

type ScanMetricsResult struct {
	TotalPlanningDuration      *TimerResult   `json:"total-planning-duration,omitempty"`
	ResultDataFiles            *CounterResult `json:"result-data-files,omitempty"`
	ResultDeleteFiles          *CounterResult `json:"result-delete-files,omitempty"`
	TotalDataManifests         *CounterResult `json:"total-data-manifests,omitempty"`
	TotalDeleteManifests       *CounterResult `json:"total-delete-manifests,omitempty"`
	ScannedDataManifests       *CounterResult `json:"scanned-data-manifests,omitempty"`
	ScannedDeleteManifests     *CounterResult `json:"scanned-delete-manifests,omitempty"`
	SkippedDataManifests       *CounterResult `json:"skipped-data-manifests,omitempty"`
	SkippedDeleteManifests     *CounterResult `json:"skipped-delete-manifests,omitempty"`
	SkippedDataFiles           *CounterResult `json:"skipped-data-files,omitempty"`
	SkippedDeleteFiles         *CounterResult `json:"skipped-delete-files,omitempty"`
	TotalFileSizeInBytes       *CounterResult `json:"total-file-size-in-bytes,omitempty"`
	TotalDeleteFileSizeInBytes *CounterResult `json:"total-delete-file-size-in-bytes,omitempty"`
	IndexedDeleteFiles         *CounterResult `json:"indexed-delete-files,omitempty"`
	EqualityDeleteFiles        *CounterResult `json:"equality-delete-files,omitempty"`
	PositionalDeleteFiles      *CounterResult `json:"positional-delete-files,omitempty"`
	DVs                        *CounterResult `json:"dvs,omitempty"`
}

ScanMetricsResult is the serializable set of scan-planning metrics. Field names and units match Java's ScanMetricsResult so the wire format is identical across implementations. Unset metrics are omitted.

type ScanReport 

type ScanReport struct {
	TableName           string            `json:"table-name"`
	SnapshotID          int64             `json:"snapshot-id"`
	SchemaID            int               `json:"schema-id"`
	ProjectedFieldIDs   []int             `json:"projected-field-ids"`
	ProjectedFieldNames []string          `json:"projected-field-names"`
	Filter              json.RawMessage   `json:"filter"`
	Metrics             ScanMetricsResult `json:"metrics"`
	Metadata            map[string]string `json:"metadata,omitempty"`
}

ScanReport is emitted after scan planning completes. It maps to the spec's ScanReport schema. Filter holds the scan filter as Expression JSON (a bare boolean for always-true/false, or a {"type":...} object for predicates and and/or/not nodes); when unset it marshals as always-true (see [alwaysTrueFilter]).

func (ScanReport) MarshalJSON 

func (s ScanReport) MarshalJSON() ([]byte, error)

MarshalJSON defaults an unset Filter to the "always true" expression so the spec-required filter field is always present.

type TimerResult 

type TimerResult struct {
	TimeUnit      string `json:"time-unit"`
	Count         int64  `json:"count"`
	TotalDuration int64  `json:"total-duration"`
}

TimerResult is the serializable snapshot of a single timer. It maps to the spec's TimerResult schema. TotalDuration is expressed in TimeUnit.

func NewNanosTimerResult 

func NewNanosTimerResult(count, totalNanos int64) *TimerResult

NewNanosTimerResult returns a TimerResult recording totalNanos nanoseconds observed over count samples.

type Unit 

type Unit string

Unit identifies what a counter measures. The wire values match Java's MetricsContext.Unit display names. 

const (
	UnitUndefined Unit = "undefined"
	UnitCount     Unit = "count"
	UnitBytes     Unit = "bytes"
)

Source Files

View all Source files

Directories

Path Synopsis
Package otel provides an OpenTelemetry-backed metrics.Reporter for iceberg-go.
 Package otel provides an OpenTelemetry-backed metrics.Reporter for iceberg-go.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.