oteldb

package module
v0.48.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: Apache-2.0 Imports: 0 Imported by: 0

README

oteldb svg logo

oteldb codecov experimental

The next generation, OpenTelemetry-first aggregation system for metrics, traces and logs.

Compatible with PromQL, TraceQL and LogQL.

Based on ClickHouse, fastest open-source (Apache 2.0) column-oriented database.

Supported query languages:

Supported ingestion protocols:

  • Prometheus remote write, including exemplars
  • OpenTelemetry protocol (gRPC) for metrics, traces and logs

Ingestion is possible with OpenTelemetry collector, supporting over 90 protocols.

Demo

https://github.com/user-attachments/assets/647d220c-052a-40d4-9358-7d6039a0f198

Prometheus Compatibility

See ch-compliance for Prometheus compatibility tests.

$ promql-compliance-tester -config-file promql-test-queries.yml -config-file test.oteldb.yml
Total: 547 / 548 (99.82%) passed, 0 unsupported

Quick Start

Setup oteldb, ClickHouse, Grafana, and telemetry generators:

docker compose -f dev/local/ch/docker-compose.yml up -d

You can open Grafana dashboard at http://localhost:3000/d/oteldb/oteldb

Embedded storage

oteldb can also run on the embedded storage engine instead of ClickHouse. The engine is in-process and requires no external dependencies, which makes it convenient for local development, testing, and small single-node deployments. It serves all signals: metrics (PromQL), traces (TraceQL), logs (LogQL) and profiles (Pyroscope). PromQL on the embedded engine passes the full ch-compliance suite (537/537, 100%).

Enable it for every signal with a single flag:

oteldb --embedded

This runs oteldb as a fully self-contained binary: ClickHouse is not started at all (not even the zero-config embedded ClickHouse) and no DSN is required. It is shorthand for setting each signal's backend to storage in the config. You can also enable it per signal, and switch the engine from the default ephemeral in-memory store to an on-disk one:

# oteldb.yml
metrics_backend: storage
traces_backend: storage
logs_backend: storage
profiles_backend: storage

storage:
  backend: file       # "memory" (default, ephemeral) or "file"
  dir: ./oteldb-data  # data directory for the file backend
  flush_interval: 1m  # max age of unflushed head data before it is flushed to a part

Any signal left unset (or set to clickhouse) keeps using ClickHouse, so the two backends can be mixed. Profiles have no ClickHouse implementation and are served only when profiles_backend is storage.

License

Apache License 2.0, see LICENSE.

Documentation

Overview

Package oteldb is OpenTelemetry-compatible storage.

Source Files

View all Source files

Directories

Path Synopsis
cmd
ch2storagebackend command
Command ch2storagebackend migrates data from chstorage's ClickHouse tables into the embedded storagebackend engine, by scanning ClickHouse directly and re-ingesting the decoded records as OTLP pdata.
 Command ch2storagebackend migrates data from chstorage's ClickHouse tables into the embedded storagebackend engine, by scanning ClickHouse directly and re-ingesting the decoded records as OTLP pdata.
chotel command
Binary chotel exports clichkouse traces to otel collector.
 Binary chotel exports clichkouse traces to otel collector.
docker-logql command
Command main implements Docker CLI plugin to run LogQL queries.
 Command main implements Docker CLI plugin to run LogQL queries.
logql-compliance-tester command
odbagent command
Program odbagent is an oteldb OpenTelemetry Collector distribution.
 Program odbagent is an oteldb OpenTelemetry Collector distribution.
odbbackup command
odbmigrate command
odbrestore command
otelbench command
Binary otelbench implements benchmarking suite for oteldb.
 Binary otelbench implements benchmarking suite for oteldb.
otelbench/chdump
Package chdump provides utilities to read ClickHouse data dumps created by oteldb.
 Package chdump provides utilities to read ClickHouse data dumps created by oteldb.
otelbench/chtracker
Package chtracker provides Clickhouse query tracker.
 Package chtracker provides Clickhouse query tracker.
otelbench/logqlbench
Package logqlbench defines utilities to benchmark LogQL queries.
 Package logqlbench defines utilities to benchmark LogQL queries.
otelbench/logsbench
oteldb command
oteldemo command
otelproxy command
Command otelproxy is a Grafana datasource proxy.
 Command otelproxy is a Grafana datasource proxy.
promql-compliance-tester command
Binary promql-compliance-tester performs promql compliance testing based on provided configuration, comparing results with reference implementation.
 Binary promql-compliance-tester performs promql compliance testing based on provided configuration, comparing results with reference implementation.
dev
local/ch-compliance/cmd/compliance-verify command
local/ch-compliance/cmd/compliance-wait command
local/storage-cluster/cmd/cluster-verify command
Command cluster-verify is an end-to-end check for the storage cluster: it pushes one deterministic metric, log, and trace via OTLP to one node, then queries them back through the PromQL, LogQL, and TraceQL APIs of *other* nodes.
 Command cluster-verify is an end-to-end check for the storage cluster: it pushes one deterministic metric, log, and trace via OTLP to one node, then queries them back through the PromQL, LogQL, and TraceQL APIs of *other* nodes.
Package integration wraps integration
 Package integration wraps integration
lokie2e
Package lokie2e provides scripts for E2E testing Loki API implementation.
 Package lokie2e provides scripts for E2E testing Loki API implementation.
prome2e
Package prome2e provides scripts for E2E testing Prometheus API implementation.
 Package prome2e provides scripts for E2E testing Prometheus API implementation.
requirex
Package requirex provides additional testing helpers.
 Package requirex provides additional testing helpers.
tempoe2e
Package tempoe2e provides scripts for E2E testing Tempo API implementation.
 Package tempoe2e provides scripts for E2E testing Tempo API implementation.
Package internal contains go:generate annotations.
 Package internal contains go:generate annotations.
autozpages
Package autozpages setups zPages handler.
 Package autozpages setups zPages handler.
ch2storagebackend
Package ch2storagebackend migrates data out of chstorage's ClickHouse tables into the embedded storagebackend engine, by scanning ClickHouse directly (bypassing chstorage's selector-oriented queriers) and re-ingesting the decoded records as OTLP pdata.
 Package ch2storagebackend migrates data out of chstorage's ClickHouse tables into the embedded storagebackend engine, by scanning ClickHouse directly (bypassing chstorage's selector-oriented queriers) and re-ingesting the decoded records as OTLP pdata.
chembed
Package embedch implements embedded clickhouse.
 Package embedch implements embedded clickhouse.
chotel
Package chotel reads ClickHouse internal OpenTelemetry spans.
 Package chotel reads ClickHouse internal OpenTelemetry spans.
chstorage
Package chstorage provides Clickhouse-based storage.
 Package chstorage provides Clickhouse-based storage.
chstorage/chsql
Package chsql provides fluent Clickhouse SQL query builder.
 Package chsql provides fluent Clickhouse SQL query builder.
chtrace
Package chtrace implements reading ClickHouse traces.
 Package chtrace implements reading ClickHouse traces.
cliversion
Package cliversion provides the version of the binary.
 Package cliversion provides the version of the binary.
ddl
Package ddl provides facilities for generating DDL statements and auxiliary code.
 Package ddl provides facilities for generating DDL statements and auxiliary code.
dockerlog
Package dockerlog provides Docker container log parser.
 Package dockerlog provides Docker container log parser.
faker
Package faker implement a fake telemetry generator.
 Package faker implement a fake telemetry generator.
globalmetric
httpmiddleware
Package httpmiddleware contains HTTP middlewares.
 Package httpmiddleware contains HTTP middlewares.
iterators
Package iterators define storage iterator interfaces and some utilities.
 Package iterators define storage iterator interfaces and some utilities.
lexerql
Package lexerql provides utilities for lexing in *QL languages.
 Package lexerql provides utilities for lexing in *QL languages.
logparser
Package logparser parses logs.
 Package logparser parses logs.
logql
Package logql contains LogQL parser and AST definitions.
 Package logql contains LogQL parser and AST definitions.
logql/lexer
Package lexer contains LogQL lexer.
 Package lexer contains LogQL lexer.
logql/logqlengine
Package logqlengine implements LogQL evaluation engine.
 Package logqlengine implements LogQL evaluation engine.
logql/logqlengine/jsonexpr
Package jsonexpr provides JSON extractor expression parser.
 Package jsonexpr provides JSON extractor expression parser.
logql/logqlengine/logqlabels
Package logqlabels contains LogQL label utilities.
 Package logqlabels contains LogQL label utilities.
logql/logqlengine/logqlerrors
Package logqlerrors defines LogQL engine errors.
 Package logqlerrors defines LogQL engine errors.
logql/logqlengine/logqlmetric
Package logqlmetric provides metric queries implementation.
 Package logqlmetric provides metric queries implementation.
logql/logqlengine/logqlpattern
Package logqlpattern contains parser for LogQL `pattern` stage pattern.
 Package logqlpattern contains parser for LogQL `pattern` stage pattern.
logstorage
Package logstorage defines storage structure for logs storage.
 Package logstorage defines storage structure for logs storage.
lokiapi
Code generated by ogen, DO NOT EDIT.
 Code generated by ogen, DO NOT EDIT.
lokicompliance
Package lokicompliance provides utilities for Loki/LogQL compliance testing.
 Package lokicompliance provides utilities for Loki/LogQL compliance testing.
lokihandler
Package lokihandler provides Loki API implementation.
 Package lokihandler provides Loki API implementation.
lokiproxy
Package lokiproxy provides Loki proxy for observability and research.
 Package lokiproxy provides Loki proxy for observability and research.
metricscache
metricstorage
Package metricstorage defines some interfaces for metric storage.
 Package metricstorage defines some interfaces for metric storage.
odbsafety
Package odbsafety contains shared log safety configuration.
 Package odbsafety contains shared log safety configuration.
otelbench
otelbotapi
otelreceiver
Package otelreceiver provides simple wrapper to setup trace receiver.
 Package otelreceiver provides simple wrapper to setup trace receiver.
otelreceiver/oteldbexporter
Package oteldbexporter contains oteldb exporter factory.
 Package oteldbexporter contains oteldb exporter factory.
otelstorage
Package otelstorage contains common types and utilities for storing OpenTelemetry data.
 Package otelstorage contains common types and utilities for storing OpenTelemetry data.
profilehandler
Package profilehandler provides a Pyroscope-compatible API implementation backed by the ProfileQL engine.
 Package profilehandler provides a Pyroscope-compatible API implementation backed by the ProfileQL engine.
profileql
Package profileql contains a parser and AST definitions for the Pyroscope profile query language.
 Package profileql contains a parser and AST definitions for the Pyroscope profile query language.
profileql/lexer
Package lexer contains ProfileQL lexer.
 Package lexer contains ProfileQL lexer.
profileql/profileqlengine
Package profileqlengine implements the ProfileQL evaluation engine.
 Package profileqlengine implements the ProfileQL evaluation engine.
profilestorage
Package profilestorage defines the storage structure and query interface for the profiles signal.
 Package profilestorage defines the storage structure and query interface for the profiles signal.
promapi
Package promapi contains generated code for OpenAPI specification.
 Package promapi contains generated code for OpenAPI specification.
promcompliance/comparer
promcompliance/config
promcompliance/output
promcompliance/testcases
promhandler
Package promhandler provides Prometheus API implementation.
 Package promhandler provides Prometheus API implementation.
prompb
promproxy
Package promproxy provides Prometheus proxy for observability and research.
 Package promproxy provides Prometheus proxy for observability and research.
promql
Package promql provides a PromQL engine implementation.
 Package promql provides a PromQL engine implementation.
pyroproxy
Package pyroproxy provides Pyroscope proxy for observability and research.
 Package pyroproxy provides Pyroscope proxy for observability and research.
pyroscopeapi
Code generated by ogen, DO NOT EDIT.
 Code generated by ogen, DO NOT EDIT.
semconv
sentryapi
stanzatest
Package stanzatest contains shared stanza test helpers for odbagent packages.
 Package stanzatest contains shared stanza test helpers for odbagent packages.
storagebackend
Package storagebackend adapts the embeddable github.com/oteldb/storage engine to oteldb's query and ingestion interfaces, so any signal can be served from the native Go storage engine instead of ClickHouse.
 Package storagebackend adapts the embeddable github.com/oteldb/storage engine to oteldb's query and ingestion interfaces, so any signal can be served from the native Go storage engine instead of ClickHouse.
tempoapi
Code generated by ogen, DO NOT EDIT.
 Code generated by ogen, DO NOT EDIT.
tempohandler
Package tempohandler provides Tempo API implementation.
 Package tempohandler provides Tempo API implementation.
tempopb
Package tempopb contains generated code for Tempo's protobuf definitions.
 Package tempopb contains generated code for Tempo's protobuf definitions.
tempoproxy
Package tempoproxy provides Tempo proxy for observability and research.
 Package tempoproxy provides Tempo proxy for observability and research.
traceql
Package traceql contains TraceQL parser and AST definitions.
 Package traceql contains TraceQL parser and AST definitions.
traceql/lexer
Package lexer contains TraceQL lexer.
 Package lexer contains TraceQL lexer.
traceql/traceqlengine
Package traceqlengine implements TraceQL evaluation engine.
 Package traceqlengine implements TraceQL evaluation engine.
tracestorage
Package tracestorage defines storage structure for trace storage.
 Package tracestorage defines storage structure for trace storage.
xattribute
Package xattribute provides some helpers to create OpenTelemetry attributes.
 Package xattribute provides some helpers to create OpenTelemetry attributes.
xbytes
xregexp
Package xregexp extends regexp/syntax with helpers for deriving index prefilters from a pattern.
 Package xregexp extends regexp/syntax with helpers for deriving index prefilters from a pattern.
xsync
Package xsync contains synchronization utilities.
 Package xsync contains synchronization utilities.
otelcolmod
chreceiver
Package chreceiver implements a ClickHouse OpenTelemetry receiver.
 Package chreceiver implements a ClickHouse OpenTelemetry receiver.
hubblereceiver
Package hubblereceiver implements an OpenTelemetry Collector receiver that streams flows from Hubble Relay via gRPC and emits OTLP logs.
 Package hubblereceiver implements an OpenTelemetry Collector receiver that streams flows from Hubble Relay via gRPC and emits OTLP logs.
odblogparser
Package odblogparser provides an oteldb log parser stanza operator.
 Package odblogparser provides an oteldb log parser stanza operator.
odbsafety
Package odbsafety provides an oteldb log safety stanza operator.
 Package odbsafety provides an oteldb log safety stanza operator.
odbsafetyprocessor
Package odbsafetyprocessor provides the oteldb agent log safety processor.
 Package odbsafetyprocessor provides the oteldb agent log safety processor.
prometheusremotewritereceiver
Package prometheusremotewritereceiver implements Prometheus Remote Write API for OTLP collector.
 Package prometheusremotewritereceiver implements Prometheus Remote Write API for OTLP collector.
tetragonreceiver
Package tetragonreceiver implements an OpenTelemetry Collector receiver that streams events from Tetragon via gRPC and emits OTLP logs.
 Package tetragonreceiver implements an OpenTelemetry Collector receiver that streams events from Tetragon via gRPC and emits OTLP logs.
Package prometheusremotewrite contains translator from Prometheus remote write format to OTLP metrics.
 Package prometheusremotewrite contains translator from Prometheus remote write format to OTLP metrics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.