bylonis-otel-collector

module
v0.144.5-bylonis.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: AGPL-3.0

README

ByLonis OpenTelemetry Collector

The OpenTelemetry Collector distribution that ingests traces, logs and metrics into ByLonis's ClickHouse, plus the schema migrator that creates and upgrades those databases.

Hard fork of SigNoz/signoz-otel-collector at v0.144.5, under the AGPL-3.0 (see LICENSE and NOTICE).

Build

Requires Go 1.25.

make build        # .build/<os>-<arch>/bylonis-otel-collector and bylonis-schema-migrator
make test

Run

bylonis-otel-collector migrate bootstrap   # create the databases
bylonis-otel-collector migrate sync up     # schema migrations
bylonis-otel-collector migrate async up    # background migrations
bylonis-otel-collector --config=config.yaml

Configuration

Every flag can be set with an env var named BYLONIS_OTEL_COLLECTOR_<FLAG> (--clickhouse-dsn → BYLONIS_OTEL_COLLECTOR_CLICKHOUSE_DSN).

Env var Default Meaning
BYLONIS_OTEL_COLLECTOR_CLICKHOUSE_DSN tcp://0.0.0.0:9001 ClickHouse used by migrate
BYLONIS_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER cluster ClickHouse cluster name
BYLONIS_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION true Replicated tables

The databases are always bylonis_traces, bylonis_metrics, bylonis_logs, bylonis_meter, bylonis_metadata and bylonis_analytics.

The components built for ByLonis are named bylonis*: bylonisspanmetrics, bylonismeter, bylonisclickhousemetrics, bylonisclickhousemeter, bylonislogspipeline, bylonistransform, bylonis_tail_sampling, bylonisllmpricing, byloniskafkareceiver, byloniskafkaexporter, bylonisawsfirehose, bylonisjsontypeexporter and bylonis_health_check.

Logs pipelines preview

The ByLonis backend runs the logs pipelines preview, and validates pipelines before saving them, in the collector of the tenant (pkg/previewserver): it sends the pipeline processors it generates and sample logs, and gets the processed logs back. The server starts with the collector, outside the config managed over OpAMP, and only when a token is set.

Env var Default Meaning
BYLONIS_OTEL_COLLECTOR_PREVIEW_LISTEN :4319 Address of the preview server
BYLONIS_OTEL_COLLECTOR_PREVIEW_TOKEN (none: off) Bearer token, at least 16 characters. The backend uses the same one (BYLONIS_LOGSPIPELINE_PREVIEW_TOKEN)

Keep the port on the internal network of the tenant: don't publish it. The endpoint is POST /v1/logs-pipelines/preview; it only accepts bylonislogspipeline processors, never expands ${...} in their config and never logs the logs it gets. Limits: 1 MiB body, 100 logs and 100 processors per request, 10 s, 2 previews at once (429 beyond).

OpAMP token

With --manager-config, the collector gets its config from the OpAMP server of the ByLonis backend (server_endpoint). Set BYLONIS_OTEL_COLLECTOR_OPAMP_TOKEN (--opamp-token) and the collector sends Authorization: Bearer <token> when it opens the WebSocket connection; the backend rejects connections without the token it expects (BYLONIS_OPAMP_TOKEN). Without it, no header is sent and the log says so with a warning.

Upgrading from a SigNoz deployment or a version before v1.0

v1.0 drops the compatibility with the upstream names that earlier versions kept. Before switching images, a deployment that still uses them has to:

Before Now If it is left
signoz* component types (signozspanmetrics, signozmeter, signoz_health_check, ...) bylonis* with the same suffix (bylonisspanmetrics, bylonismeter, bylonis_health_check, ...) the config fails to load: unknown type: "signozspanmetrics"
SIGNOZ_OTEL_COLLECTOR_<FLAG> env vars BYLONIS_OTEL_COLLECTOR_<FLAG> the collector doesn't start and names the variable to rename
BYLONIS_DB_PREFIX (databases signoz_* or another prefix) databases bylonis_*; unset the variable (bylonis is still accepted) the collector and the migrator don't start
SQL naming signoz_traces, signoz_logs, ... (rewritten until now) the bylonis_* names ClickHouse answers that the database doesn't exist
legacy_metric_names in bylonisspanmetrics and bylonismeter (copies of the metrics as signoz_* and signoz.meter.*, with signoz.collector.id) nothing: only the bylonis_* and bylonis.meter.* names, with bylonis.collector.id true fails validation; false is ignored with a warning, and will be rejected in a later release
/signoz-otel-collector and /signoz-schema-migrator in the images /bylonis-otel-collector, /bylonis-schema-migrator the container doesn't start: no such file or directory

The usage metering no longer skips metrics named signoz* or resource attributes signoz.workspace.*; it skips bylonis* metrics and bylonis.workspace.* attributes.

Directories

Path Synopsis
cmd
config
connectors
exporter
byloniskafkaexporter
package byloniskafkaexporter exports trace data to Kafka.
package byloniskafkaexporter exports trace data to Kafka.
byloniskafkaexporter/internal/awsmsk
Package msk implements the required IAM auth used by AWS' managed Kafka platform to be used with the Surama kafka producer.
Package msk implements the required IAM auth used by AWS' managed Kafka platform to be used with the Surama kafka producer.
jsontypeexporter
Package jsontypeexporter exports telemetry data to JSON files.
Package jsontypeexporter exports telemetry data to JSON files.
extension
healthcheckextension
Package healthcheckextension implements an extension that enables an HTTP endpoint that can be used to check the overall health and status of the service.
Package healthcheckextension implements an extension that enables an HTTP endpoint that can be used to check the overall health and status of the service.
internal
kafka/awsmsk
Package msk implements the required IAM auth used by AWS' managed Kafka platform to be used with the Surama kafka producer.
Package msk implements the required IAM auth used by AWS' managed Kafka platform to be used with the Surama kafka producer.
sharedcomponent
Package sharedcomponent exposes util functionality for receivers and exporters that need to share state between different signal types instances such as net.Listener or os.File.
Package sharedcomponent exposes util functionality for receivers and exporters that need to share state between different signal types instances such as net.Listener or os.File.
pkg
dbnames
Package dbnames names the ClickHouse databases ByLonis uses: bylonis_traces, bylonis_metrics, bylonis_logs, bylonis_meter, bylonis_metadata and bylonis_analytics.
Package dbnames names the ClickHouse databases ByLonis uses: bylonis_traces, bylonis_metrics, bylonis_logs, bylonis_meter, bylonis_metadata and bylonis_analytics.
previewserver
Package previewserver serves the logs pipelines preview of the ByLonis backend: it runs the pipeline processors the backend generates on sample logs, in an in-memory collector built with the same processor this binary runs for ingestion, and returns the processed logs.
Package previewserver serves the logs pipelines preview of the ByLonis backend: it runs the pipeline processors the backend generates on sample logs, in an in-memory collector built with the same processor this binary runs for ingestion, and returns the processed logs.
processor
bylonislogspipelineprocessor
Brought in as is from logstransform processor in opentelemetry-collector-contrib
Brought in as is from logstransform processor in opentelemetry-collector-contrib
bylonislogspipelineprocessor/stanza/adapter
Register copies of stanza operators dedicated to signoz logs pipelines
Register copies of stanza operators dedicated to signoz logs pipelines
bylonislogspipelineprocessor/stanza/entry
brought in as-is from opentelemetry-collector-contrib with changes to Get() to enable reading fields inside JSON body directly
brought in as-is from opentelemetry-collector-contrib with changes to Get() to enable reading fields inside JSON body directly
bylonislogspipelineprocessor/stanza/operator/helper
brought in as is from "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper" with minor changes to use bylonisstanzaentry.Field instead of entry.Field
brought in as is from "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper" with minor changes to use bylonisstanzaentry.Field instead of entry.Field
bylonislogspipelineprocessor/stanza/operator/operators/remove
Brought in as is from opentelemetry-collector-contrib
Brought in as is from opentelemetry-collector-contrib
bylonislogspipelineprocessor/stanza/operator/operatortest
Brought in as is from opentelemetry-collector-contrib with anyOpConfig implementation updated to use operator config in logspipelineprocessor
Brought in as is from opentelemetry-collector-contrib with anyOpConfig implementation updated to use operator config in logspipelineprocessor
bylonistailsampler
package bylonistailsampler contains the logic to execute telemetry transform based on the OpenTelemetry Transformation Language.
package bylonistailsampler contains the logic to execute telemetry transform based on the OpenTelemetry Transformation Language.
bylonistailsampler/internal/idbatcher
Package idbatcher defines a pipeline of fixed size in which the elements are batches of ids.
Package idbatcher defines a pipeline of fixed size in which the elements are batches of ids.
bylonistailsampler/internal/sampling
Package sampling contains the interfaces and data types used to implement the various sampling policies.
Package sampling contains the interfaces and data types used to implement the various sampling policies.
bylonistransformprocessor
package bylonistransformprocessor contains the logic to execute telemetry transform based on the OpenTelemetry Transformation Language.
package bylonistransformprocessor contains the logic to execute telemetry transform based on the OpenTelemetry Transformation Language.
receiver
byloniskafkareceiver
Package kafkareceiver receives traces from Kafka.
Package kafkareceiver receives traces from Kafka.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL