prometheus-aggregator

README

Prometheus metrics aggregator

Receive and aggregate metrics for consumption by a Prometheus server.

DON'T USE THIS TOOL. If at all possible, you should expose a /metrics endpoint in your service and have Prometheus scrape it directly. If you're running a cron job, prefer the pushgateway. This tool only exists to help with edge case scenarios, for edxample Perl web services that use a Unicorn-style forked process model to handle concurrency, and are difficult or impossible to get Prometheus to scrape.

Getting

[Download the latest release][release] if you're lazy, or build it yourself from latest laster if you have the Go toolchain installed and have YOLO tattooed on your knuckles or whatever.

go get github.com/peterbourgon/prometheus-aggregator
$GOPATH/bin/prometheus-aggregator -h

How it works

The prometheus-aggregator expects clients to connect and emit newline-delimited JSON objects for each metric observation. Each object needs to be fully specified with name, type, help, and value. Here's an example of three counter increments.

{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos.", "value": 1}
{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos.", "value": 1}
{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos.", "value": 1}

Obviously this is wildly inefficient, so, as an optimization, once a metric has been "declared" with name, type, and help, subsequent emissions may refer to it simply by name.

{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos.", "value": 1}
{"name": "myapp_foo_total", "value": 1}
{"name": "myapp_foo_total", "value": 1}

You can delare a metric without making an observation by omitting the value.

{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos."}
{"name": "myapp_foo_total", "value": 1}  # value is now 1
{"name": "myapp_foo_total", "value": 2}  # value is now 3

You can declare metrics at runtime, like this, or you can predeclare metrics in a JSON file, and pass it to the program at startup. Or mix and match both. What do I care? I'm just some stupid README file.

Prometheus exposition format

If serializing JSON is a bottleneck, you can optionally emit observations (but not declarations) in the Prometheus exposition format.

{"name": "myapp_foo_total", "type": "counter", "help": "Total number of foos."}
myapp_foo_total{} 1
myapp_foo_total{} 2

Labels

Labels are supported in both formats as you might expect.

{"name": "myapp_foo_total", "labels": {"success": "false", "code": "401"}, "value": 1}
myapp_foo_total{success="true",code="200"} 1

Supported types

Counters are obviously supported. Gauges are also supported and work just like counters, but default to setting themselves to the most recent value.

{"name": "myapp_worker_pool", "type": "gauge", "help": "Size of worker pool."}
myapp_worker_pool{} 1  # value is now 1
myapp_worker_pool{} 2  # value is now 2

Histograms are supported too. Provide buckets with the declaration.

{"name": "myapp_req_dur_seconds", "type": "histogram",
  "help": "Duration of request in seconds.", 
    "buckets": [0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10]}
myapp_req_dur_seconds{} 0.0123\n
myapp_req_dur_seconds{} 0.99\n

Summaries are not supported. This is fine, you can't do meaningful aggregation over summaries at query time anyway. You'll need to define some buckets and I know that sounds hard, and it is hard, life is hard, I'm sorry for that but you can take some solace in the fact that we all kind of share this suffering as a species.