README
¶
Carbonapi
Carbonapi is a Go-based Graphite frontend. It provides two binaries,
carbonapi and carbonzipper, that unify responses from multiple Graphite
backends and provide math and graphing functions.
This project is run in production at Booking.com. We are in the process of documenting its installation and setup, but can answer any questions that interested persons have.
CarbonAPI supports a significant subset of graphite functions; see COMPATIBILITY. In our testing it has shown to be 5x-10x faster than requesting data from graphite-web.
Build
To build both the carbonapi and carbonzipper binaries, run:
make
To build the binaries with debug symbols, run:
make debug
Note: build process might require pkg-config to be installed:
Mac OS X
find pkg-config version on you want to install and run script below with replaced "VERSION_TO_INSTALL":
PKG_CONFIG_VERSION="VERSION_TO_INSTALL" bash -c 'curl https://pkgconfig.freedesktop.org/releases/pkg-config-$PKG_CONFIG_VERSION.tar.gz -o pkgconfig.tgz'
mkdir pkg-config && tar -zxf pkgconfig.tgz -C pkg-config --strip-components 1 && cd pkg-config
There is a circular dependency between pkg-config and glib. To break it, pkg-config includes a version of glib, which is enough to break the dependency cycle and compile it with --with-internal-glib key:
env LDFLAGS="-framework CoreFoundation -framework Carbon" ./configure --with-internal-glib && make install
We do not provide packages for install at this time. Contact us if you're interested in those.
Run
Run the full stack carbonapi -> zipper -> go-carbon with:
docker-compose up
You can feed in sample data with:
echo "test.test 5 `date +%s`" | nc -c localhost 2003
and get it back with:
curl 'http://localhost:8081/render?target=test.test&format=json&from=-10m'
Requirements
We officially support go 1.18. Booking.com builds its binaries
with the latest stable release of Go at any time.
OSX Build Notes
Some additional steps may be needed to build carbonapi with cairo rendering on MacOSX.
Install cairo:
brew install Caskroom/cask/xquartz
brew install cairo
Xquartz is a required dependency for cairo.
Backend support
go-carbon
The main supported backend is go-carbon Graphite store.
Clickhouse
Clickhouse support is experimental.
The pair carbonapi and carbonzipper works with Clickhouse via graphite-clickhouse.
The access chain then looks like this:
carbonapi -> zipper -> graphite-clickhouse -> clickhouse
(this presumes the data was written into Clickhouse with carbon-clickhouse).
Run
docker-compose --file docker-compose-clickhouse.yaml up
to get the setup up and running in several Docker containers.
Acknowledgement and history
This program was originally developed for Booking.com. With approval from Booking.com, the code was generalised and published as Open Source on GitHub, for which the author would like to express his gratitude.
This is Booking.com's fork of go-graphite/carbonapi. That project's current performance characteristics are not sufficient for our production needs, and we decided it had moved too far ahead for us to be able to improve them effectively. We thus reverted back to versions 0.9.2 of carbonapi and 0.74 of carbonzipper, and are moving more slowly in the same direction as the original project.
License
This code is licensed under the BSD-2 license.
Directories
¶
| Path | Synopsis |
|---|---|
|
app
|
|
|
carbonzipper
Package zipper exposes following non-monitoring endpoints:
|
Package zipper exposes following non-monitoring endpoints: |
|
cmd
|
|
|
pkg
|
|
|
backend
Package backend defines an interface and RPC methods for communication with Graphite backends.
|
Package backend defines an interface and RPC methods for communication with Graphite backends. |
|
backend/mock
Package mock defines a mock backend for testing.
|
Package mock defines a mock backend for testing. |
|
backend/net
Package net implements a backend that communicates over a network.
|
Package net implements a backend that communicates over a network. |
|
types
Package types defines the main Graphite types we use internally.
|
Package types defines the main Graphite types we use internally. |
|
types/encoding/carbonapi_v2
Package carbonapi_v2 defines encoding and decoding methods for Find, Info and Render responses.
|
Package carbonapi_v2 defines encoding and decoding methods for Find, Info and Render responses. |
|
types/encoding/json
Package json defines encoding and decoding methods for Find, Info and Render responses.
|
Package json defines encoding and decoding methods for Find, Info and Render responses. |
|
types/encoding/pickle
Package pickle defines encoding methods for Find and Render responses.
|
Package pickle defines encoding methods for Find and Render responses. |
|
player
Package player for pre-recorded TCP traffic for carbonapi
|
Package player for pre-recorded TCP traffic for carbonapi |
|
Package util provides UUIDs for CarbonAPI and CarbonZipper HTTP requests.
|
Package util provides UUIDs for CarbonAPI and CarbonZipper HTTP requests. |