tgres

command module
Version: v0.0.0-...-18ed4a1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 25, 2018 License: Apache-2.0 Imports: 8 Imported by: 0

README

Build Status

Tgres is a program comprised of several packages which together can be used to receive, store and present time-series data using a relational database as persistent storage (currently only PostgreSQL).

You can currently use the standalone Tgres daemon as Graphite-like API and Statsd replacement all-in-one, or you can use the Tgres packages to incorporate time series collection and reporting functionality into your application.

See GoDoc for package details.

Whether you use standalone Tgres or as a package, the time series data will appear in your database in a compact and efficient format (by default as a view called tv), while at the same time simple to process using any other tool, language, or framework because it is just a table (or a view, rather). For a more detailed description of how Tgres stores data see this article

Current Status

Feb 7 2018: This project is not actively maintained. You may find quite a bit of time-series wisdom here, but there are probably still a lot of bugs.

Jul 5 2017: See this status update

Jun 15 2017: Many big changes since March, most notably data point versioning and instoduction of ds_state and rra_state tables which contain frequently changed attributes as arrays, similar to the way data points are stored eliminating the need to update ds and rra tables, these are now essentially immutable. Ability to delete series with NOTIFY to Tgres to purge it from the cache.

Mar 22 2017: Version 0.10.0b was tagged. This is our first beta (which is more stable than alpha). Please try it out, and take a minute to open an issue or even a PR if you see/fix any problems. Your feedback is most appreciated!

Feb 2017 Note: A major change in the database structure has been made, Tgres now uses the "write optimized" / "vertical" storage. This change affected most of the internal code, and as far overall status, it set us back a bit, all tests are currently broken, but on the bright side, write performance is amazing now.

Phase 1 or proof-of-concept for the project is the ability to (mostly) act as a drop-in replacement for Graphite (except for chart generation) and Statsd. Currently Tgres supports nearly all of Graphite functions.

As of Aug 2016 Tgres is feature-complete for phase 1, which means that the development will focus on tests, documentation and stability for a while.

Tgres is not ready for production use, but is definitely stable enough for tinkering for those interested.

Getting Started

You need a newer Go (1.7+) and PostgreSQL 9.5 or later. To get the daemon compiled all you need is:

$ go get github.com/tgres/tgres

Now you should have a tgres binary in $GOPATH/bin.

There is also a Makefile which lets you build Tgres with make which will use a slightly more elaborate command and the resulting tgres binary will be able to report its build time and git revision, but otherwise it's the same.

Look in $GOPATH/src/github.com/tgres/tgres/etc for a sample config file. Make a copy of this file and edit it, at the very least check the db-connect-string setting. Also check log-file directory, it must be writable.

The user of the PostgreSQL database needs CREATE TABLE permissions. On first run tgres will create three tables (ds, rra and ts) and two views (tv and tvd).

Tgres is invoked like this:

$ $GOPATH/bin/tgres -c /path/to/config

For Developers

There is nothing specific you need to know. If you'd like to submit a bug fix, or for anything else - use Github.

Migrating Graphite Data

Included in cmd/whisper_import is a program that can copy whisper data into Tgres, its command-line arguments are self-explanatory. You should be able to start sending data to Tgres and then migrate your Graphite data retroactively by running whisper_import to avoid gaps in data. It's probably a good idea to test a small subset of series first, migrations can be time consuming and resource-intensive.

Documentation

Overview

Tgres is a tool for receiving and reporting on simple time series written in Go which uses PostgreSQL for storage.

Source Files

Directories

Path Synopsis
Package aggregator provides the ability to aggregate data points from various sources similar to statsd.
Package aggregator provides the ability to aggregate data points from various sources similar to statsd.
Package blaster provides some stress testing capabilities.
Package blaster provides some stress testing capabilities.
Package cluster is a simplistic clustering implementaion built on top of https://godoc.org/github.com/hashicorp/memberlist.
Package cluster is a simplistic clustering implementaion built on top of https://godoc.org/github.com/hashicorp/memberlist.
cmd
Package daemon is the tgres command line server.
Package daemon is the tgres command line server.
Package dsl is the Domain Specific Language for the series query.
Package dsl is the Domain Specific Language for the series query.
Package graceful provides graceful restart
Package graceful provides graceful restart
Package http provides HTTP functionality for querying TS data as well as submitting data points to a receiver.
Package http provides HTTP functionality for querying TS data as well as submitting data points to a receiver.
Package misc is misc stuff.
Package misc is misc stuff.
Package receiver manages the receiving end of the data.
Package receiver manages the receiving end of the data.
Package rrd contains the logic for updating in-memory partial Round-Robin Archives from incoming and usually unevenly-spaced data points by converting the input to a rate, consolidating and aggregating the data across a list of evenly spaced series of pre-defined resolution and time span.
Package rrd contains the logic for updating in-memory partial Round-Robin Archives from incoming and usually unevenly-spaced data points by converting the input to a rate, consolidating and aggregating the data across a list of evenly spaced series of pre-defined resolution and time span.
Package serde is the interface (and currently PostgreSQL implementaiton) for Serialization/Deserialization of data.
Package serde is the interface (and currently PostgreSQL implementaiton) for Serialization/Deserialization of data.
Package series provides fundamental series operations.
Package series provides fundamental series operations.
Package statsd provides the statsd-like functionality.
Package statsd provides the statsd-like functionality.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto