README

Loggregator Tools

HTTPS Drain

The HTTPS Drain is an example HTTPS server that accepts syslog messages (RFC-5424). This is a useful tool for debugging and monitoring cf-syslog-drain-release. The HTTPS drain can be configured to POST a counter.

Syslog Drain

The Syslog Drain is an example TCP server that accepts syslog messages (RFC-5424) via TCP. This is a useful tool for debugging and monitoring cf-syslog-drain-release. The Syslog drain can be configured to POST a counter.

Syslog to Datadog

The Syslog to Datadog is an example HTTPS server that accepts syslog messages (RFC-5424) with metrics in the structured data. The metrics will be sent to datadog.

The Syslog to Datadog application can be configured with DATADOG_API_KEY and PORT environment variables.

Datadog Accumulator

The Datadog Accumulator scrapes Gauge values for a given source-id and send the results to Datadog.

Log Spinner

Log Spinner is a sample CF application that is written in go. It is compatible with the go-buildpack.

JSON Spinner

JSON Spinner is a sample CF application that is written in go. It is compatible with the go-buildpack. It is used by the cf-syslog-drain black box tests.

Request Spinner

Request Spinner reads from Log Cache for load testing purposes. The request can be configured to hit specific source IDs, at a given cycle and delay.

Log Cache Siege

Log Cache Siege is configured with the address of a request-spinner to instruct request-spinner to hit every available source ID.

Log Cache Emitter

Log Cache Emitter is meant to be bosh deployed and will emit 10,000 logs to log cache for each source id given. request takes the form of GET to http://:/emit?sourceIDs=source1&sourceIDs=source2

Slow Consumer

The Slow Consumer is a firehose nozzle that will induce the TrafficController to cut off the nozzle.

Post Printer

The post printer is a CF application that prints every request to stderr.

Datadog Forwarder

The Datadog forwarder reads from Log Cache and forwards metrics to Datadog.

CF LogMon

The CF LogMon performs a blacbox test for measuring message reliability when running the command cf logs. This is accomplished by writing groups of logs, measuring the time it took to produce the logs, and then counting the logs received in the log stream. This is one way to measure message reliability of the Loggregator system. The results of this test are displayed in a simple UI and available via JSON and the Firehose.

Syslog Nozzle

This is a nozzle that converts envelopes into syslog messages. This is meant to be deployed in Kubernetes. The dockerfile for packaging this app is located here. The kubernetes deployment manifest and instructions for configuring it can be found here.

Expand ▾ Collapse ▴

Directories

Path Synopsis
backpressure
biglogger
counter
counter/web
dopplerclient
dummymetron dummymetron: a program that accepts envelopes via UDP (v1) and gRPC (v2).
echo/cmd/http_server
echo/cmd/tcp_server
emitter emitter: a tool to emit envelopes to the agent via v2 gRPC.
envelopeemitter envelopeemitter: a tool to emit envelopes via v2 gRPC
experimental/data-dog-accumulator
https_drain
jsonspinner
latency
linter
linter/cmd/linter
log-cache-forwarders/cmd/datadog
log-cache-forwarders/cmd/space_syslog
log-cache-forwarders/cmd/space_syslog/internal/logcacheutil
log-cache-forwarders/cmd/syslog
log-cache-forwarders/pkg/egress/datadog
log-cache-forwarders/pkg/egress/syslog
log-cache-forwarders/pkg/egress/syslog/config
log-cache-forwarders/pkg/expvarfilter
log-cache-forwarders/pkg/groupmanager
log-cache-forwarders/pkg/metrics
log-cache-forwarders/pkg/sourceidprovider
log-cache-siege
log-cache-siege/internal/handlers
logspinner
metronreader Messages that have the origin "fast" will be aggregated and a rate will print every second.
postprinter
reliability/api
reliability/server
reliability/server/internal/api
reliability/worker
reliability/worker/internal/client
reliability/worker/internal/reporter
request-spinner
request-spinner/internal/handlers
rlpreader rlpreader: a tool that reads messages from RLP.
rlptypereader rlpreader: a tool that reads messages from RLP.
rlpwriter rlpwriter: a tool that writes messages into RLP.
slow_consumer
syslog-forwarder/cmd/syslog-forwarder
syslog-forwarder/internal/cloudcontroller
syslog-forwarder/internal/drain
syslog-forwarder/internal/egress
syslog-forwarder/internal/egress/config
syslog-forwarder/internal/metrics
syslog-forwarder/internal/stream
syslog-forwarder/internal/testhelper
syslog-nozzle
syslog-nozzle/app
syslog_drain
syslog_to_datadog
syslog_to_datadog/app
syslog_to_datadog/internal/processor
syslog_to_datadog/internal/web
udpwriter udpwriter: a tool that writes messages into a metron via UDP.
wsdestroyer