Documentation
¶
Overview ¶
Package receiver provides various skogul Receivers that accept data and execute a handler. They are the "inbound" API of Skogul.
Index ¶
Examples ¶
Constants ¶
const (
UDP_MAX_READ_SIZE = 65535
)
Variables ¶
var Auto skogul.ModuleMap
Auto maps names to Receivers to allow auto configuration
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
File string `doc:"Path to the file to read from once."`
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
// contains filtered or unexported fields
}
File reads from a FILE, a single JSON object per line, and exits at EOF.
type HTTP ¶
type HTTP struct {
Address string `doc:"Address to listen to." example:"[::1]:80 [2001:db8::1]:443"`
Handlers map[string]*skogul.HandlerRef `doc:"Paths to handlers. Need at least one." example:"{\"/\": \"someHandler\" }"`
Auth map[string]*HTTPAuth `doc:"A map corresponding to Handlers; specifying authentication for the given path, if required."`
Certfile string `doc:"Path to certificate file for TLS. If left blank, un-encrypted HTTP is used."`
Keyfile string `doc:"Path to key file for TLS."`
ClientCertificateCAs []string `doc:"Paths to files containing CAs which are accepted for Client Certificate authentication."`
Log204OK bool `` /* 141-byte string literal not displayed */
// contains filtered or unexported fields
}
HTTP accepts HTTP connections on the Address specified, and requires at least one handler to be set up, using Handle. This is done implicitly if the HTTP receiver is created using New()
Example ¶
HTTP can have different skogul.Handler's for different paths, with potentially different behaviors.
package main
import (
"github.com/telenornms/skogul"
"github.com/telenornms/skogul/parser"
"github.com/telenornms/skogul/receiver"
"github.com/telenornms/skogul/sender"
"github.com/telenornms/skogul/transformer"
)
func main() {
h := receiver.HTTP{Address: "localhost:8080"}
template := skogul.Handler{Transformers: []skogul.Transformer{transformer.Templater{}}, Sender: &sender.Debug{}}
template.SetParser(parser.SkogulJSON{})
noTemplate := skogul.Handler{Sender: &sender.Debug{}}
noTemplate.SetParser(parser.SkogulJSON{})
h.Handlers = map[string]*skogul.HandlerRef{
"/template": {H: &template},
"/notemplate": {H: &noTemplate},
}
h.Start()
}
Output:
type HTTPAuth ¶ added in v0.3.0
type HTTPAuth struct {
Username string `doc:"Username for basic authentication. No authentication is required if left blank."`
Password skogul.Secret `doc:"Password for basic authentication."`
SANDNSName string `doc:"DNS name which has to be present in SAN extension of x509 certificate when using Client Certificate authentication"`
SkipCertificateVerify bool `doc:"Skip verifying certificate. (default: false)"`
// contains filtered or unexported fields
}
HTTPAuth contains ways to authenticate a HTTP request, e.g. Username/Password for Basic Auth.
type Kafka ¶ added in v0.16.0
type Kafka struct {
Topic string `doc:"Topic to read from."`
Brokers []string `doc:"Array of brokeraddresses."`
Handler skogul.HandlerRef `doc:"Handler to use"`
TLS bool `doc:"Enable TLS, off by default."`
Username string `doc:"Username for SASL auth."`
Password string `doc:"Password for SASL auth."`
ClientID string `doc:"ClientID to use - uses lower-case skogul by default."`
}
Kafka receiver is a MVP-variant, and further features are reasonable and expected, including but not limited to:
- Authentication (coming before release) - Dynamic keys from metadata - Adjustment of various timeouts
type LineFile ¶
type LineFile struct {
File string `doc:"Path to the fifo or file from which to read from repeatedly."`
Handler skogul.HandlerRef `doc:"Handler used to parse and transform and send data."`
Delay skogul.Duration `doc:"Delay before re-opening the file, if any."`
}
LineFile will keep reading File over and over again, assuming one collection per line. Best suited for pointing at a FIFO, which will allow you to 'cat' stuff to Skogul.
type LineFileAdvanced ¶ added in v0.20.0
type LineFileAdvanced struct {
File string `doc:"Path to the fifo or file from which to read from repeatedly."`
NewFile string `doc:"Path to the fifo or file that 'File'will be moved to."`
Handler skogul.HandlerRef `doc:"Handler used to parse and transform and send data."`
Delay skogul.Duration `doc:"Delay before re-opening the file, if any."`
Pre string `doc:"Command to run AFTER moving the file, but BEFORE reading it. E.g.: Sighup/reload."`
Post string `doc:"Shell command to execute after reading file is finished."`
Shell string `doc:"Shell used to execute post command. Default: /bin/sh -c"`
}
func (*LineFileAdvanced) Start ¶ added in v0.20.0
func (lf *LineFileAdvanced) Start() error
Start never returns.
type LogrusLog ¶ added in v0.3.0
type LogrusLog struct {
Loglevel string
Handler skogul.HandlerRef
}
LogrusLog configures the logrus log receiver
type LogrusSkogulHook ¶ added in v0.3.0
LogrusSkogulHook is a logrus.Hook made for skogul
func (*LogrusSkogulHook) Fire ¶ added in v0.3.0
func (hook *LogrusSkogulHook) Fire(entry *logrus.Entry) error
Fire implements the logrus.Hook interface and handles each log entry
func (*LogrusSkogulHook) Levels ¶ added in v0.3.0
func (hook *LogrusSkogulHook) Levels() []logrus.Level
Levels returns the log levels the hook should care about
type MQTT ¶
type MQTT struct {
Broker string `doc:"Address of broker to connect to." example:"[::1]:8888"`
Topics []string `doc:"List of topics to subscribe to"`
Handler *skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
Password string `doc:"Username for authenticating to the broker."`
Username string `doc:"Password for authenticating."`
ClientID string `doc:"Custom client id to use (default: random)"`
RenewClientID bool `` /* 140-byte string literal not displayed */
DisplayMQTTLogs bool
// contains filtered or unexported fields
}
MQTT connects to a MQTT broker and listens for messages on a topic.
type Nats ¶ added in v0.21.0
type Nats struct {
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
Servers string `doc:"Comma separated list of nats URLs"`
Subject string `doc:"Subject to subscribe to messages on"`
Queue string `doc:"Worker queue to distribute messages on"`
Name string `doc:"Client name"`
Username string `doc:"Client username"`
Password string `doc:"Client password"`
TLSClientKey string `doc:"TLS client key file path"`
TLSClientCert string `doc:"TLS client cert file path"`
TLSCACert string `doc:"CA cert file path"`
UserCreds string `doc:"Nats credentials file path"`
NKeyFile string `doc:"Nats nkey file path"`
Insecure bool `doc:"TLS InsecureSkipVerify"`
// contains filtered or unexported fields
}
Nats basic pub/sub receiver implementing all Authentication & Authorization features in the nats golang client. Basic queue groups is also supported.
type Rabbitmq ¶ added in v0.24.0
type Rabbitmq struct {
Username skogul.Secret `doc:"Username for rabbitmq instance"`
Password skogul.Secret `doc:"Password for rabbitmq instance"`
Host string `doc:"Hostname for rabbitmq instance. Fallback is localhost"`
Port string `doc:"Port for rabbitmq instance. Fallback is 5672"`
Queue string `doc:"Queue to read from"`
Handler *skogul.HandlerRef `doc:"Handler used to parse, transform and send data. Default skogul."`
}
type SQL ¶ added in v0.17.0
type SQL struct {
ConnStr string `` /* 377-byte string literal not displayed */
Query string `doc:"Query run for each metric. Any column named 'time' will be used as the metric time stamp."`
Metadata []string `doc:"Array of which columns to treat as metadata, the rest will be data fields."`
Driver string `doc:"Database driver/system. Currently suported: mysql and postgres."`
Interval skogul.Duration `doc:"How often to run the query. Set to negative value to run it just once."`
Handler skogul.HandlerRef `doc:"Handler to use for data transmission."`
UnmarshalJson []string `doc:"Unmarshal fields containing json strings into objects "`
}
type Stats ¶ added in v0.12.0
type Stats struct {
Handler *skogul.HandlerRef
ChanSize int
// contains filtered or unexported fields
}
Stats receives metrics from skogul and forwards it to a handler.
func (*Stats) Start ¶ added in v0.12.0
Start starts listening for Skogul stats and emits them on the configured interval.
type Stdin ¶
type Stdin struct {
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
// contains filtered or unexported fields
}
Stdin reads from /dev/stdin
type TCPLine ¶
type TCPLine struct {
Address string `doc:"Address and port to listen to." example:"[::1]:3306"`
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
}
TCPLine listens on a IP:TCP port specified in the Address string and accepts one container per line to be sent to the parser.
Example usage, assuming JSON parser:
$ cat payloads/simple.json | jq -c . | nc '::1' '1234'
Since this is not possible to secure, it should be avoided where possible and placed as close to the data source. A good use of this model is to use a TCPLine receiver on the same box that needs to write to it, combined with skogul.senders.HTTP to forward over a more sensible channel.
type Tester ¶
type Tester struct {
Metrics int64 `doc:"Number of metrics in each container"`
Values int64 `doc:"Number of unique values for each metric"`
Threads int `doc:"Threads to spawn"`
Delay skogul.Duration `doc:"Sleep time between each metric is generated, if any."`
Handler skogul.HandlerRef `doc:"Reference to a handler where the data is sent"`
// contains filtered or unexported fields
}
Tester synthesise dummy data.
type UDP ¶
type UDP struct {
Address string `doc:"Address and port to listen to." example:"[::1]:3306"`
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
Backlog int `` /* 333-byte string literal not displayed */
Threads int `` /* 227-byte string literal not displayed */
PacketSize int `doc:"UDP Packet size note: max. UDP read size is 65535"`
FailureLevel string `doc:"Level to log receiver failures as. Error, Warning, Info, Debug, or Trace. (default: Error)"`
Buffer int `doc:"Set kernel read buffer. Default is kernel-specific. Bumping this will make it easier to handler bursty UDP traffic."`
EmitStats skogul.Duration `doc:"How often to emit internal skogul stats for this receiver. Default: 10s (from stats.DefaultInterval)"`
// contains filtered or unexported fields
}
UDP contains the configuration for the receiver
func (*UDP) GetStats ¶ added in v0.12.0
GetStats prepares a skogul metric with stats for the UDP receiver.
type WholeFile ¶ added in v0.16.0
type WholeFile struct {
File string `doc:"Path to the file to read from."`
Handler skogul.HandlerRef `doc:"Handler used to parse, transform and send data."`
Frequency skogul.Duration `doc:"How often to re-read the same file. Leave blank or set to a negative value to only read once."`
}
WholeFile reads the whole file and parses it as a single container
Source Files