processing

package
v1.0.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 20, 2021 License: BSD-3-Clause Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// ContextQueueLength is the length of the queue buffering incoming context
	// bundles to balance out potential transmission delays.
	ContextQueueLength = 100
)
View Source 
const DebugOutputInterval = 100000

DebugOutputInterval specifies the amount of cache operations before printing the current cache size, in verbose mode.

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregateDNSEvent 

type AggregateDNSEvent struct {
	Timestamp []string             `json:"timestamp"`
	EventType string               `json:"event_type"`
	SrcIP     []string             `json:"src_ip,omitempty"`
	SrcPort   []int                `json:"src_port,omitempty"`
	DestIP    []string             `json:"dest_ip,omitempty"`
	DestPort  int                  `json:"dest_port,omitempty"`
	DNS       AggregatedDNSDetails `json:"dns,omitempty"`
}

AggregateDNSEvent holds aggregated flow data.

type AggregateDNSReplyDetails 

type AggregateDNSReplyDetails struct {
	Rrtype string `json:"rrtype,omitempty"`
	Rdata  string `json:"rdata,omitempty"`
	Rcode  string `json:"rcode,omitempty"`
	Type   string `json:"type,omitempty"`
}

AggregateDNSReplyDetails holds data for a query tuple.

type AggregateFlowEvent 

type AggregateFlowEvent struct {
	Timestamp []string              `json:"timestamp"`
	EventType string                `json:"event_type"`
	SrcIP     string                `json:"src_ip,omitempty"`
	SrcPort   []int                 `json:"src_port,omitempty"`
	DestIP    string                `json:"dest_ip,omitempty"`
	DestPort  int                   `json:"dest_port,omitempty"`
	Flow      AggregatedFlowDetails `json:"flow,omitempty"`
}

AggregateFlowEvent holds aggregated flow data.

type AggregatedDNSDetails 

type AggregatedDNSDetails struct {
	Rrname  string                     `json:"rrname,omitempty"`
	Details []AggregateDNSReplyDetails `json:"rdata,omitempty"`
}

AggregatedDNSDetails holds summarized traffic stats for a given AggregateDNSEvent.

type AggregatedFlowDetails 

type AggregatedFlowDetails struct {
	PktsToserver  int64 `json:"pkts_toserver"`
	PktsToclient  int64 `json:"pkts_toclient"`
	BytesToserver int64 `json:"bytes_toserver"`
	BytesToclient int64 `json:"bytes_toclient"`
}

AggregatedFlowDetails holds summarized traffic stats for a given AggregateFlowEvent.

type BloomHandler 

type BloomHandler struct {
	sync.Mutex
	Logger                *log.Entry
	Name                  string
	EventType             string
	IocBloom              *bloom.BloomFilter
	BloomFilename         string
	BloomFileIsCompressed bool
	DatabaseEventChan     chan types.Entry
	ForwardHandler        Handler
	DoForwardAlert        bool
	AlertPrefix           string
	Alertifier            *util.Alertifier
	BlacklistIOCs         map[string]struct{}
}

BloomHandler is a Handler which is meant to check for the presence of event type-specific keywords in a Bloom filter, raising new 'alert' type events when matches are found.

func MakeBloomHandler 

func MakeBloomHandler(iocBloom *bloom.BloomFilter,
	databaseChan chan types.Entry, forwardHandler Handler, alertPrefix string) *BloomHandler

MakeBloomHandler returns a new BloomHandler, checking against the given Bloom filter and sending alerts to databaseChan as well as forwarding them to a given forwarding handler.

func MakeBloomHandlerFromFile 

func MakeBloomHandlerFromFile(bloomFilename string, compressed bool,
	databaseChan chan types.Entry, forwardHandler Handler, alertPrefix string,
	blacklistIOCs []string) (*BloomHandler, error)

MakeBloomHandlerFromFile returns a new BloomHandler created from a new Bloom filter specified by the given file name.

func (*BloomHandler) Consume 

func (a *BloomHandler) Consume(e *types.Entry) error

Consume processes an Entry, emitting alerts if there is a match

func (*BloomHandler) GetEventTypes 

func (a *BloomHandler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*BloomHandler) GetName 

func (a *BloomHandler) GetName() string

GetName returns the name of the handler

func (*BloomHandler) Reload 

func (a *BloomHandler) Reload() error

Reload triggers a reload of the contents of the file with the name.

type BloomNoFileErr 

type BloomNoFileErr struct {
	// contains filtered or unexported fields
}

BloomNoFileErr is an error thrown when a file-based operation (e.g. reloading) is attempted on a bloom filter object with no file information attached.

func (*BloomNoFileErr) Error 

func (e *BloomNoFileErr) Error() string

Error returns the error message.

type ConcurrentHandler 

type ConcurrentHandler interface {
	Handler
	Run()
	Stop(chan bool)
}

ConcurrentHandler is an interface describing the behaviour for a component to handle events parsed from EVE input, while concurrently performing other actions, such as collecting, integrating and/or forwarding data.

type Context added in v1.0.8 

type Context []string

Context is a collection of JSON events that belong to a given flow.

type ContextChunk added in v1.0.8 

type ContextChunk struct {
	Timestamp time.Time     `json:"timestamp"`
	SensorID  string        `json:"sensor_id"`
	Events    []interface{} `json:"events"`
}

ContextChunk represents a collection of events for transmission via AMQP.

type ContextCollector added in v1.0.8 

type ContextCollector struct {
	PerfStats          ContextCollectorPerfStats
	StatsEncoder       *util.PerformanceStatsEncoder
	StopChan           chan bool
	StoppedChan        chan bool
	StopCounterChan    chan bool
	StoppedCounterChan chan bool
	Running            bool
	StatsLock          sync.Mutex
	FlowListeners      []chan types.Entry

	Cache    *cache.Cache
	MarkLock sync.Mutex
	Marked   map[string]struct{}
	Logger   *log.Entry

	Ship ContextShipper
	// contains filtered or unexported fields
}

ContextCollector is a component that maintains a cache of metadata per flow ID, forwarding it to a specified sink if associated with an alert. 

var GlobalContextCollector *ContextCollector

GlobalContextCollector is a shared ContextCollector to be used by FEVER.

func MakeContextCollector added in v1.0.8 

func MakeContextCollector(shipper ContextShipper, defaultTTL time.Duration) *ContextCollector

MakeContextCollector creates a new ContextCollector.

func (*ContextCollector) AddFlowListener added in v1.0.9 

func (c *ContextCollector) AddFlowListener(flowChan chan types.Entry)

AddFlowListener registers flowChan as a channel to emit a 'flow' Entry on whenever a marked flow is forwarded

func (*ContextCollector) Consume added in v1.0.8 

func (c *ContextCollector) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*ContextCollector) GetEventTypes added in v1.0.8 

func (c *ContextCollector) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*ContextCollector) GetName added in v1.0.8 

func (c *ContextCollector) GetName() string

GetName returns the name of the handler

func (*ContextCollector) Mark added in v1.0.8 

func (c *ContextCollector) Mark(flowID string)

Mark queues metadata for a given flow for forwarding, identified by its flow ID.

func (*ContextCollector) Run added in v1.0.8 

func (c *ContextCollector) Run()

Run starts the metrics collection and submission in the ContextCollector.

func (*ContextCollector) Stop added in v1.0.8 

func (c *ContextCollector) Stop(stoppedChan chan bool)

Stop stops the metrics collection and submission in the ContextCollector.

func (*ContextCollector) SubmitStats added in v1.0.8 

func (c *ContextCollector) SubmitStats(sc *util.PerformanceStatsEncoder)

SubmitStats registers a PerformanceStatsEncoder for runtime stats submission.

type ContextCollectorPerfStats added in v1.0.8 

type ContextCollectorPerfStats struct {
	Flows     uint64 `influx:"context_flows"`
	Events    uint64 `influx:"context_events"`
	JSONBytes uint64 `influx:"context_json_bytes"`
}

ContextCollectorPerfStats contains performance stats written to InfluxDB for monitoring.

type ContextShipper added in v1.0.8 

type ContextShipper func(Context, *log.Entry) error

ContextShipper is a function that processes a slice of Entries that make up a context of an alert, e.g. all events that share a flow ID relevant for the alert.

type ContextShipperAMQP added in v1.0.8 

type ContextShipperAMQP struct {
	Submitter util.StatsSubmitter
	InChan    chan Context
	SensorID  string
}

ContextShipperAMQP is a ContextShipper that sends incoming context bundles to an AMQP exchange.

func (*ContextShipperAMQP) Start added in v1.0.8 

func (cs *ContextShipperAMQP) Start(s util.StatsSubmitter) (chan<- Context, error)

Start initiates the concurrent handling of incoming context bundles in the Shipper's input channel. It will stop automatically once this channel is closed.

type DBHandler added in v1.0.4 

type DBHandler struct {
	OutChan chan types.Entry
}

DBHandler writes consumed events to a database.

func (*DBHandler) Consume added in v1.0.4 

func (h *DBHandler) Consume(e *types.Entry) error

Consume simply emits ths consumed entry on the default output channel

func (*DBHandler) GetEventTypes added in v1.0.4 

func (h *DBHandler) GetEventTypes() []string

GetEventTypes here is a dummy method -- since this handler is never registered we don't need to set this to an actual event type

func (*DBHandler) GetName added in v1.0.4 

func (h *DBHandler) GetName() string

GetName just returns the name of the default handler

type DNSAggregator 

type DNSAggregator struct {
	SensorID        string
	Count           int64
	DNSMutex        sync.RWMutex
	DNS             map[string]*AggregateDNSEvent
	PerfStats       DNSAggregatorPerfStats
	StatsEncoder    *util.PerformanceStatsEncoder
	SrcIPSet        map[string]bool
	DestIPSet       map[string]bool
	AnswerSet       map[string]bool
	StringBuf       bytes.Buffer
	FlushPeriod     time.Duration
	DatabaseOutChan chan types.Entry
	CloseChan       chan bool
	ClosedChan      chan bool
	Logger          *log.Entry
}

DNSAggregator is an aggregator that groups DNS events with the same domain name.

func MakeDNSAggregator 

func MakeDNSAggregator(flushPeriod time.Duration, outChan chan types.Entry) *DNSAggregator

MakeDNSAggregator creates a new empty DNSAggregator.

func (*DNSAggregator) Consume 

func (a *DNSAggregator) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*DNSAggregator) GetEventTypes 

func (a *DNSAggregator) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*DNSAggregator) GetName 

func (a *DNSAggregator) GetName() string

GetName returns the name of the handler

func (*DNSAggregator) Run 

func (a *DNSAggregator) Run()

Run starts the background aggregation service for this handler

func (*DNSAggregator) Stop 

func (a *DNSAggregator) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

func (*DNSAggregator) SubmitStats 

func (a *DNSAggregator) SubmitStats(sc *util.PerformanceStatsEncoder)

SubmitStats registers a PerformanceStatsEncoder for runtime stats submission.

type DNSAggregatorPerfStats 

type DNSAggregatorPerfStats struct {
	DNSAggregateRawCount uint64 `influx:"dns_aggregate_raw_count"`
	DNSAggregateCount    uint64 `influx:"dns_aggregate_count"`
}

DNSAggregatorPerfStats contains performance stats written to InfluxDB for monitoring.

type EventProfile 

type EventProfile struct {
	CountMap map[string]uint64
	SizeMap  map[string]uint64
}

EventProfile contains counts per event_type such as occurrences and JSON size.

type EventProfiler 

type EventProfiler struct {
	SensorID      string
	Host          string
	Profile       EventProfile
	FlushPeriod   time.Duration
	ProfileMutex  sync.Mutex
	CloseChan     chan bool
	ClosedChan    chan bool
	Logger        *log.Entry
	Submitter     util.StatsSubmitter
	SubmitChannel chan []byte
}

EventProfiler counts EVE event type statistics, such as number and size of JSON data received from the input.

func MakeEventProfiler 

func MakeEventProfiler(flushPeriod time.Duration, submitter util.StatsSubmitter) (*EventProfiler, error)

MakeEventProfiler creates a new EventProfiler.

func (*EventProfiler) Consume 

func (a *EventProfiler) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*EventProfiler) GetEventTypes 

func (a *EventProfiler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*EventProfiler) GetName 

func (a *EventProfiler) GetName() string

GetName returns the name of the handler

func (*EventProfiler) Run 

func (a *EventProfiler) Run()

Run starts the background aggregation service for this handler

func (*EventProfiler) Stop 

func (a *EventProfiler) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

type FlowAggregator 

type FlowAggregator struct {
	SensorID        string
	Count           int64
	FlowsMutex      sync.RWMutex
	Flows           map[string]*AggregateFlowEvent
	PerfStats       FlowAggregatorPerfStats
	StatsEncoder    *util.PerformanceStatsEncoder
	FlushPeriod     time.Duration
	StringBuf       bytes.Buffer
	DatabaseOutChan chan types.Entry
	CloseChan       chan bool
	ClosedChan      chan bool
	Logger          *log.Entry
}

FlowAggregator is an aggregator that groups flows with the same combination of srcIP/destIP/destPort.

func MakeFlowAggregator 

func MakeFlowAggregator(flushPeriod time.Duration, outChan chan types.Entry) *FlowAggregator

MakeFlowAggregator creates a new empty FlowAggregator.

func (*FlowAggregator) Consume 

func (a *FlowAggregator) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*FlowAggregator) GetEventTypes 

func (a *FlowAggregator) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*FlowAggregator) GetName 

func (a *FlowAggregator) GetName() string

GetName returns the name of the handler

func (*FlowAggregator) Run 

func (a *FlowAggregator) Run()

Run starts the background aggregation service for this handler

func (*FlowAggregator) Stop 

func (a *FlowAggregator) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

func (*FlowAggregator) SubmitStats 

func (a *FlowAggregator) SubmitStats(sc *util.PerformanceStatsEncoder)

SubmitStats registers a PerformanceStatsEncoder for runtime stats submission.

type FlowAggregatorPerfStats 

type FlowAggregatorPerfStats struct {
	FlowAggregateRawCount uint64 `influx:"flow_aggregate_raw_count"`
	FlowAggregateCount    uint64 `influx:"flow_aggregate_count"`
}

FlowAggregatorPerfStats contains performance stats written to InfluxDB for monitoring.

type FlowExtractor 

type FlowExtractor struct {
	SensorID    string
	BloomPath   string
	BloomFilter *bloom.BloomFilter
	FlowsMutex  sync.RWMutex

	Flows         *bytes.Buffer
	SubmitChannel chan []byte
	Submitter     util.StatsSubmitter
	FlushPeriod   time.Duration
	FlushCount    int
	CloseChan     chan bool
	ClosedChan    chan bool
	Logger        *log.Entry
	// contains filtered or unexported fields
}

FlowExtractor is an aggregator that extracts the flows from "hosts of interest" and sends them to the backend.

func MakeFlowExtractor 

func MakeFlowExtractor(flushPeriod time.Duration, flushCount int, bloomPath string, submitter util.StatsSubmitter) (*FlowExtractor, error)

MakeFlowExtractor creates a new empty FlowExtractor.

func (*FlowExtractor) Consume 

func (fe *FlowExtractor) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the flows

func (*FlowExtractor) GetEventTypes 

func (fe *FlowExtractor) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*FlowExtractor) GetName 

func (fe *FlowExtractor) GetName() string

GetName returns the name of the handler

func (*FlowExtractor) Run 

func (fe *FlowExtractor) Run()

Run starts the background aggregation service for this handler

func (*FlowExtractor) Stop 

func (fe *FlowExtractor) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

type FlowNotifier added in v1.0.9 

type FlowNotifier struct {
	FlowNotifyChan chan types.Entry
}

FlowNotifier is a handler that just passes flow events on a given channel once encountered.

func MakeFlowNotifier added in v1.0.9 

func MakeFlowNotifier(outChan chan types.Entry) *FlowNotifier

MakeFlowNotifier creates a new FlowNotifier.

func (*FlowNotifier) Consume added in v1.0.9 

func (n *FlowNotifier) Consume(e *types.Entry) error

Consume processes an Entry, emitting an Entry on the output channel

func (*FlowNotifier) GetEventTypes added in v1.0.9 

func (n *FlowNotifier) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to -- flow in this case.

func (*FlowNotifier) GetName added in v1.0.9 

func (n *FlowNotifier) GetName() string

GetName returns the name of the handler

type FlowProfiler added in v1.0.13 

type FlowProfiler struct {
	SensorID          string
	Host              string
	Profile           map[string]ProtoProfile
	FlushPeriod       time.Duration
	ProfileMutex      sync.Mutex
	CloseChan         chan bool
	ClosedChan        chan bool
	Logger            *log.Entry
	Submitter         util.StatsSubmitter
	SubmitChannel     chan []byte
	SubmitChannelFull bool
}

FlowProfiler counts EVE event type statistics, such as number and size of JSON data received from the input.

func MakeFlowProfiler added in v1.0.13 

func MakeFlowProfiler(flushPeriod time.Duration, submitter util.StatsSubmitter) (*FlowProfiler, error)

MakeFlowProfiler creates a new FlowProfiler.

func (*FlowProfiler) Consume added in v1.0.13 

func (a *FlowProfiler) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*FlowProfiler) GetEventTypes added in v1.0.13 

func (a *FlowProfiler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*FlowProfiler) GetName added in v1.0.13 

func (a *FlowProfiler) GetName() string

GetName returns the name of the handler

func (*FlowProfiler) Run added in v1.0.13 

func (a *FlowProfiler) Run()

Run starts the background aggregation service for this handler

func (*FlowProfiler) Stop added in v1.0.13 

func (a *FlowProfiler) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

type ForwardHandler 

type ForwardHandler struct {
	Logger              *log.Entry
	DoRDNS              bool
	RDNSHandler         *RDNSHandler
	AddedFields         map[string]string
	ContextCollector    *ContextCollector
	StenosisIface       string
	StenosisConnector   *StenosisConnector
	ForwardEventChan    chan []byte
	FlowNotifyChan      chan types.Entry
	OutputSocket        string
	OutputConn          net.Conn
	Reconnecting        bool
	ReconnLock          sync.Mutex
	ReconnectNotifyChan chan bool
	StopReconnectChan   chan bool
	ReconnectTimes      int
	PerfStats           ForwardHandlerPerfStats
	StatsEncoder        *util.PerformanceStatsEncoder
	StopChan            chan bool
	StoppedChan         chan bool
	StopCounterChan     chan bool
	StoppedCounterChan  chan bool
	Running             bool
	Lock                sync.Mutex
}

ForwardHandler is a handler that processes events by writing their JSON representation into a UNIX socket. This is limited by a list of allowed event types to be forwarded.

func MakeForwardHandler 

func MakeForwardHandler(reconnectTimes int, outputSocket string) *ForwardHandler

MakeForwardHandler creates a new forwarding handler

func (*ForwardHandler) AddFields added in v1.0.11 

func (fh *ForwardHandler) AddFields(fields map[string]string)

AddFields enables the addition of a custom set of top-level fields to the forwarded JSON.

func (*ForwardHandler) Consume 

func (fh *ForwardHandler) Consume(e *types.Entry) error

Consume processes an Entry and prepares it to be sent off to the forwarding sink

func (*ForwardHandler) EnableRDNS added in v1.0.6 

func (fh *ForwardHandler) EnableRDNS(expiryPeriod time.Duration)

EnableRDNS switches on reverse DNS enrichment for source and destination IPs in outgoing EVE events.

func (*ForwardHandler) EnableStenosis added in v1.0.9 

func (fh *ForwardHandler) EnableStenosis(endpoint string, timeout, timeBracket time.Duration,
	notifyChan chan types.Entry, cacheExpiry time.Duration, tlsConfig *tls.Config, iface string) (err error)

EnableStenosis ...

func (*ForwardHandler) GetEventTypes 

func (fh *ForwardHandler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*ForwardHandler) GetName 

func (fh *ForwardHandler) GetName() string

GetName returns the name of the handler

func (*ForwardHandler) Run 

func (fh *ForwardHandler) Run()

Run starts forwarding of JSON representations of all consumed events

func (*ForwardHandler) Stop 

func (fh *ForwardHandler) Stop(stoppedChan chan bool)

Stop stops forwarding of JSON representations of all consumed events

func (*ForwardHandler) SubmitStats 

func (fh *ForwardHandler) SubmitStats(sc *util.PerformanceStatsEncoder)

SubmitStats registers a PerformanceStatsEncoder for runtime stats submission.

type ForwardHandlerPerfStats 

type ForwardHandlerPerfStats struct {
	ForwardedPerSec uint64 `influx:"forwarded_events_per_sec"`
}

ForwardHandlerPerfStats contains performance stats written to InfluxDB for monitoring.

type Handler 

type Handler interface {
	GetEventTypes() []string
	GetName() string
	Consume(*types.Entry) error
}

Handler is an interface describing the behaviour for a component to handle events parsed from EVE input.

type HandlerDispatcher 

type HandlerDispatcher struct {
	Lock               sync.Mutex
	DispatchMap        map[string]([]Handler)
	DBHandler          Handler
	PerfStats          HandlerDispatcherPerfStats
	Logger             *log.Entry
	StatsEncoder       *util.PerformanceStatsEncoder
	StopCounterChan    chan bool
	StoppedCounterChan chan bool
}

HandlerDispatcher is a component to collect and properly apply a set of Handlers to a stream of Entry objects. Handlers can register the event types they are meant to act on and are called with relevant Entries to perform their job.

func MakeHandlerDispatcher 

func MakeHandlerDispatcher(databaseOut chan types.Entry) *HandlerDispatcher

MakeHandlerDispatcher returns a new HandlerDispatcher. The channel passed as an argument is used as an output channel for the default handler, which simply forwards events to a given channel (for example to be written to a database)

func (*HandlerDispatcher) Dispatch 

func (ad *HandlerDispatcher) Dispatch(e *types.Entry)

Dispatch applies the set of handlers currently registered in the dispatcher to the Entry object passed to it.

func (*HandlerDispatcher) RegisterHandler 

func (ad *HandlerDispatcher) RegisterHandler(agg Handler)

RegisterHandler adds the given Handler to the set of callbacks to be called on the relevant Entries received by the dispatcher.

func (*HandlerDispatcher) Run 

func (ad *HandlerDispatcher) Run()

Run starts the background service for this handler

func (*HandlerDispatcher) Stop 

func (ad *HandlerDispatcher) Stop(stopChan chan bool)

Stop causes the handler to cease counting and submitting data

func (*HandlerDispatcher) SubmitStats 

func (ad *HandlerDispatcher) SubmitStats(sc *util.PerformanceStatsEncoder)

SubmitStats registers a PerformanceStatsEncoder for runtime stats submission.

type HandlerDispatcherPerfStats 

type HandlerDispatcherPerfStats struct {
	DispatchedPerSec uint64 `influx:"dispatch_calls_per_sec"`
}

HandlerDispatcherPerfStats contains performance stats written to InfluxDB for monitoring.

type HeartbeatInjector added in v1.0.14 

type HeartbeatInjector struct {
	SensorID       string
	Times          []string
	CloseChan      chan bool
	Logger         *log.Entry
	ForwardHandler Handler
}

HeartbeatInjector regularly adds a date-based pseudo-event to the forwarded event stream.

func MakeHeartbeatInjector added in v1.0.14 

func MakeHeartbeatInjector(forwardHandler Handler, injectTimes []string) (*HeartbeatInjector, error)

MakeHeartbeatInjector creates a new HeartbeatInjector.

func (*HeartbeatInjector) Run added in v1.0.14 

func (a *HeartbeatInjector) Run()

Run starts the background service.

func (*HeartbeatInjector) Stop added in v1.0.14 

func (a *HeartbeatInjector) Stop()

Stop causes the service to cease the background work.

type IPAlertJSONProviderDstIP added in v1.0.12 

type IPAlertJSONProviderDstIP struct{}

IPAlertJSONProviderDstIP is an AlertJSONProvider for destination IP address matches.

func (IPAlertJSONProviderDstIP) GetAlertJSON added in v1.0.12 

func (a IPAlertJSONProviderDstIP) GetAlertJSON(inputEvent types.Entry,
	prefix string, ioc string) ([]byte, error)

GetAlertJSON returns the "alert" subobject for an alert EVE event.

type IPAlertJSONProviderSrcIP added in v1.0.12 

type IPAlertJSONProviderSrcIP struct{}

IPAlertJSONProviderSrcIP is an AlertJSONProvider for source IP address matches.

func (IPAlertJSONProviderSrcIP) GetAlertJSON added in v1.0.12 

func (a IPAlertJSONProviderSrcIP) GetAlertJSON(inputEvent types.Entry,
	prefix string, ioc string) ([]byte, error)

GetAlertJSON returns the "alert" subobject for an alert EVE event.

type IPHandler added in v1.0.3 

type IPHandler struct {
	sync.Mutex
	Logger            *log.Entry
	Name              string
	EventType         string
	Ranger            cidranger.Ranger
	IPListFilename    string
	DatabaseEventChan chan types.Entry
	ForwardHandler    Handler
	DoForwardAlert    bool
	AlertPrefix       string
	Alertifier        *util.Alertifier
}

IPHandler is a Handler which is meant to check for the presence of event type-specific keywords in a Bloom filter, raising new 'alert' type events when matches are found.

func MakeIPHandler added in v1.0.3 

func MakeIPHandler(ranger cidranger.Ranger,
	databaseChan chan types.Entry, forwardHandler Handler,
	alertPrefix string) *IPHandler

MakeIPHandler returns a new IPHandler, checking against the given IP ranges and sending alerts to databaseChan as well as forwarding them to a given forwarding handler.

func MakeIPHandlerFromFile added in v1.0.3 

func MakeIPHandlerFromFile(IPListFilename string,
	databaseChan chan types.Entry, forwardHandler Handler, alertPrefix string) (*IPHandler, error)

MakeIPHandlerFromFile returns a new IPHandler created from a new IP range list specified by the given file name.

func (*IPHandler) Consume added in v1.0.3 

func (a *IPHandler) Consume(e *types.Entry) error

Consume processes an Entry, emitting alerts if there is a match

func (*IPHandler) GetEventTypes added in v1.0.3 

func (a *IPHandler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*IPHandler) GetName added in v1.0.3 

func (a *IPHandler) GetName() string

GetName returns the name of the handler

func (*IPHandler) Reload added in v1.0.3 

func (a *IPHandler) Reload() error

Reload triggers a reload of the contents of the IP list file.

type PDNSCollector 

type PDNSCollector struct {
	SensorID      string
	Count         int64
	DNSMutex      sync.RWMutex
	DNS           pDNSEvent
	StringBuf     bytes.Buffer
	FlushPeriod   time.Duration
	CloseChan     chan bool
	ClosedChan    chan bool
	Logger        *log.Entry
	Submitter     util.StatsSubmitter
	SubmitChannel chan []byte
}

PDNSCollector extracts and aggregates DNS response data from EVE events and sends them to the backend.

func MakePDNSCollector 

func MakePDNSCollector(flushPeriod time.Duration, submitter util.StatsSubmitter) (*PDNSCollector, error)

MakePDNSCollector creates a new pDNSCollector.

func (*PDNSCollector) Consume 

func (a *PDNSCollector) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*PDNSCollector) GetEventTypes 

func (a *PDNSCollector) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*PDNSCollector) GetName 

func (a *PDNSCollector) GetName() string

GetName returns the name of the handler

func (*PDNSCollector) Run 

func (a *PDNSCollector) Run()

Run starts the background aggregation service for this handler

func (*PDNSCollector) Stop 

func (a *PDNSCollector) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

type ProtoProfile added in v1.0.13 

type ProtoProfile struct {
	PacketsToSrv uint64
	PacketsToClt uint64
	BytesToSrv   uint64
	BytesToClt   uint64
}

ProtoProfile contains flow statistics for a give app layer protocol.

type RDNSHandler added in v1.0.6 

type RDNSHandler struct {
	sync.Mutex
	Logger            *log.Entry
	HostNamer         util.HostNamer
	PrivateRanges     cidranger.Ranger
	PrivateRangesOnly bool
}

RDNSHandler is a handler that enriches events with reverse DNS information looked up on the sensor, for both source and destination IP addresses.

func MakeRDNSHandler added in v1.0.6 

func MakeRDNSHandler(hn util.HostNamer) *RDNSHandler

MakeRDNSHandler returns a new RDNSHandler, backed by the passed HostNamer.

func (*RDNSHandler) Consume added in v1.0.6 

func (a *RDNSHandler) Consume(e *types.Entry) error

Consume processes an Entry and enriches it

func (*RDNSHandler) EnableOnlyPrivateIPRanges added in v1.0.6 

func (a *RDNSHandler) EnableOnlyPrivateIPRanges()

EnableOnlyPrivateIPRanges ensures that only private (RFC1918) IP ranges are enriched

func (*RDNSHandler) GetEventTypes added in v1.0.6 

func (a *RDNSHandler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*RDNSHandler) GetName added in v1.0.6 

func (a *RDNSHandler) GetName() string

GetName returns the name of the handler

type StatsGeneratingHandler 

type StatsGeneratingHandler interface {
	Handler
	SubmitStats(*util.PerformanceStatsEncoder)
}

StatsGeneratingHandler is an interface describing a Handler which also periodically outputs performance statistics using the provided PerformanceStatsEncoder.

type StenosisConnector added in v1.0.9 

type StenosisConnector struct {
	Endpoint       string
	Client         api.StenosisServiceQueryClient
	TimeBracket    time.Duration
	Timeout        time.Duration
	ErrorCount     uint64
	FlowNotifyChan chan types.Entry
	Cache          *cache.Cache
}

StenosisConnector is a handler that caches alerts and waits for the associated flow to finish, then annotates all alerts with flow IDs and performs queries against a specified Stenosis server. Alerts will then be annotated with returned tokens and forwarded.

func MakeStenosisConnector added in v1.0.9 

func MakeStenosisConnector(endpoint string, timeout, timeBracket time.Duration,
	notifyChan chan types.Entry, forwardChan chan []byte, alertCacheExpiry time.Duration,
	tlsConfig *tls.Config) (*StenosisConnector, error)

MakeStenosisConnector returns a new StenosisConnector for the given parameters.

func (*StenosisConnector) Accept added in v1.0.9 

func (s *StenosisConnector) Accept(e *types.Entry)

Accept registers the given Entry into the connector's cache setup.

type UnicornAggregate 

type UnicornAggregate struct {
	SensorID       string                        `json:"sensor-id"`
	TimestampStart time.Time                     `json:"time-start"`
	TimestampEnd   time.Time                     `json:"time-end"`
	FlowTuples     map[string](map[string]int64) `json:"tuples"`
	ProxyMap       map[string](map[string]int64) `json:"proxy-map"`
}

UnicornAggregate represents UNICORN relevant aggregated flow stats.

func MakeUnicornAggregate 

func MakeUnicornAggregate() *UnicornAggregate

MakeUnicornAggregate creates a new empty UnicornAggregate object.

type UnicornAggregator 

type UnicornAggregator struct {
	Logger               *log.Entry
	Name                 string
	EventType            string
	Aggregate            UnicornAggregate
	Submitter            util.StatsSubmitter
	DummyMode            bool
	SubmitPeriod         time.Duration
	CloseChan            chan bool
	ClosedChan           chan bool
	StringBuf            bytes.Buffer
	UnicornTuplesMutex   sync.RWMutex `json:"-"`
	UnicornProxyMapMutex sync.RWMutex `json:"-"`
}

UnicornAggregator collects and updates an internal structure of flow events grouped by route

func MakeUnicornAggregator 

func MakeUnicornAggregator(statsSubmitter util.StatsSubmitter,
	submitPeriod time.Duration, dummyMode bool) *UnicornAggregator

MakeUnicornAggregator creates a new empty UnicornAggregator object.

func (*UnicornAggregator) Consume 

func (a *UnicornAggregator) Consume(e *types.Entry) error

Consume processes an Entry, adding the data within to the internal aggregated state

func (*UnicornAggregator) CountFlowTuple 

func (a *UnicornAggregator) CountFlowTuple(key string, bytestoclient int64,
	bytestoserver int64)

CountFlowTuple increments the flow tuple counter for the given key.

func (*UnicornAggregator) CountHTTPHost 

func (a *UnicornAggregator) CountHTTPHost(destip string, hostname string)

CountHTTPHost increments the count for the given IP-hostname pair.

func (*UnicornAggregator) GetEventTypes 

func (a *UnicornAggregator) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*UnicornAggregator) GetName 

func (a *UnicornAggregator) GetName() string

GetName returns the name of the handler

func (*UnicornAggregator) Run 

func (a *UnicornAggregator) Run()

Run starts the background aggregation service for this handler

func (*UnicornAggregator) Stop 

func (a *UnicornAggregator) Stop(stopChan chan bool)

Stop causes the aggregator to cease aggregating and submitting data

type VoidHandler added in v1.0.4 

type VoidHandler struct {
	Logger *log.Entry
}

VoidHandler is a handler that does nothing.

func MakeVoidHandler added in v1.0.4 

func MakeVoidHandler() *VoidHandler

MakeVoidHandler creates a new forwarding handler

func (*VoidHandler) Consume added in v1.0.4 

func (fh *VoidHandler) Consume(e *types.Entry) error

Consume processes an Entry and discards it

func (*VoidHandler) GetEventTypes added in v1.0.4 

func (fh *VoidHandler) GetEventTypes() []string

GetEventTypes returns a slice of event type strings that this handler should be applied to

func (*VoidHandler) GetName added in v1.0.4 

func (fh *VoidHandler) GetName() string

GetName returns the name of the handler

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.