Documentation

Overview

    Package collector implements the LogDog Collector daemon's log parsing and registration logic.

    The LogDog Collector is responsible for ingesting logs from the Butler sent via transport, stashing them in intermediate storage, and registering them with the LogDog Coordinator service.

    Index

    Constants

    View Source
    const (
    	// DefaultMaxMessageWorkers is the default number of concurrent worker
    	// goroutones to employ for a single message.
    	DefaultMaxMessageWorkers = 4
    )

    Variables

    This section is empty.

    Functions

    This section is empty.

    Types

    type Collector

    type Collector struct {
    	// Coordinator is used to interface with the Coordinator client.
    	//
    	// On production systems, this should wrapped with a caching client (see
    	// the stateCache sub-package) to avoid overwhelming the server.
    	Coordinator coordinator.Coordinator
    
    	// Storage is the intermediate storage instance to use.
    	Storage storage.Storage
    
    	// StreamStateCacheExpire is the maximum amount of time that a cached stream
    	// state entry is valid. If zero, DefaultStreamStateCacheExpire will be used.
    	StreamStateCacheExpire time.Duration
    
    	// MaxMessageWorkers is the maximum number of concurrent workers to employ
    	// for any given message. If <= 0, DefaultMaxMessageWorkers will be applied.
    	MaxMessageWorkers int
    }

      Collector is a stateful object responsible for ingesting LogDog logs, registering them with a Coordinator, and stowing them in short-term storage for streaming and processing.

      A Collector's Close should be called when finished to release any internal resources.

      func (*Collector) Close

      func (c *Collector) Close()

        Close releases any internal resources and blocks pending the completion of any outstanding operations. After Close, no new Process calls may be made.

        func (*Collector) Process

        func (c *Collector) Process(ctx context.Context, msg []byte) error

          Process ingests an encoded ButlerLogBundle message, registering it with the LogDog Coordinator and stowing it in a temporary Storage for streaming retrieval.

          If a transient error occurs during ingest, Process will return an error. If no error occurred, or if there was an error with the input data, no error will be returned.

          Directories

          Path Synopsis
          Package coordinator implements a minimal interface to the Coordinator service that is sufficient for Collector usage.
          Package coordinator implements a minimal interface to the Coordinator service that is sufficient for Collector usage.