ingestor

Documentation

Overview

ingestor package contains a code which helps to build a data ingestor for the aggregator

Index

• type Config
  • func NewDefaultConfig() *Config
  • func (ic *Config) Apply(ic1 *Config)
• type Ingestor
  • func NewIngestor(cfg *Config, ctx context.Context) (*Ingestor, error)
  • func (i *Ingestor) GetKnownTags() map[interface{}]interface{}
  • func (i *Ingestor) IsConnected() bool
  • func (i *Ingestor) Run(ctx context.Context, events <-chan *model.Event) chan bool
• type SchemaConfig
  • func NewDefaultSchemaConfigs() []*SchemaConfig
  • func (s *SchemaConfig) String() string

Types

type Config

type Config struct {
	Server           string          `json:"server"`
	RetrySec         int             `json:"retrySec"`
	HeartBeatMs      int             `json:"heartBeatMs"`
	PacketMaxRecords int             `json:"packetMaxRecords"`
	AccessKey        string          `json:"accessKey"`
	SecretKey        string          `json:"secretKey"`
	Schemas          []*SchemaConfig `json:"schemas"`
}

Config struct bears the ingestor configuration information. The ingestor will use it for sending data to the aggragator. The structure is used for configuring the ingestor

func NewDefaultConfig

func NewDefaultConfig() *Config

func (*Config) Apply

func (ic *Config) Apply(ic1 *Config)

Apply sets non-empty fields value from ic1 to ic

type Ingestor

type Ingestor struct {
	// contains filtered or unexported fields
}

Ingestor is used for sending data received from scanner to an log aggregator.

func NewIngestor

func NewIngestor(cfg *Config, ctx context.Context) (*Ingestor, error)

func (*Ingestor) GetKnownTags

func (i *Ingestor) GetKnownTags() map[interface{}]interface{}

func (*Ingestor) IsConnected

func (i *Ingestor) IsConnected() bool

func (*Ingestor) Run

func (i *Ingestor) Run(ctx context.Context, events <-chan *model.Event) chan bool

type SchemaConfig

type SchemaConfig struct {
	PathMatcher string            `json:"pathMatcher"`
	SourceId    string            `json:"sourceId"`
	Tags        map[string]string `json:"tags"`
}

SchemaConfig struct contains information by matching a file (PathMatcher) to the journal id (SourceId) it also contains tags that will be used for the match.

func NewDefaultSchemaConfigs

func NewDefaultSchemaConfigs() []*SchemaConfig

func (*SchemaConfig) String

func (s *SchemaConfig) String() string