payload

package
v0.0.0-...-4214274 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 4, 2021 License: MIT Imports: 11 Imported by: 0

README

Payload

Payload is a interface that is used to transfer data between processors.
A payload is a interface to allow different data to be transferd based on the handlers.

// Payload is a interface that will allows different Processors to send data between them in a unified fashion
type Payload interface {
	// GetPayloadLength returns the payload length in flota64
	GetPayloadLength() float64
	// GetPayload will return a byte array with the Payload from the ingress
	// Payload should be limited to 512 MB since thats the MAX cap for a redis payload
	// Also note that JSON payloads will be base64 encoded
	GetPayload() []byte
	// SetPayload will change the values of the payload
	SetPayload([]byte)
	// GetSource should return a string containing the name of the source, etc for a file its the filename or the recdis queue topic
	GetSource() string
	// SetSource should change the value of the source
	SetSource(string)
	// GetMetaData should return a configuration object that contains metadata about the payload
	GetMetaData() *property.Configuration
}

Currently available payloads are

Payload Description Filterable
BasePayload A simple payload used by most handlers, it is used when transfering a []byte is enough true
CsvPayload A Csv payload that contains information about the csv header aswell as the delimiter to decode the payload true
NetworkPayload A payload that holds network packets. The payload is a gopacket.Packet false

Documentation

Overview

Package payload contains Structs that fulfills payload interface

Package payload contains all the payload related stuff Payload is used to hold data between Processors, Payload is just an interface so each Processor could create its own Struct to handle data as long as it fulfills our interface

Index

Constants

This section is empty.

Variables

View Source 
var (
	//ErrFilterOrDirectory is thrown when neither filter or Directory to load is set
	ErrFilterOrDirectory error = errors.New("Either filter or filterDirectory property needs to be set, or both, but atleast one of them")
	//ErrEmptyFilterDirectory is when a empty filter directory is set
	ErrEmptyFilterDirectory error = errors.New("The filterdirectory value is empty ")
	//ErrBadFilterFormat is when a filter is being parsed but is not correct
	ErrBadFilterFormat error = errors.New("the filter seems to be poorly formatted, the format is key:regexp")
)
View Source 
var (
	// ErrPayloadIsNotANetworkPayload is when trying to convert a payload that does not fit the Payload of Network package
	ErrPayloadIsNotANetworkPayload = errors.New("this payload does not match the network.Payload type")
)

Functions

func LoadFilterDirectory 

func LoadFilterDirectory(path string) (map[string][]*Filter, error)

LoadFilterDirectory is used to load the filter directory into the handler

Types

type BasePayload 

type BasePayload struct {
	Payload  []byte                  `json:"payload"`
	Source   string                  `json:"source"`
	Metadata *property.Configuration `json:"metadata"`
}

BasePayload is a simple struct for processor to use if they dont have a custom payload

func NewBasePayload 

func NewBasePayload(payload []byte, source string, meta *property.Configuration) *BasePayload

NewBasePayload will spawn a basic default payload

func (*BasePayload) ApplyFilter 

func (bp *BasePayload) ApplyFilter(f *Filter) bool

ApplyFilter is used to make it part of the filterable interface

func (*BasePayload) GetMetaData 

func (bp *BasePayload) GetMetaData() *property.Configuration

GetMetaData returns a configuration object that can be used to store metadata

func (*BasePayload) GetPayload 

func (bp *BasePayload) GetPayload() []byte

GetPayload will return a payload without any processing

func (*BasePayload) GetPayloadLength 

func (bp *BasePayload) GetPayloadLength() float64

GetPayloadLength is used to get the number of bytes in a float

func (*BasePayload) GetSource 

func (bp *BasePayload) GetSource() string

GetSource returns the source of the payload

func (*BasePayload) MarshalBinary 

func (bp *BasePayload) MarshalBinary() ([]byte, error)

MarshalBinary is used to marshal the whole payload into a Byte array This is particullary used to enable Redis Pub/Sub

func (*BasePayload) SetPayload 

func (bp *BasePayload) SetPayload(p []byte)

SetPayload changes the payload

func (*BasePayload) SetSource 

func (bp *BasePayload) SetSource(s string)

SetSource will change the value of the payload source

func (*BasePayload) UnmarshalBinary 

func (bp *BasePayload) UnmarshalBinary(data []byte) error

UnmarshalBinary is used to Decode a byte array into the proper fields In base payloads case its simple JSON

type CsvPayload 

type CsvPayload struct {
	Payload   string                  `json:"payload"`
	Header    string                  `json:"header"`
	Delimiter string                  `json:"delimiter"`
	Source    string                  `json:"source"`
	Error     error                   `json:"error"`
	Metadata  *property.Configuration `json:"metadata"`
}

CsvPayload is a struct representing Csv data as a map Its also a part of the Payload interface

func NewCsvPayload 

func NewCsvPayload(header, payload, delimiter string, meta *property.Configuration) *CsvPayload

NewCsvPayload is used to Create a new Payload

func (*CsvPayload) ApplyFilter 

func (nf *CsvPayload) ApplyFilter(f *Filter) bool

ApplyFilter is used to make this part of the Filterable interface

func (*CsvPayload) GetMetaData 

func (nf *CsvPayload) GetMetaData() *property.Configuration

GetMetaData returns a configuration object that can be used to store metadata

func (*CsvPayload) GetPayload 

func (nf *CsvPayload) GetPayload() []byte

GetPayload is used to return an actual value for the Flow Csv hedaer will be appended with a newline aswell

func (*CsvPayload) GetPayloadLength 

func (nf *CsvPayload) GetPayloadLength() float64

GetPayloadLength will return the payload X Bytes

func (*CsvPayload) GetSource 

func (nf *CsvPayload) GetSource() string

GetSource will return the source of the flow

func (*CsvPayload) MarshalBinary 

func (nf *CsvPayload) MarshalBinary() ([]byte, error)

MarshalBinary is used to marshal the whole payload into a Byte array This is particullary used to enable Redis Pub/Sub

func (*CsvPayload) SetPayload 

func (nf *CsvPayload) SetPayload(newpayload []byte)

SetPayload will change the value of the Flow

func (*CsvPayload) SetSource 

func (nf *CsvPayload) SetSource(s string)

SetSource will change the value of the configured source The source value should represent something that makes it possible to traceback Errors, so for files etc its the filename.

func (*CsvPayload) UnmarshalBinary 

func (nf *CsvPayload) UnmarshalBinary(data []byte) error

UnmarshalBinary is used to Decode a byte array into the proper fields In base payloads case its simple JSON

type Filter 

type Filter struct {
	// Regexp is the regexp to apply
	Regexp *regexp.Regexp
	// Key is the field to apply the Regexp to, like in csv it resembles the header field value.
	Key string
	// Groupname is a reference to the group that the filter belongs too
	GroupName string
}

Filter is a struct used to check if some data is part of this filter Will use Regexp as a placeholder for the filter value, but regexp can hold regular strings aswell so it works both ways

func ParseFilterLine 

func ParseFilterLine(line string) (*Filter, error)

ParseFilterLine is used to parse out a filter line

type Filterable 

type Filterable interface {
	ApplyFilter(f *Filter) bool
}

Filterable is a interface that is used to apply Filters to payloads

type NetworkPayload 

type NetworkPayload struct {
	Payload  gopacket.Packet         `json:"payload"`
	Source   string                  `json:"source"`
	Error    error                   `json:"error"`
	Metadata *property.Configuration `json:"-"`
}

NetworkPayload is a struct representing pcap.PacketSource as NetworkPayload Its also a part of the Payload interface

func NewNetworkPayload 

func NewNetworkPayload(pay Payload) (*NetworkPayload, error)

NewNetworkPayload is used to convert a regular payload into a network payload

func (*NetworkPayload) GetMetaData 

func (nf *NetworkPayload) GetMetaData() *property.Configuration

GetMetaData returns a configuration object that can be used to store metadata

func (*NetworkPayload) GetPayload 

func (nf *NetworkPayload) GetPayload() []byte

GetPayload is used to return an actual value for the Flow

func (*NetworkPayload) GetPayloadLength 

func (nf *NetworkPayload) GetPayloadLength() float64

GetPayloadLength will return the payload X Bytes

func (*NetworkPayload) GetSource 

func (nf *NetworkPayload) GetSource() string

GetSource will return the source of the flow

func (*NetworkPayload) MarshalBinary 

func (nf *NetworkPayload) MarshalBinary() ([]byte, error)

MarshalBinary is used to marshal the whole payload into a Byte array This is particullary used to enable Redis Pub/Sub

func (*NetworkPayload) SetPayload 

func (nf *NetworkPayload) SetPayload(newpayload []byte)

SetPayload will change the value of the Flow

func (*NetworkPayload) SetSource 

func (nf *NetworkPayload) SetSource(s string)

SetSource will change the value of the configured source The source value should represent something that makes it possible to traceback Errors, so for files etc its the filename.

func (*NetworkPayload) UnmarshalBinary 

func (nf *NetworkPayload) UnmarshalBinary(data []byte) error

UnmarshalBinary is used to Decode a byte array into the proper fields In base payloads case its simple JSON

type Payload 

type Payload interface {
	// GetPayloadLength returns the payload length in flota64
	GetPayloadLength() float64
	// GetPayload will return a byte array with the Payload from the ingress
	// Payload should be limited to 512 MB since thats the MAX cap for a redis payload
	// Also note that JSON payloads will be base64 encoded
	GetPayload() []byte
	// SetPayload will change the values of the payload
	SetPayload([]byte)
	// GetMetaData should return a configuration object that contains metadata about the payload
	GetMetaData() *property.Configuration
	// Force Payloads to also be part of the Encoding package interfaces
	// This is needed for Redis purpose
	encoding.BinaryMarshaler
	encoding.BinaryUnmarshaler
}

Payload is a interface that will allows different Processors to send data between them in a unified fashion

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.