message

Documentation

Index

• Constants
• Variables
• func StatusToSeverity(status string) []byte
• type BasicStructuredContent
  • func (m *BasicStructuredContent) GetContent() []byte
  • func (m *BasicStructuredContent) Render() ([]byte, error)
  • func (m *BasicStructuredContent) SetContent(content []byte)
• type Lambda
• type Message
  • func NewMessage(content []byte, origin *Origin, status string, ingestionTimestamp int64) *Message
  • func NewMessageFromLambda(content []byte, origin *Origin, status string, utcTime time.Time, ...) *Message
  • func NewMessageWithSource(content []byte, status string, source *sources.LogSource, ...) *Message
  • func NewStructuredMessage(content StructuredContent, origin *Origin, status string, ...) *Message
  • func (m *Message) GetLatency() int64
  • func (m *Message) GetStatus() string
  • func (m *Message) Render() ([]byte, error)
  • func (m *Message) Tags() []string
  • func (m *Message) TagsToString() string
• type MessageContent
  • func (m *MessageContent) GetContent() []byte
  • func (m *MessageContent) SetContent(content []byte)
  • func (m *MessageContent) SetEncoded(content []byte)
  • func (m *MessageContent) SetRendered(content []byte)
• type MessageContentState
• type Origin
  • func NewOrigin(source *sources.LogSource) *Origin
  • func (o *Origin) Service() string
  • func (o *Origin) SetService(service string)
  • func (o *Origin) SetSource(source string)
  • func (o *Origin) SetTags(tags []string)
  • func (o *Origin) Source() string
  • func (o *Origin) Tags(processingTags []string) []string
  • func (o *Origin) TagsPayload(processingTags []string) []byte
  • func (o *Origin) TagsToString(processingTags []string) string
• type ParsingExtra
• type Payload
• type ServerlessExtra
• type StructuredContent

Constants

const (
	StatusEmergency = "emergency"
	StatusAlert     = "alert"
	StatusCritical  = "critical"
	StatusError     = "error"
	StatusWarning   = "warn"
	StatusNotice    = "notice"
	StatusInfo      = "info"
	StatusDebug     = "debug"
)

Status values

Variables

var (
	SevEmergency = []byte("<40>")
	SevAlert     = []byte("<41>")
	SevCritical  = []byte("<42>")
	SevError     = []byte("<43>")
	SevWarning   = []byte("<44>")
	SevNotice    = []byte("<45>")
	SevInfo      = []byte("<46>")
	SevDebug     = []byte("<47>")
)

Syslog severity levels

Functions

func StatusToSeverity

func StatusToSeverity(status string) []byte

StatusToSeverity transforms a severity into a status.

Types

type BasicStructuredContent

type BasicStructuredContent struct {
	Data map[string]interface{}
}

BasicStructuredContent is used by tailers creating structured logs but with basic needs for transport. The message from the log is stored in the "message" key.

func (*BasicStructuredContent) GetContent

func (m *BasicStructuredContent) GetContent() []byte

GetContent returns the message part of the structured log, in the "message" key of the underlying map.

func (*BasicStructuredContent) Render

func (m *BasicStructuredContent) Render() ([]byte, error)

Render renders in json the underlying data, it is then ready to be encoded and sent to the intake. See the `MessageContent` comment.

func (*BasicStructuredContent) SetContent

func (m *BasicStructuredContent) SetContent(content []byte)

SetContent stores the message part of the structured log, in the "message" key of the underlying map.

type Lambda

type Lambda struct {
	ARN       string
	RequestID string
}

Lambda is a struct storing information about the Lambda function and function execution.

type Message

type Message struct {
	MessageContent
	Hostname           string
	Origin             *Origin
	Status             string
	IngestionTimestamp int64
	RawDataLen         int
	// Tags added on processing
	ProcessingTags []string
	// Extra information from the parsers
	ParsingExtra
	// Extra information for Serverless Logs messages
	ServerlessExtra
}

Message represents a log line sent to datadog, with its metadata

func NewMessage

func NewMessage(content []byte, origin *Origin, status string, ingestionTimestamp int64) *Message

NewMessage constructs an unstructured message with content, status, origin and the ingestion timestamp.

func NewMessageFromLambda

func NewMessageFromLambda(content []byte, origin *Origin, status string, utcTime time.Time, ARN, reqID string, ingestionTimestamp int64) *Message

NewMessageFromLambda construts a message with content, status, origin and with the given timestamp and Lambda metadata.

func NewMessageWithSource

func NewMessageWithSource(content []byte, status string, source *sources.LogSource, ingestionTimestamp int64) *Message

NewMessageWithSource constructs an unstructured message with content, status and a log source.

func NewStructuredMessage

func NewStructuredMessage(content StructuredContent, origin *Origin, status string, ingestionTimestamp int64) *Message

NewStructuredMessage creates a new message that had some structure the moment it has been captured through a tailer. e.g. a journald message which is a JSON object containing extra information, including the actual message of the entry. We need these objects to be able to apply processing on the message entry only, while we still have to send all the information to the intake.

func (*Message) GetLatency

func (m *Message) GetLatency() int64

GetLatency returns the latency delta from ingestion time until now

func (*Message) GetStatus

func (m *Message) GetStatus() string

GetStatus gets the status of the message. if status is not set, StatusInfo will be returned.

func (*Message) Render

func (m *Message) Render() ([]byte, error)

Render renders the message. The only state in which this call is changing the content for a StateStructured message.

func (*Message) Tags

func (m *Message) Tags() []string

Message returns all tags that this message is attached with.

func (*Message) TagsToString

func (m *Message) TagsToString() string

Message returns all tags that this message is attached with, as a string.

type MessageContent

type MessageContent struct {
	State MessageContentState
	// contains filtered or unexported fields
}

MessageContent contains the message and possibly the tailer internal representation of every message.

To use the MessageContent struct, use `GetContent() []byte` or SetContent([]byte)` makes sure of doing the right thing depending on the MessageContent state.

MessageContent different states:

+-------------------+
| StateUnstructured | ------
+-------------------+      |
                           |
                           v
                      ( Processor )    +---------------+    ( Processor )    +--------------+
                      (  Renders  ) -> | StateRendered | -> (  Encodes  ) -> | StateEncoded |
                           ^           +---------------+                     +--------------+
                           |                 |
+-------------------+      |                 v
|  StateStructured  | ------          (   Diagnostic   )
+-------------------+                 (Message Receiver)

In `StateUnstructured`, the content in `Content` is the raw log collected by the tailer. In `StateStructured`, `Content` is empty and the log information are in `StructuredContent`. In `StateRendered`, `Content` contains rendered data (from raw/structured logs to something ready to be encoded), the rest should not be used. In `StateEncoded`, `Content` contains the encoded data, the rest should not be used.

Note that there is no state distinction between parsed and unparsed content as none was needed for the current implementation, but it is a potential future change with a `StateParsed` state.

func (*MessageContent) GetContent

func (m *MessageContent) GetContent() []byte

GetContent returns the bytes array containing only the message content E.g. from a structured log:

Sep 12 14:38:14 user my-app[1316]: time="2023-09-12T14:38:14Z" level=info msg="Starting the main execution"

It would only return the `[]byte` containing "Starting the main execution" While for unstructured log and for source configured with ProcessRawMessage=true, the whole `[]byte` content is returned. See `MessageContent` comment for more information as this method could also return the message content in different state (rendered, encoded).

func (*MessageContent) SetContent

func (m *MessageContent) SetContent(content []byte)

SetContent stores the given content as the content message. SetContent uses the current message state to know where to store the content.

func (*MessageContent) SetEncoded

func (m *MessageContent) SetEncoded(content []byte)

SetEncoded sets the content for the MessageContent and sets MessageContent state to encoded.

func (*MessageContent) SetRendered

func (m *MessageContent) SetRendered(content []byte)

SetRendered sets the content for the MessageContent and sets MessageContent state to rendered.

type MessageContentState

type MessageContentState uint32 // nolint:revive

MessageContentState is used to represent the MessageContent state.

const (
	// StateUnstructured for unstructured content (e.g. file tailing)
	StateUnstructured MessageContentState = iota
	// StateStructured for structured content (e.g. journald tailing, windowsevent tailing)
	StateStructured
	// StateRendered means that the MessageContent contains rendered (i.e. structured content has been rendered)
	StateRendered
	// StateEncoded means the MessageContent passed through the encoder (e.g. json encoder, proto encoder, ...)
	StateEncoded
)

type Origin

type Origin struct {
	Identifier string
	LogSource  *sources.LogSource
	Offset     string
	// contains filtered or unexported fields
}

Origin represents the Origin of a message

func NewOrigin

func NewOrigin(source *sources.LogSource) *Origin

NewOrigin returns a new Origin

func (*Origin) Service

func (o *Origin) Service() string

Service returns the service of the configuration if set or the service of the message, if none are defined, returns an empty string by default.

func (*Origin) SetService

func (o *Origin) SetService(service string)

SetService sets the service of the origin.

func (*Origin) SetSource

func (o *Origin) SetSource(source string)

SetSource sets the source of the origin.

func (*Origin) SetTags

func (o *Origin) SetTags(tags []string)

SetTags sets the tags of the origin.

func (*Origin) Source

func (o *Origin) Source() string

Source returns the source of the configuration if set or the source of the message, if none are defined, returns an empty string by default.

func (*Origin) Tags

func (o *Origin) Tags(processingTags []string) []string

Tags returns the tags of the origin.

The returned slice must not be modified by the caller.

func (*Origin) TagsPayload

func (o *Origin) TagsPayload(processingTags []string) []byte

TagsPayload returns the raw tag payload of the origin.

func (*Origin) TagsToString

func (o *Origin) TagsToString(processingTags []string) string

TagsToString encodes tags to a single string, in a comma separated format

type ParsingExtra

type ParsingExtra struct {
	// Used by docker parsers to transmit an offset.
	Timestamp string
	IsPartial bool
}

ParsingExtra ships extra information parsers want to make available to the rest of the pipeline. E.g. Timestamp is used by the docker parsers to transmit a tailing offset.

type Payload

type Payload struct {
	// The slice of sources messages encoded in the payload
	Messages []*Message
	// The encoded bytes to be sent to the intake (sometimes compressed)
	Encoded []byte
	// The content encoding. A header for HTTP, empty for TCP
	Encoding string
	// The size of the unencoded payload
	UnencodedSize int
}

Payload represents an encoded collection of messages ready to be sent to the intake

type ServerlessExtra

type ServerlessExtra struct {
	// Optional. Must be UTC. If not provided, time.Now().UTC() will be used
	// Used in the Serverless Agent
	Timestamp time.Time
	// Optional.
	// Used in the Serverless Agent
	Lambda *Lambda
}

ServerlessExtra ships extra information from logs processing in serverless envs.

type StructuredContent

type StructuredContent interface {
	Render() ([]byte, error)
	GetContent() []byte
	SetContent([]byte)
}

StructuredContent stores enough information from a tailer to manipulate a structured log message (from journald or windowsevents) and to render it to be encoded later on in the pipeline.