jsonlog

README

THOR structure definitions

Introduction

This library provides definitions of structures used in the output of the THOR APT Forensic Scanner. These structures can be used for different use cases:

• generate a schema for THOR JSON logs
• convert JSON logs into text logs
• parse JSON logs

Usage

Each object in the THOR log contains a type field that indicates the object type. This type determines how the object should be interpreted and what fields it contains.

Event Types

The object types contained in a THOR log are THOR finding and THOR message:

• Findings are the results of THOR's analysis, such as detected threats or anomalies.
• Messages are informational or status updates from THOR, such as progress updates.

Both findings and messages are together called events.

Parsing Events

There is a parser in the thorlog/parser package which can be used to parse an event independently of its type. This parser is also version aware, meaning it can handle different versions of the THOR log format.

Reportable Objects

Findings may contain more objects, e.g. as a subject that they report. Object types that can appear as subjects are called reportable objects. The most common reportable objects are:

• file
• process

Reportable objects should contain only fields that relate directly to the object itself. E.g. when extracting a file from an archive, the file object should contain only fields that relate to the file itself, not to the archive. The archive data will instead appear in the context of the finding.

Textlog conversion

The jsonlog.TextlogFormatter type provides a way to convert an object to a text log format.

This formatter can be used to convert findings and messages to a human-readable format. However, the text log format is not as rich as the JSON format and may not contain all fields. When in doubt, use the JSON format for analysis.

Schema

A schema for the THOR log format can be generated using the thorlog/jsonschema package.

Documentation

Index

• func ConcatTextLabels(prefix string, label string) string
• type JsonReferenceResolver
• type Object
• type ObjectHeader
  • func (l ObjectHeader) EmbeddedHeader() ObjectHeader
• type Reference
  • func NewReference(base Object, field any) *Reference
  • func (r Reference) JSONSchemaAlias() any
  • func (r Reference) MarshalJSON() ([]byte, error)
  • func (r *Reference) SetLabels(jsonPointer jsonpointer.Pointer, textLabel string)
  • func (r Reference) String() string
  • func (r *Reference) ToJsonPointer() jsonpointer.Pointer
  • func (r *Reference) ToTextLabel() string
  • func (r *Reference) UnmarshalJSON(data []byte) error
  • func (r Reference) Value() any
• type TextReferenceResolver
• type TextlogEntry
• type TextlogFormatter
  • func (t TextlogFormatter) Format(object any) TextlogEntry
• type TextlogMarshaler
• type TextlogValuePair

Functions

func ConcatTextLabels

func ConcatTextLabels(prefix string, label string) string

Types

type JsonReferenceResolver

type JsonReferenceResolver interface {
	// RelativeJsonPointer returns a pointer to the given field of the object that implements this interface.
	// If the field is not found, nil is returned.
	// The given field must be a pointer to a field of the object.
	RelativeJsonPointer(pointee any) jsonpointer.Pointer
}

JsonReferenceResolver is an interface that can be implemented by a struct to create custom JSON pointers to its fields. This interface should be implemented by log objects when the object has a custom MarshalJSON() and thus the default JSON pointer would not work.

type Object

type Object interface {
	// EmbeddedHeader returns the header of the log object.
	EmbeddedHeader() ObjectHeader
	// contains filtered or unexported methods
}

Object is the interface that all log objects must implement. Each log object has a `type` field in its JSON representation that is used to identify the object type.

type ObjectHeader

type ObjectHeader struct {
	// Type is the type of the object. It should be unique across all log objects
	// and can be used to identify the object type that has embedded this header.
	Type string `json:"type"`
}

ObjectHeader is the header of a log object. It must be embedded in all log objects.

func (ObjectHeader) EmbeddedHeader

func (l ObjectHeader) EmbeddedHeader() ObjectHeader

type Reference

type Reference struct {
	Base         any // Must be a pointer to a Object
	PointedField any // Must be a pointer to a (possibly nested) field of Base
	// contains filtered or unexported fields
}

Reference is a reference to a field of a Object

func NewReference

func NewReference(base Object, field any) *Reference

NewReference creates a new reference to a field of a Object. The base must be a pointer to a struct implementing Object. The field must be a pointer to a field within the base.

func (Reference) JSONSchemaAlias

func (r Reference) JSONSchemaAlias() any

func (Reference) MarshalJSON

func (r Reference) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON pointer to the pointed field as a JSON string.

func (*Reference) SetLabels

func (r *Reference) SetLabels(jsonPointer jsonpointer.Pointer, textLabel string)

SetLabels sets the JSON pointer and text label for the reference. This is an unsafe operation since it does not update the base and pointed field, nor does it check if the passed values match the pointed field. However, it can be used on a fresh reference where the labels are already known to avoid the overhead of looking them up.

func (Reference) String

func (r Reference) String() string

String returns the text label of the pointed field.

func (*Reference) ToJsonPointer

func (r *Reference) ToJsonPointer() jsonpointer.Pointer

ToJsonPointer returns a JSON pointer to the pointed field.

func (*Reference) ToTextLabel

func (r *Reference) ToTextLabel() string

ToTextLabel returns a text label for the pointed field.

func (*Reference) UnmarshalJSON

func (r *Reference) UnmarshalJSON(data []byte) error

func (Reference) Value

func (r Reference) Value() any

Value returns the pointed field.

type TextReferenceResolver

type TextReferenceResolver interface {
	// RelativeTextPointer returns a label for the given field of the object that implements this interface.
	// If the field is not found, nil is returned.
	// The given field must be a pointer to a field of the object.
	RelativeTextPointer(pointee any) (string, bool)
}

TextReferenceResolver is an interface that can be implemented by a struct to specify custom text labels for its fields that are used in references.

type TextlogEntry

type TextlogEntry []TextlogValuePair

TextlogEntry represents a single entry in a text log. It is a list of key-value pairs.

type TextlogFormatter

type TextlogFormatter struct {
	// FormatValue is a function that formats a single value into a string. If it is nil, fmt.Sprint is used.
	FormatValue func(data any, modifiers []string) string
	// Omit is a function that determines whether a field should be omitted from the log entry.
	// If it is nil, no fields are omitted.
	Omit func(modifiers []string, value any) bool
}

TextlogFormatter is a formatter for text logs. It provides a way to format log objects into text log entries. Text log keys are derived from the textlog tag in the struct field tags. If the tag is empty, the field is not marshalled. This tag can contain modifiers that control how the field is marshalled. Supported modifiers are:

• expand: causes the field to be expanded into its subfields. Each subfield is prefixed with this field's tag name.
• omitempty: causes the field to be omitted if it is the zero value or implements IsZero() and it returns true.
• explicit: causes the field to be marshalled even if the tag name is empty (requires that the containing struct has a non-empty tag name to avoid an empty text log key).

Slice and map values are marshalled as a list of key-value pairs. The key is the index (one-based) for slices and the map key for maps.

func (TextlogFormatter) Format

func (t TextlogFormatter) Format(object any) TextlogEntry

Format formats an object into a text log entry. The object must be a struct, pointer to a struct, slice, or map.

type TextlogMarshaler

type TextlogMarshaler interface {
	MarshalTextLog(formatter TextlogFormatter) TextlogEntry
}

type TextlogValuePair

type TextlogValuePair struct {
	Key   string
	Value string
}

TextlogValuePair is a key-value pair in a text log entry.