nf9

Documentation

Overview

Package nf9 provides structures and functions to decode and analyze NetFlow v9 packets.

This package does only packet decoding in a single packet context. It keeps no state when decoding multiple packets. As a result Data FlowSets can not be decoded during initial packet decoding. To decode Data FlowSets user must keep track of Template Records and Options Template Records manually.

Examples of NetFlow v9 packets:

+--------+--------------------------------------------------------+
|        | +----------+ +---------+     +-----------+ +---------+ |
| Packet | | Template | | Data    |     | Options   | | Data    | |
| Header | | FlowSet  | | FlowSet | ... | Template  | | FlowSet | |
|        | |          | |         |     | FlowSet   | |         | |
|        | +----------+ +---------+     +-----------+ +---------+ |
+--------+--------------------------------------------------------+

+--------+----------------------------------------------+
|        | +---------+     +---------+      +---------+ |
| Packet | | Data    | ... | Data    | ...  | Data    | |
| Header | | FlowSet | ... | FlowSet | ...  | FlowSet | |
|        | +---------+     +---------+      +---------+ |
+--------+----------------------------------------------+

+--------+-------------------------------------------------+
|        | +----------+     +----------+      +----------+ |
| Packet | | Template |     | Template |      | Options  | |
| Header | | FlowSet  | ... | FlowSet  | ...  | Template | |
|        | |          |     |          |      | FlowSet  | |
|        | +----------+     +----------+      +----------+ |
+--------+-------------------------------------------------+

Example of struct hierarchy after packet decoding:

Package
|
+--TemplateFlowSet
|  |
|  +--TemplateRecord
|  |  |
|  |  +--Field
|  |  +--...
|  |  +--Field
|  |
|  +--...
|  |
|  +--TemplateRecord
|     |
|     +--Field
|     +--...
|     +--Field
|
+--DataFlowSet
|
+--...
|
+--OptionsTemplateFlowSet
|  |
|  +--OptionsTemplateRecord
|  |  |
|  |  +--Field (scope)
|  |  +--...   (scope)
|  |  +--Field (scope)
|  |  |
|  |  +--Field (option)
|  |  +--...   (option)
|  |  +--Field (option)
|  |
|  +--...
|  |
|  +--OptionsTemplateRecord
|     |
|     +--Field (scope)
|     +--...   (scope)
|     +--Field (scope)
|     |
|     +--Field (option)
|     +--...   (option)
|     +--Field (option)
|
+--DataFlowSet

When matched with appropriate template Data FlowSet can be decoded to list of Flow Data Records or list of Options Data Records. Struct hierarchy example:

[]FlowDataRecord
  |
  +--FlowDataRecord
  |  |
  |  +--[]byte
  |  +--...
  |  +--[]byte
  |
  +--...
  |
  +--FlowDataRecord
     |
     +--[]byte
     +--...
     +--[]byte

[]OptionsDataRecord
  |
  +--OptionsDataRecord
  |  |
  |  +--[]byte (scope)
  |  +--...    (scope)
  |  +--[]byte (scope)
  |  |
  |  +--[]byte (option)
  |  +--...    (option)
  |  +--[]byte (option)
  |
  +--...
  |
  +--OptionsDataRecord
     |
     +--[]byte
     +--...
     +--[]byte
     |
     +--[]byte (option)
     +--...    (option)
     +--[]byte (option)

Most of structure names and comments are taken directly from RFC 3954. Reading the NetFlow v9 protocol specification is highly recommended before using this package.

Index

• Constants
• func PrintHeader(p *Packet)
• type FlowDataRecord
  • func DecodeFlowSet(templateRecords []*TemplateRecord, set FlowSet) (list []FlowDataRecord)
• type FlowSet
• type FlowSetHeader
• type Header
• type OptionScope
• type OptionsTemplateRecordHeader
• type Packet
  • func Decode(raw []byte, remote net.IP) (*Packet, error)
  • func (p *Packet) DataFlowSets() []*FlowSet
  • func (p *Packet) GetTemplateRecords() []*TemplateRecords
• type TemplateRecord
• type TemplateRecordHeader
• type TemplateRecords

Constants

const (
	InBytes                   = 1
	InPkts                    = 2
	Flows                     = 3
	Protocol                  = 4
	SrcTos                    = 5
	TCPFlags                  = 6
	L4SrcPort                 = 7
	IPv4SrcAddr               = 8
	SrcMask                   = 9
	InputSnmp                 = 10
	L4DstPort                 = 11
	IPv4DstAddr               = 12
	DstMask                   = 13
	OutputSnmp                = 14
	IPv4NextHop               = 15
	SrcAs                     = 16
	DstAs                     = 17
	BGPIPv4NextHop            = 18
	MulDstPkts                = 19
	MulDstBytes               = 20
	LastSwitched              = 21
	FirstSwitched             = 22
	OutBytes                  = 23
	OutPkts                   = 24
	MinPktLngth               = 25
	MaxPktLngth               = 26
	IPv6SrcAddr               = 27
	IPv6DstAddr               = 28
	IPv6SrcMask               = 29
	IPv6DstMask               = 30
	IPv6FlowLabel             = 31
	IcmpType                  = 32
	MulIgmpType               = 33
	SamplingInterval          = 34
	SamplingAlgorithm         = 35
	FlowActiveTimeout         = 36
	FlowInactiveTimeout       = 37
	EngineType                = 38
	EngineID                  = 39
	TotalBytesExp             = 40
	TotalPktsExp              = 41
	TotalFlowsExp             = 42
	VendorProprietary43       = 43
	IPv4SrcPrefix             = 44
	IPv4DstPrefix             = 45
	MplsTopLabelType          = 46
	MplsTopLabelIPAddr        = 47
	FlowSamplerID             = 48
	FlowSamplerMode           = 49
	FlowSamplerRandomInterval = 50
	VendorProprietary51       = 51
	MinTTL                    = 52
	MaxTTL                    = 53
	IPv4Ident                 = 54
	DstTos                    = 55
	InSrcMac                  = 56
	OutDstMac                 = 57
	SrcVlan                   = 58
	DstVlan                   = 59
	IPProtocolVersion         = 60
	Direction                 = 61
	IPv6NextHop               = 62
	BgpIPv6NextHop            = 63
	IPv6OptionsHeaders        = 64
	VendorProprietary65       = 65
	VendorProprietary66       = 66
	VendorProprietary67       = 67
	VendorProprietary68       = 68
	VendorProprietary69       = 69
	MplsLabel1                = 70
	MplsLabel2                = 71
	MplsLabel3                = 72
	MplsLabel4                = 73
	MplsLabel5                = 74
	MplsLabel6                = 75
	MplsLabel7                = 76
	MplsLabel8                = 77
	MplsLabel9                = 78
	MplsLabel10               = 79
	InDstMac                  = 80
	OutSrcMac                 = 81
	IfName                    = 82
	IfDesc                    = 83
	SamplerName               = 84
	InPermanentBytes          = 85
	InPermanentPkts           = 86
	VendorProprietary87       = 87
	FragmentOffset            = 88
	ForwardingStatus          = 89
	MplsPalRd                 = 90
	MplsPrefixLen             = 91
	SrcTrafficIndex           = 92
	DstTrafficIndex           = 93
	ApplicationDescription    = 94
	ApplicationTag            = 95
	ApplicationName           = 96
)

const FlowSetIDTemplateMax = 255

FlowSetIDTemplateMax is the maximum FlowSetID being used for templates according to RFC3954

const OptionTemplateFlowSetID = 1

OptionTemplateFlowSetID is the FlowSetID reserved for option template flow sets

const TemplateFlowSetID = 0

TemplateFlowSetID is the FlowSetID reserved for template flow sets

Functions

func PrintHeader

func PrintHeader(p *Packet)

PrintHeader prints the header of `packet`

Types

type FlowDataRecord

type FlowDataRecord struct {
	// List of Flow Data Record values stored in raw format as []byte
	Values [][]byte
}

FlowDataRecord is actual NetFlow data. This structure does not contain any information about the actual data meaning. It must be combined with corresponding TemplateRecord to be decoded to a single NetFlow data row.

func DecodeFlowSet

func DecodeFlowSet(templateRecords []*TemplateRecord, set FlowSet) (list []FlowDataRecord)

DecodeFlowSet uses current TemplateRecord to decode data in Data FlowSet to a list of Flow Data Records.

type FlowSet

type FlowSet struct {
	Header *FlowSetHeader
	Flows  []byte
}

FlowSet represents a FlowSet as described in RFC3954

type FlowSetHeader

type FlowSetHeader struct {
	Length    uint16
	FlowSetID uint16
}

FlowSetHeader is a decoded representation of the header of a FlowSet

type Header

type Header struct {
	// A 32-bit value that identifies the Exporter Observation Domain.
	SourceID uint32

	// Incremental sequence counter of all Export Packets sent from the
	// current Observation Domain by the Exporter.
	//SequenceNumber uint32
	SequenceNumber uint32

	// Time in seconds since 0000 UTC 1970, at which the Export Packet
	// leaves the Exporter.
	//UnixSecs uint32
	UnixSecs uint32

	// Time in milliseconds since this device was first booted.
	//SysUpTime uint32
	SysUpTime uint32

	// The total number of records in the Export Packet, which is the sum
	// of Options FlowSet records, Template FlowSet records, and Data
	// FlowSet records.
	//Count uint16
	Count uint16

	// Version of Flow Record format exported in this packet. The value of
	//this field is 9 for the current version.
	//Version uint16
	Version uint16
}

Header is the NetFlow version 9 header

type OptionScope

type OptionScope struct {
	// The length (in bytes) of the Scope field, as it would appear in
	//an Options Data Record.
	ScopeFieldLength uint16

	//A numeric value that represents the type of field that would
	//appear in the Options Template Record.  Refer to the Field Type
	//Definitions section.
	ScopeFieldType uint16
}

OptionScope represents an option scope in an options template flowset

type OptionsTemplateRecordHeader

type OptionsTemplateRecordHeader struct {
	// The length (in bytes) of any options field definitions
	// contained in this Options Template Record.
	OptionLength uint16

	// Number of fields in this Template Record. Because a Template FlowSet
	// usually contains multiple Template Records, this field allows the
	// Collector to determine the end of the current Template Record and
	// the start of the next.
	OptionScopeLength uint16

	// Each of the newly generated Template Records is given a unique
	// Template ID. This uniqueness is local to the Observation Domain that
	// generated the Template ID. Template IDs of Data FlowSets are numbered
	// from 256 to 65535.
	TemplateID uint16
}

OptionsTemplateRecordHeader represents the header of an option template record

type Packet

type Packet struct {
	// A pointer to the packets headers
	Header *Header

	// A slice of pointers to FlowSet. Each element is instance of (Data)FlowSet
	// found in this packet
	FlowSets []*FlowSet

	// A slice of pointers to TemplateRecords. Each element is instance of TemplateRecords
	// representing a template found in this packet.
	Templates []*TemplateRecords

	// Buffer is a slice pointing to the original byte array that this packet was decoded from.
	// This field is only populated if debug level is at least 2
	Buffer []byte
}

Packet is a decoded representation of a single NetFlow v9 UDP packet.

func Decode

func Decode(raw []byte, remote net.IP) (*Packet, error)

Decode is the main function of this package. It converts raw packet bytes to Packet struct.

func (*Packet) DataFlowSets

func (p *Packet) DataFlowSets() []*FlowSet

DataFlowSets generate a list of all Data FlowSets in the packet. If matched with appropriate templates Data FlowSets can be decoded to Data Records or Options Data Records.

func (*Packet) GetTemplateRecords

func (p *Packet) GetTemplateRecords() []*TemplateRecords

GetTemplateRecords returns a list of all Template Records in the packet. Template Records can be used to decode Data FlowSets to Data Records.

type TemplateRecord

type TemplateRecord struct {
	// The length (in bytes) of the field.
	Length uint16

	// A numeric value that represents the type of field.
	Type uint16
}

TemplateRecord represents a Template Record as described in RFC3954

type TemplateRecordHeader

type TemplateRecordHeader struct {
	// Number of fields in this Template Record. Because a Template FlowSet
	// usually contains multiple Template Records, this field allows the
	// Collector to determine the end of the current Template Record and
	// the start of the next.
	FieldCount uint16

	// Each of the newly generated Template Records is given a unique
	// Template ID. This uniqueness is local to the Observation Domain that
	// generated the Template ID. Template IDs of Data FlowSets are numbered
	// from 256 to 65535.
	TemplateID uint16
}

TemplateRecordHeader represents the header of a template record

type TemplateRecords

type TemplateRecords struct {
	Header *TemplateRecordHeader

	// List of scopes
	OptionScopes []*OptionScope

	// List of fields in this Template Record.
	Records []*TemplateRecord

	Packet *Packet

	Values [][]byte
}

TemplateRecords is a single template that describes structure of a Flow Record (actual Netflow data).