ipfix

Documentation

Overview

Package ipfix provides structures and functions to decode and analyze IPFIX packets.

This package does only packet decoding in a single packet context. It keeps no state when decoding multiple packets. As a result Data FlowSets can not be decoded during initial packet decoding. To decode Data FlowSets user must keep track of Template Records and Options Template Records manually.

Examples of IPFIX packets:

1. An IPFIX Message consisting of interleaved Template, Data, and Options Template Sets, as shown in Figure C. Here, Template and Options Template Sets are transmitted "on demand", before the first Data Set whose structure they define.

   +--------+--------------------------------------------------------+ | | +----------+ +---------+ +-----------+ +---------+ | |Message | | Template | | Data | | Options | | Data | | | Header | | Set | | Set | ... | Template | | Set | | | | | | | | | Set | | | | | | +----------+ +---------+ +-----------+ +---------+ | +--------+--------------------------------------------------------+

   +--------+----------------------------------------------+ | | +---------+ +---------+ +---------+ | |Message | | Data | | Data | | Data | | | Header | | Set | ... | Set | ... | Set | | | | +---------+ +---------+ +---------+ | +--------+----------------------------------------------+

   Figure D: IPFIX Message: Example 2

3. An IPFIX Message consisting entirely of Template and Options Template Sets, as shown in Figure E. Such a message can be used to define or redefine Templates and Options Templates in bulk.

   +--------+-------------------------------------------------+ | | +----------+ +----------+ +----------+ | |Message | | Template | | Template | | Options | | | Header | | Set | ... | Set | ... | Template | | | | | | | | | Set | | | | +----------+ +----------+ +----------+ | +--------+-------------------------------------------------+

Example of struct hierarchy after packet decoding:

Package
|
+--TemplateFlowSet
|  |
|  +--TemplateRecord
|  |  |
|  |  +--Field
|  |  +--...
|  |  +--Field
|  |
|  +--...
|  |
|  +--TemplateRecord
|     |
|     +--Field
|     +--...
|     +--Field
|
+--DataFlowSet
|
+--...
|
+--OptionsTemplateFlowSet
|  |
|  +--OptionsTemplateRecord
|  |  |
|  |  +--Field (scope)
|  |  +--...   (scope)
|  |  +--Field (scope)
|  |  |
|  |  +--Field (option)
|  |  +--...   (option)
|  |  +--Field (option)
|  |
|  +--...
|  |
|  +--OptionsTemplateRecord
|     |
|     +--Field (scope)
|     +--...   (scope)
|     +--Field (scope)
|     |
|     +--Field (option)
|     +--...   (option)
|     +--Field (option)
|
+--DataFlowSet

When matched with appropriate template Data FlowSet can be decoded to list of Flow Data Records or list of Options Data Records. Struct hierarchy example:

[]FlowDataRecord
  |
  +--FlowDataRecord
  |  |
  |  +--[]byte
  |  +--...
  |  +--[]byte
  |
  +--...
  |
  +--FlowDataRecord
     |
     +--[]byte
     +--...
     +--[]byte

[]OptionsDataRecord
  |
  +--OptionsDataRecord
  |  |
  |  +--[]byte (scope)
  |  +--...    (scope)
  |  +--[]byte (scope)
  |  |
  |  +--[]byte (option)
  |  +--...    (option)
  |  +--[]byte (option)
  |
  +--...
  |
  +--OptionsDataRecord
     |
     +--[]byte
     +--...
     +--[]byte
     |
     +--[]byte (option)
     +--...    (option)
     +--[]byte (option)

Most of structure names and comments are taken directly from RFC 7011. Reading the IPFIX protocol specification is highly recommended before using this package.

Index

• Constants
• func PrintHeader(p *Packet)
• type FlowDataRecord
• type Header
• type Packet
  • func Decode(raw []byte) (*Packet, error)
  • func (p *Packet) DataFlowSets() []*Set
  • func (p *Packet) GetTemplateRecords() []*TemplateRecords
• type Set
• type SetHeader
• type TemplateRecord
• type TemplateRecordHeader
• type TemplateRecords
  • func (dtpl *TemplateRecords) DecodeFlowSet(set Set) (list []FlowDataRecord)

Constants

const (
	InBytes                   = 1
	InPkts                    = 2
	Flows                     = 3
	Protocol                  = 4
	SrcTos                    = 5
	TCPFlags                  = 6
	L4SrcPort                 = 7
	IPv4SrcAddr               = 8
	SrcMask                   = 9
	InputSnmp                 = 10
	L4DstPort                 = 11
	IPv4DstAddr               = 12
	DstMask                   = 13
	OutputSnmp                = 14
	IPv4NextHop               = 15
	SrcAs                     = 16
	DstAs                     = 17
	BGPIPv4NextHop            = 18
	MulDstPkts                = 19
	MulDstBytes               = 20
	LastSwitched              = 21
	FirstSwitched             = 22
	OutBytes                  = 23
	OutPkts                   = 24
	MinPktLngth               = 25
	MaxPktLngth               = 26
	IPv6SrcAddr               = 27
	IPv6DstAddr               = 28
	IPv6SrcMask               = 29
	IPv6DstMask               = 30
	IPv6FlowLabel             = 31
	IcmpType                  = 32
	MulIgmpType               = 33
	SamplingInterval          = 34
	SamplingAlgorithm         = 35
	FlowActiveTimeout         = 36
	FlowInactiveTimeout       = 37
	EngineType                = 38
	EngineID                  = 39
	TotalBytesExp             = 40
	TotalPktsExp              = 41
	TotalFlowsExp             = 42
	VendorProprietary43       = 43
	IPv4SrcPrefix             = 44
	IPv4DstPrefix             = 45
	MplsTopLabelType          = 46
	MplsTopLabelIPAddr        = 47
	FlowSamplerID             = 48
	FlowSamplerMode           = 49
	FlowSamplerRandomInterval = 50
	VendorProprietary51       = 51
	MinTTL                    = 52
	MaxTTL                    = 53
	IPv4Ident                 = 54
	DstTos                    = 55
	InSrcMac                  = 56
	OutDstMac                 = 57
	SrcVlan                   = 58
	DstVlan                   = 59
	IPProtocolVersion         = 60
	Direction                 = 61
	IPv6NextHop               = 62
	BgpIPv6NextHop            = 63
	IPv6OptionsHeaders        = 64
	VendorProprietary65       = 65
	VendorProprietary66       = 66
	VendorProprietary67       = 67
	VendorProprietary68       = 68
	VendorProprietary69       = 69
	MplsLabel1                = 70
	MplsLabel2                = 71
	MplsLabel3                = 72
	MplsLabel4                = 73
	MplsLabel5                = 74
	MplsLabel6                = 75
	MplsLabel7                = 76
	MplsLabel8                = 77
	MplsLabel9                = 78
	MplsLabel10               = 79
	InDstMac                  = 80
	OutSrcMac                 = 81
	IfName                    = 82
	IfDesc                    = 83
	SamplerName               = 84
	InPermanentBytes          = 85
	InPermanentPkts           = 86
	VendorProprietary87       = 87
	FragmentOffset            = 88
	ForwardingStatus          = 89
	MplsPalRd                 = 90
	MplsPrefixLen             = 91
	SrcTrafficIndex           = 92
	DstTrafficIndex           = 93
	ApplicationDescription    = 94
	ApplicationTag            = 95
	ApplicationName           = 96
	SamplingPacketInterval    = 305
)

const SetIDTemplateMax = 255

SetIDTemplateMax is the maximum FlowSetID being used for templates according to RFC3954

const TemplateSetID = 2

TemplateSetID is the set ID reserved for template sets

Functions

func PrintHeader

func PrintHeader(p *Packet)

PrintHeader prints the header of `packet`

Types

type FlowDataRecord

type FlowDataRecord struct {
	// List of Flow Data Record values stored in raw format as []byte
	Values [][]byte
}

FlowDataRecord is actual NetFlow data. This structure does not contain any information about the actual data meaning. It must be combined with corresponding TemplateRecord to be decoded to a single NetFlow data row.

type Header

type Header struct {
	// A 32-bit value that identifies the Exporter Observation Domain.
	DomainID uint32

	// Incremental sequence counter of all Export Packets sent from the
	// current Observation Domain by the Exporter.
	SequenceNumber uint32

	// Time in seconds since 0000 UTC 1970, at which the Export Packet
	// leaves the Exporter.
	ExportTime uint32

	// The total number of bytes in this Export Packet
	Length uint16

	// Version of Flow Record format exported in this packet. The value of
	//this field is 9 for the current version.
	Version uint16
}

Header is an IPFIX message header

type Packet

type Packet struct {
	// A pointer to the packets headers
	Header *Header

	// A slice of pointers to FlowSet. Each element is instance of (Data)FlowSet
	// found in this packet
	FlowSets []*Set

	// A slice of pointers to TemplateRecords. Each element is instance of TemplateRecords
	// representing a template found in this packet.
	Templates []*TemplateRecords

	// Buffer is a slice pointing to the original byte array that this packet was decoded from.
	// This field is only populated if debug level is at least 2
	Buffer []byte
}

Packet is a decoded representation of a single NetFlow v9 UDP packet.

func Decode

func Decode(raw []byte) (*Packet, error)

Decode is the main function of this package. It converts raw packet bytes to Packet struct.

func (*Packet) DataFlowSets

func (p *Packet) DataFlowSets() []*Set

DataFlowSets generate a list of all Data FlowSets in the packet. If matched with appropriate templates Data FlowSets can be decoded to Data Records or Options Data Records.

func (*Packet) GetTemplateRecords

func (p *Packet) GetTemplateRecords() []*TemplateRecords

GetTemplateRecords generate a list of all Template Records in the packet. Template Records can be used to decode Data FlowSets to Data Records.

type Set

type Set struct {
	Header  *SetHeader
	Records []byte
}

Set represents a Set as described in RFC7011

type SetHeader

type SetHeader struct {
	Length uint16
	SetID  uint16
}

SetHeader is a decoded representation of the header of a Set

type TemplateRecord

type TemplateRecord struct {
	// The length (in bytes) of the field.
	Length uint16

	// A numeric value that represents the type of field.
	Type uint16
}

TemplateRecord represents a Template Record as described in RFC3954

type TemplateRecordHeader

type TemplateRecordHeader struct {
	// Number of fields in this Template Record. Because a Template FlowSet
	// usually contains multiple Template Records, this field allows the
	// Collector to determine the end of the current Template Record and
	// the start of the next.
	FieldCount uint16

	// Each of the newly generated Template Records is given a unique
	// Template ID. This uniqueness is local to the Observation Domain that
	// generated the Template ID. Template IDs of Data FlowSets are numbered
	// from 256 to 65535.
	TemplateID uint16
}

TemplateRecordHeader represents the header of a template record

type TemplateRecords

type TemplateRecords struct {
	Header *TemplateRecordHeader

	// List of fields in this Template Record.
	Records []*TemplateRecord

	Packet *Packet

	Values [][]byte
}

TemplateRecords is a single template that describes structure of a Flow Record (actual Netflow data).

func (*TemplateRecords) DecodeFlowSet

func (dtpl *TemplateRecords) DecodeFlowSet(set Set) (list []FlowDataRecord)

DecodeFlowSet uses current TemplateRecord to decode data in Data FlowSet to a list of Flow Data Records.