idmef

README

IDMEF for Go

Intrusion Detection Message Exchange Format

Go library for authoring and parsing data in IDMEF format (IETF RFC 4765).

Usage

There are two sets of Message structs, one for authoring and one for parsing. The reason is due to Go's lack of support for parsing XML with tag prefixes.

Authoring

Use the go-idmef (idmef) package structs to create the idmef.Message struct and then call xml.Marshal() or idmef.Message.Bytes().

Example messages from the RFC are available in the testdata folder in both XML and Go code. These are compared in the tests](unmarshal/)

Parsing

See unmarshal.ReadFile() function for an example to parse aa IDMEF XML file.

Coverage

• IDMEF-Message
  • Alert
    • Analyzer
      • Node
        • Address
      • Process
    • CreateTime
    • DetectTime
    • AnalyzerTime
    • Source
      • Node
        • Address
      • User
        • UserId
      • Process
      • Service
    • Target
      • Node
        • Address
      • User
        • UserId
      • Process
      • Service
      • File
        • FileAccess
          • Permission
        • Linkage
    • Classification
      • Reference
    • Assessment
    • CorrelationAlert
    • AdditionalData
  • Heartbeat
    • Analyzer
    • CreateTime
    • AdditionalData

Examples

The examples in RFC 4765 are included and tested in this repo. Go and XML representations are provided, parsed and compared. The following is a lists of the examples in RFC 4765. RFC descriptions are provided.

• Denial-of-Service Attacks: The following examples show how some common denial-of-service attacks could be represented in the IDMEF.
  • The "teardrop" Attack (xml): Network-based detection of the "teardrop" attack. This shows the basic format of an alert.
  • The "ping of death" Attack (xml): Network-based detection of the "ping of death" attack. Note the identification of multiple targets, and the identification of the source as a spoofed address. NOTE: The URL has been cut to fit the IETF formating requirements.
• Port Scanning Attacks: The following examples show how some common port scanning attacks could be represented in the IDMEF.
  • Connection to a Disallowed Service (xml): Host-based detection of a policy violation (attempt to obtain information via "finger"). Note the identification of the target service, as well as the originating user (obtained, e.g., through RFC 1413).
  • Simple Port Scanning (xml): Network-based detection of a port scan. This shows detection by a single analyzer; see Section 7.5 for the same attack as detected by a correlation engine. Note the use of to show the ports that were scanned.
• Local Attacks: The following examples show how some common local host attacks could be represented in the IDMEF.
  • The "loadmodule" Attack (xml): Host-based detection of the "loadmodule" exploit. This attack involves tricking the "loadmodule" program into running another program; since "loadmodule" is set-user-id "root", the executed program runs with super-user privileges. Note the use of and to identify the user attempting the exploit and how he's doing it.
  • The "loadmodule" Attack with root target user (xml): The Intrusion Detection System (IDS) could also indicate that the target user is the "root" user, and show the attempted command; the alert might then look like:
  • The "phf" Attack (xml): Network-based detection of the "phf" attack. Note the use of the element to provide more details about this particular attack.
  • File Modification (xml): Host-based detection of a race condition attack. Note the use of the to provide information about the files that are used to perform the attack.
• System Policy Violation (xml): In this example, logins are restricted to daytime hours. The alert reports a violation of this policy that occurs when a user logs in a little after 10:00 pm. Note the use of to provide information about the policy being violated.
• Correlated Alerts (xml): The following example shows how the port scan alert from Section 7.2.2 could be represented if it had been detected and sent from a correlation engine, instead of a single analyzer.
• Analyzer Assessments (xml): Host-based detection of a successful unauthorized acquisition of root access through the eject buffer overflow. Note the use of to provide information about the analyzer's evaluation of and reaction to the attack.
• Heartbeat (xml): This example shows a Heartbeat message that provides "I'm alive and working" information to the manager. Note the use of elements, with "meaning" attributes, to provide some additional information.

Notes

1. idmef is the authoring package and creates XML with the idmef tag.
2. unmarshal is the parsing package which reads in XML files but does not support the idmef tag prefix due to Go issue 9519. Unmarshal or parse a file using unmarshal to receive a *unmarshal.Message which can then be converted to an authoring struct with *unmarshal.Message.Common().

References

IDMEF

• IETF RFC 4765: Format Details
• IETF RFC 4766: Format Requirements
• IETF RFC 4767: Recommended Transport Protocol (IDXP)
• IDMEF on Wikipedia
• XML Schema-Based Minification for Communication of Security Information and Event
• awesome-idmef

Alternative Formats

• OSSEC: https://www.ossec.net/docs/formats/alerts.html
• OSSEM: https://github.com/OTRF/OSSEM

Go XML situation

1. encoding/xml: support for XML namespace prefixes
2. xml namespace prefix issue at go: "To fix that you need to use two structs, one for Unmarshalling and second to Marshalling data"
3. Unable to parse xml in GO with : in tags

Other Implementations

1. PHP - https://github.com/fpoirotte/php-idmef
2. Protocol Buffers - https://github.com/DaGuich/IDMEF_protobuf

Comparisons

1. Power of the IDMEF format
2. SDEE vs IDMEF?
3. IDMEF 'Lingua Franca' for Security Incident Management by Douglas Corner
4. Security Log Standard: Still an Open Question

Credits

1. timestamp.Timestamp is based on code from github.com/coreos/mantle under the Apache 2.0 license and MIT compatible. This is a large, archived codebase with many dependencies.
2. diffmatchpatch from github.com/sergi/go-diff was used during development to analyze failed test results.

Documentation

Index

• Constants
• type Action
• type AdditionalData
• type Address
• type Alert
• type AlertIdent
• type Analyzer
• type Assessment
• type Classification
• type Confidence
• type CorrelationAlert
• type File
• type FileAccess
• type Heartbeat
• type Impact
• type Linkage
• type Message
  • func (m *Message) Bytes(prefix, indent string) ([]byte, error)
• type Node
• type Permission
• type Process
• type Reference
• type Service
• type Source
• type Target
• type Time
  • func NewTime(t time.Time) Time
  • func (t *Time) InflateNtpStamp()
• type User
• type UserID
• type WebService

Constants

const (
	IPV4Addr    = "ipv4-addr"
	IPV4AddrHex = "ipv4-addr-hex"
	IPV4NetMask = "ipv4-net-mask"

	CategoryOSDevice = "os-device"

	LocationConsole = "console"
	LocationLocal   = "local"

	ServiceDNS     = "dns"
	ServiceFinger  = "finger"
	ServiceLogin   = "login"
	ServiceNIS     = "nis"
	ServiceNISPlus = "nisplus"

	UserIDTypeCurrentUser  = "current-user"
	UserIDTypeOriginalUser = "original-user"
	UserIDTypeTargetUser   = "target-user"
	UserIDTypeGroupPrivs   = "group-privs"
	UserIDTypeUserPrivs    = "user-privs"

	OriginUserSpecific   = "user-specific"
	OriginVendorSpecific = "vendor-specific"

	DateTime  = "date-time"
	StartTime = "start-time"
	StopTime  = "stop-time"

	TypeReal = "real"
)

const (
	XMLNSIDMEFURL     = "http://iana.org/idmef"
	XMLNSIDMEFVersion = "1.0"
)

Types

type Action

type Action struct {
	Action   string `xml:",chardata"`
	Category string `xml:"category,attr,omitempty"`
}

type AdditionalData

type AdditionalData struct {
	Type     string    `xml:"type,attr,omitempty"`
	Meaning  string    `xml:"meaning,attr,omitempty"`
	DateTime time.Time `xml:"idmef:date-time,omitempty"`
	Real     *float64  `xml:"idmef:real"`
}

type Address

type Address struct {
	Ident    string `xml:"ident,attr,omitempty"`
	Category string `xml:"category,attr,omitempty"`
	Address  string `xml:"idmef:address,omitempty"`
	Netmask  string `xml:"idmef:netmask,omitempty"`
}

type Alert

type Alert struct {
	MessageID        string            `xml:"messageid,attr,omitempty"`
	Analyzer         Analyzer          `xml:"idmef:Analyzer"`       // Exactly one.
	CreateTime       Time              `xml:"idmef:CreateTime"`     // Exactly one.
	DetectTime       *Time             `xml:"idmef:DetectTime"`     // Zero or one
	AnalyzerTime     *Time             `xml:"idmef:AnalyzerTime"`   // Zero or one.
	Source           []Source          `xml:"idmef:Source"`         // Zero or more.
	Target           []Target          `xml:"idmef:Target"`         // Zero or more.
	Classification   Classification    `xml:"idmef:Classification"` // Exactly one.
	Assessment       *Assessment       `xml:"idmef:Assessment"`
	CorrelationAlert *CorrelationAlert `xml:"idmef:CorrelationAlert"` // Zero or one.
	AdditionalData   []AdditionalData  `xml:"idmef:AdditionalData"`
}

type AlertIdent

type AlertIdent struct {
	AlertIdent string `xml:",chardata"`
	AnalyzerID string `xml:"analyzerid,attr,omitempty"`
}

type Analyzer

type Analyzer struct {
	AnalyzerID   string   `xml:"analyzerid,attr"`
	Name         string   `xml:"name,attr,omitempty"`
	Manufacturer string   `xml:"manufacturer,attr,omitempty"`
	Model        string   `xml:"model,attr,omitempty"`
	Version      string   `xml:"version,attr,omitempty"`
	Class        string   `xml:"class,attr,omitempty"`
	OSType       string   `xml:"ostype,attr,omitempty"`
	OSVersion    string   `xml:"osversion,attr,omitempty"`
	Node         *Node    `xml:"idmef:Node"`
	Process      *Process `xml:"idmef:Process"`
}

Analyzer class identifies the analyzer from which the Alert or Heartbeat message originates. Only one analyzer may be encoded for each alert or heartbeat, and that MUST be the analyzer at which the alert or heartbeat originated. Although the IDMEF data model does not prevent the use of hierarchical intrusion detection systems (where alerts get relayed up the tree), it does not provide any way to record the identity of the "relay" analyzers along the path from the originating analyzer to the manager that ultimately receives the alert. (from RFC 4765)

type Assessment

type Assessment struct {
	Impact     *Impact     `xml:"idmef:Impact,omitempty"`
	Action     []Action    `xml:"idmef:Action,omitempty"`
	Confidence *Confidence `xml:"idmef:Confidence,omitempty"`
}

type Classification

type Classification struct {
	Ident     string      `xml:"ident,attr,omitempty"`
	Text      string      `xml:"text,attr"`
	Reference []Reference `xml:"idmef:Reference,omitempty"`
}

type Confidence

type Confidence struct {
	Rating string `xml:"rating,attr,omitempty"`
}

type CorrelationAlert

type CorrelationAlert struct {
	Name       string       `xml:"idmef:name,omitempty"`
	AlertIdent []AlertIdent `xml:"idmef:alertident,omitempty"`
}

type File

type File struct {
	Category   string       `xml:"category,attr,omitempty"`
	FSType     string       `xml:"fstype,attr,omitempty"`
	Name       string       `xml:"idmef:name,omitempty"`
	Path       string       `xml:"idmef:path,omitempty"`
	FileAccess []FileAccess `xml:"idmef:FileAccess,omitempty"`
	Linkage    *Linkage     `xml:"idmef:Linkage,omitempty"`
}

type FileAccess

type FileAccess struct {
	UserID     *UserID      `xml:"idmef:UserId,omitempty"`
	Permission []Permission `xml:"idmef:permission,omitempty"`
}

type Heartbeat

type Heartbeat struct {
	MessageID      string           `xml:"messageid,attr,omitempty"`
	Analyzer       Analyzer         `xml:"idmef:Analyzer"`   // Exactly one.
	CreateTime     Time             `xml:"idmef:CreateTime"` // Exactly one.
	AdditionalData []AdditionalData `xml:"idmef:AdditionalData"`
}

type Impact

type Impact struct {
	Severity   string `xml:"severity,attr,omitempty"`
	Completion string `xml:"completion,attr,omitempty"`
	Type       string `xml:"type,attr,omitempty"`
}

type Linkage

type Linkage struct {
	Category string `xml:"category,attr,omitempty"`
	Name     string `xml:"idmef:name,omitempty"`
	Path     string `xml:"idmef:path,omitempty"`
}

type Message

type Message struct {
	XMLName    xml.Name   `xml:"idmef:IDMEF-Message"`
	XMLNSIDMEF string     `xml:"xmlns:idmef,attr"`
	Version    string     `xml:"version,attr"`
	Alert      *Alert     `xml:"idmef:Alert"`
	Heartbeat  *Heartbeat `xml:"idmef:Heartbeat"`
}

Message is for authoring. For parsing use `github.com/grokify/go-idmef/unmarshal/Message`.

func (*Message) Bytes

func (m *Message) Bytes(prefix, indent string) ([]byte, error)

type Node

type Node struct {
	Ident    string   `xml:"ident,attr,omitempty"`
	Category string   `xml:"category,attr,omitempty"`
	Address  *Address `xml:"idmef:Address,omitempty"`
	Location string   `xml:"idmef:location,omitempty"`
	Name     string   `xml:"idmef:name,omitempty"`
}

type Permission

type Permission struct {
	Perms string `xml:"perms,attr,omitempty"`
}

type Process

type Process struct {
	Ident string   `xml:"ident,attr,omitempty"`
	Name  string   `xml:"idmef:name,omitempty"`
	PID   int32    `xml:"idmef:pid,omitempty"`
	Path  string   `xml:"idmef:path,omitempty"`
	Arg   []string `xml:"idmef:arg,omitempty"`
}

type Reference

type Reference struct {
	Origin  string `xml:"origin,attr,omitempty"`
	Meaning string `xml:"meaning,attr,omitempty"`
	Name    string `xml:"idmef:name,omitempty"`
	URL     string `xml:"idmef:url,omitempty"`
}

type Service

type Service struct {
	Ident      string      `xml:"ident,attr,omitempty"`
	Name       string      `xml:"idmef:name,omitempty"`
	Port       int         `xml:"idmef:port,omitempty"`
	Portlist   string      `xml:"idmef:portlist,omitempty"`
	WebService *WebService `xml:"idmef:WebService,omitempty"`
}

type Source

type Source struct {
	Ident   string   `xml:"ident,attr,omitempty"`
	Spoofed string   `xml:"spoofed,attr,omitempty"` // Source
	Node    *Node    `xml:"idmef:Node,omitempty"`
	User    *User    `xml:"idmef:User,omitempty"`
	Process *Process `xml:"idmef:Process,omitempty"`
	Service *Service `xml:"idmef:Service,omitempty"`
}

type Target

type Target struct {
	Ident   string   `xml:"ident,attr,omitempty"`
	Decoy   string   `xml:"decoy,attr,omitempty"` // Target
	Node    *Node    `xml:"idmef:Node,omitempty"`
	User    *User    `xml:"idmef:User,omitempty"`
	Process *Process `xml:"idmef:Process,omitempty"`
	Service *Service `xml:"idmef:Service,omitempty"`
	File    *File    `xml:"idmef:File,omitempty"`
}

type Time

type Time struct {
	Time     time.Time `xml:",chardata"`
	NtpStamp string    `xml:"ntpstamp,attr"`
}

func NewTime

func NewTime(t time.Time) Time

func (*Time) InflateNtpStamp

func (t *Time) InflateNtpStamp()

type User

type User struct {
	Ident    string   `xml:"ident,attr,omitempty"`
	Category string   `xml:"category,attr,omitempty"`
	UserID   []UserID `xml:"idmef:UserId,omitempty"`
}

type UserID

type UserID struct {
	Ident  string `xml:"ident,attr,omitempty"`
	Type   string `xml:"type,attr,omitempty"`
	Name   string `xml:"idmef:name,omitempty"`
	Number string `xml:"idmef:number,omitempty"`
}

type WebService

type WebService struct {
	URL        string `xml:"idmef:url,omitempty"`
	CGI        string `xml:"idmef:cgi,omitempty"`
	HTTPMethod string `xml:"idmef:http-method,omitempty"`
}