netcap

package module
Version: v0.6.11 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 30, 2022 License: BSD-3-Clause, GPL-3.0 Imports: 0 Imported by: 5

README

Netcap Logo


Go Report Card License Golang Linux macOS windows GoDoc Homepage Documentation Total alerts FOSSA Status

The Netcap (NETwork CAPture) framework efficiently converts a stream of network packets into platform neutral type-safe structured audit records that represent specific protocols or custom abstractions. These audit records can be stored on disk or exchanged over the network, and are well suited as a data source for machine learning algorithms. Since parsing of untrusted input can be dangerous and network data is potentially malicious, a programming language that provides a garbage collected memory safe runtime is used for the implementation.

It was developed for a series of experiments in my bachelor thesis: Implementation and evaluation of secure and scalable anomaly-based network intrusion detection. Slides from my presentation at the Leibniz Supercomputing Centre of the Bavarian Academy of Sciences and Humanities are available on researchgate.

The project won the 2nd Place at Kaspersky Labs SecurIT Cup 2018 in Budapest.

Netcap uses Google's Protocol Buffers to encode its output, which allows accessing it across a wide range of programming languages. Alternatively, output can be emitted as comma separated values, which is a common input format for data analysis tools and systems. The tool is extensible and provides multiple ways of adding support for new protocols, while implementing the parsing logic in a memory safe way. It provides high dimensional data about observed traffic and allows the researcher to focus on experimenting with novel approaches for detecting malicious behavior in network environments, instead of fiddling with data collection mechanisms and post processing steps. It has a concurrent design that makes use of multi-core architectures. The name Netcap was chosen to be simple and descriptive. The command-line tool was designed with usability and readability in mind, and displays progress when processing packets. The latest version offers 58 audit record types of which 53 are protocol specific and 5 are flow models.

For more details please refer to the Documentation, visit the Homepage and read the thesis.

A simple demonstration of generating audit records from a PCAP dump file, querying and displaying the collected information in various ways:

asciicast

And live operation decoding traffic from my wireless network interface, while I am surfing the web:

asciicast

Design Goals

  • memory safety when parsing untrusted input
  • ease of extension
  • output format interoperable with many different programming languages
  • concurrent design
  • output with small storage footprint on disk
  • gather everything, separate what can be understood from what can't
  • allow implementation of custom abstractions
  • rich platform and architecture support

Use Cases

  • monitoring honeypots
  • monitoring medical / industrial devices
  • research on anomaly-based detection mechanisms
  • forensic data analysis

Framework Components

The framework consists of 9 logically separate tools compiled into a single binary:

  • capture (capture audit records live or from dumpfiles)
  • dump (dump with audit records in various formats)
  • label (tool for creating labeled CSV datasets from netcap data)
  • collect (collection server for distributed collection)
  • agent (sensor agent for distributed collection)
  • proxy (http reverse proxy for capturing traffic from web services)
  • util (utility tool for validating audit records and converting timestamps)
  • export (exporter for prometheus metrics)
  • transform (maltego transformation plugin)

Integrations

Prometheus Metrics

Overview Dashboard example:

HTTP Dashboard example:

You can read more about the Prometheus integration in the docs.

Maltego

Extract DHCP information from local network devices of a PCAP dump file:

HTTP File extraction:

HTTP parameter command injection analysis:

You can read more about the Maltego integration in the docs.

Contributing

Contributions welcome, there's plenty of stuff to do, from simple additions to low level framework engineering!

Please see the Contributing Page for more information.

Bug Reports

If you encounter a bug while using Netcap, please fill out the bugreport template and open a github issue.

License

Netcap is licensed under the GNU General Public License v3, which is a very permissive open source license, that allows others to do almost anything they want with the project, except to distribute closed source versions. This license type was chosen with Netcaps research purpose in mind, and in the hope that it leads to further improvements and new capabilities contributed by other researchers on the long term.

FOSSA Status

Documentation

Overview

Package netcap is a framework for network traffic analysis. Learn more at https://netcap.io.

Index

Constants

This section is empty.

Variables

View Source
var Commit = "39a85c6357a0dc7aeca2e869176903adef5c0ed3"

Commit is the git commit id of the current version.

View Source
var Version = "v0.6.11"

Version is the current version identifier for netcap.

Functions

This section is empty.

Types

This section is empty.

Source Files

Directories

Path Synopsis
cmd
Package collector provides a mechanism to collect network packets from a network interface on macOS, linux and windows
Package collector provides a mechanism to collect network packets from a network interface on macOS, linux and windows
db
packet
Package packet Package decoder implements decoders to transform network packets into protocol buffers for various protocols
Package packet Package decoder implements decoders to transform network packets into protocol buffers for various protocols
Package defaults contains default settings for the netcap core.
Package defaults contains default settings for the netcap core.
Package delimited implements a simple reader and writer for streams of length-delimited byte records.
Package delimited implements a simple reader and writer for streams of length-delimited byte records.
Package dpi implements an interface for application layer classification via bindings to nDPI and libprotoident
Package dpi implements an interface for application layer classification via bindings to nDPI and libprotoident
Package env contains static environment variable names that netcap uses.
Package env contains static environment variable names that netcap uses.
Package io implements IO primitives
Package io implements IO primitives
Package label implements mapping alerts from suricata to netcap audit records
Package label implements mapping alerts from suricata to netcap audit records
Package metrics provides primitives for instrumentation via prometheus
Package metrics provides primitives for instrumentation via prometheus
Package reassembly provides TCP stream re-assembly.
Package reassembly provides TCP stream re-assembly.
Package resolvers implements primitives to resolve various identifiers against external data
Package resolvers implements primitives to resolve various identifiers against external data
Package types Contains the type definitions for the supported network protocols
Package types Contains the type definitions for the supported network protocols

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL