csharg

package module
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2023 License: MIT Imports: 18 Imported by: 2

README

Siemens Industrial Edge Edgeshark

csharg

PkgGoDev GitHub build and test Go Report Card

csharg is a Golang module for easy CLI access to the Packetflix capture service deployed to container hosts, such as Edgeshark on Siemens Industrial Edge and also standalone Docker hosts.

csharg is part of the "Edgeshark" project that consist of several repositories:

Usage

csharg can be used in different ways:

  • as the standalone CLI program csharg, such as in:

    csharg --host localhost:5001 capture mycontainer | wireshark -k -i -

    For details, see further down below.

  • via the Containershark Extcap plugin for Wireshark with a nice Wireshark UI.

  • and finally, directly in your own applications and (micro) services by integrating the csharg module without having to care about details of the Packetflix capture service API.

Overview

key:

  • $ csharg ... denotes a CLI command.
  • csharg.New...() denotes a csharg package constructor for connecting to the capture service of container hosts.

Build

To build csharg binary distribution packages for serveral Linux architectures, as well as for Windows (only amd64):

make dist

Afterwards, the dist/ directory contains the built packages and archives. Supported CPU architectures:

  • amd64
  • arm64

The following package formats are currently supported:

  • .apk
  • .deb
  • .rpm
  • .tar.gz

csharg CLI

Commands

This section doesn't aim to be a comprehensive documentation of the cshargCLI, but will give you an idea as to what you might be able to do with the csharg CLI. If in doubt, please use csharg help for up-to-date documentation.

  • csharg list: list available capture targets in a Kubernetes cluster.
  • csharg capture: capture and live stream network traffic from a capture target, such as a pod, standalone container, et cetera.
  • csharg help: ask for help about any of the csharg commands.
  • csharg options: list the global command-line options which apply to all commands.
  • csharg version: show csharg version.

The CLI --host http://$HOSTNAME[:$PORT] argument specifies hostname (DNS/label or IP address) and optional port number of the Packetflix service on container host. Standard deployments use port :5001. Please note that the port always needs to be specified, unless it is port :80 (or :443 for HTTPS).

To list available capture targets in your container host or local KinD deployment:

  • list only stand-alone containers, which are not inside pods: csharg --host ... list containers. Please note that due to the Kubernetes sandbox model with shared network stacks csharg doesn't show or handle the containers inside pods, but only containers that do not belong to any pod.
  • list network stacks which neither belong to pods nor stand-alone containers: csharg --host ... list networks (includes bind-mount'ed network stacks).
  • list only pods: csharg --host ... list pods (KinD).
  • list all capture targets (and we really mean all): csharg -host ... list.
Controlling Capture Target List Output

csharg will list capture targets in tabular format, unless another output format is specified. For example:

$ csharg -host localhost:5001 list
TARGET                                                       TYPE         NODE
accounts-daemon(997)                                         proc         localhost
chrome(79357)                                                proc         localhost
default/nginx                                                pod          localhost
gostwire-gostwire-1                                          docker       localhost
haveged(984)                                                 proc         localhost
kind-control-plane                                           docker       localhost
kube-system/coredns-787d4945fb-787ms                         pod          localhost
kube-system/coredns-787d4945fb-s6drr                         pod          localhost
kube-system/etcd-kind-control-plane                          pod          localhost
kube-system/kindnet-zhgf2                                    pod          localhost
kube-system/kube-apiserver-kind-control-plane                pod          localhost
kube-system/kube-controller-manager-kind-control-plane       pod          localhost
kube-system/kube-proxy-hgbpl                                 pod          localhost
kube-system/kube-scheduler-kind-control-plane                pod          localhost
local-path-storage/local-path-provisioner-75f5b54ffd-xrzg8   pod          localhost
packetflix-packetflix-1                                      docker       localhost
rtkit-daemon(1528)                                           proc         localhost
systemd(1)                                                   proc         localhost

The following output formatting options are available (see also csharg help list):

  • -o wide: wide tabular multi-column format.
  • -o json: list capture target details in JSON format.
  • -o jsonpath=${JSONPATH-EXPRESSION}: print fields defined in the JSONPath expression.
  • -o jsonpath-file=$FILENAME: print fields defined in the JSONPath expression from file filename.
  • -o yaml: lots of ugly YAML data for the available capture targets.
  • -o custom-columns=$SPEC, where $SPEC is a kubectl-like custom columns specification. For instance: -o custom-columns=NAME:.Name,NODE:.NodeName.
  • -o custom-columns-file=$FILENAME: use the custom columns format from file filename.
  • -o name: simply lists only target names, but no other details.

Please note that --no-headers can be used with the standard format, as well as -o wide, -o custom-columns=..., and -o custom-columns-file=... to hide the column headers.

Capture Live Network Traffic

In the most simple case, just specify a unique capture target name (such as a container name) and then csharg will stream the captured network traffic to stdout in pcapng format (please note the podnamespace/podname notation, with podnamespace being mandatory):

csharg --host localhost:5001 capture container-name

Use option -w filename to write the packet capture stream into the file filename. As it is custom, -w - again writes to stdout (which is the default anyway).

Probably more typical is to feed the live stream directly into Wireshark, this works without having to install the Containershark extcap plugin:

csharg --host localhost:5001 capture special/mypod | wireshark -k -i -

The capture will run until you terminate/interrupt csharg with SIGINT or SIGTERM, for instance, by pressing ^C in your terminal session where you started csharg in the foreground.

By default, captures will capture from all network interfaces of the specified target. Use one or multiple -i/--interface options to specify only those network interfaces you want to capture from:

csharg --host ... capture container container-name -i lo

Filter captures at the source using the --filter option -- the capture filter syntax is Wireshark's dumpcap filter syntax.

Standalone Host: as long as the target name is unique, csharg capture will start a capture even without having to specify the node/host. This makes capturing from a standalone container host especially convenient when using csharg capture ... instead of csharg capture container ....

Look Mum, My First Csharg Program!

Capture five minutes of network traffic on all network interfaces of container "my-container" and write the captured packets into a file named "my-container.pcapng".

package main

import (
    "github.com/siemens/csharg"
    "os"
    "time"
)

func main() {
    // Connect to the host-local Packetflix capture service. Default settings
    // and service time limits (30s) apply. Sensibly, capture  streaming is
    // not time limited.
    st, _ := csharg.NewSharkTankOnHost("http://localhost:5001", nil)
    // Create a new packet capture file...
    f, _ := os.Create("my-container.pcapng")
    defer f.Close()
    // ...and run a network packet capture for five minutes on the container
    // "my-container".
    capt, _ := st.CaptureContainer(f, "my-container", nil)
    capt.StopAfter(5*time.Minute)
}

FAQ

  • What does "csharg" mean?

    "csharg" is short for "Container Shark" or "Containershark".

  • Shouldn't this be "cshark"?

    Not necessarily, as this is a Go package.

VSCode Tasks

The included csharg.code-workspace defines the following tasks:

  • View Go module documentation task: installs pkgsite, if not done already so, then starts pkgsite and opens VSCode's integrated ("simple") browser to show the csharg documentation.
Aux Tasks
  • pksite service: auxilliary task to run pkgsite as a background service using scripts/pkgsite.sh. The script leverages browser-sync and nodemon to hot reload the Go module documentation on changes; many thanks to @mdaverde's Build your Golang package docs locally for paving the way. scripts/pkgsite.sh adds automatic installation of pkgsite, as well as the browser-sync and nodemon npm packages for the local user.
  • view pkgsite: auxilliary task to open the VSCode-integrated "simple" browser and pass it the local URL to open in order to show the module documentation rendered by pkgsite. This requires a detour via a task input with ID "pkgsite".

Make Targets

  • make: lists all targets.
  • make clean: removes the build artefacts.
  • make dist: builds snapshot packages and archives of the csharg CLI binary.
  • make pkgsite: installs x/pkgsite, as well as the browser-sync and nodemon npm packages first, if not already done so. Then runs the pkgsite and hot reloads it whenever the documentation changes.
  • make report: installs @gojp/goreportcard if not yet done so and then runs it on the code base.
  • make vuln: install (or updates) govuln and then checks the Go sources.

Contributing

Please see CONTRIBUTING.md.

(c) Siemens AG 2023

SPDX-License-Identifier: MIT

Documentation

Overview

Package csharg captures network traffic inside container hosts that are either stand-alone or part of a Kubernetes cluster. These are live captures: the captured network packets are immediately streamed to capture clients. There is no need to record first, and then download; instead, you can analyse network traffic live as it happens. So, Streaming Killed the Download Star – with apologies to Trevor Horn.

Live network captures can be taken not only from standalone containers, but also from Kubernetes pods, the node (host) network stacks, and even process-less standalone virtual network stacks.

Please note that due to the “sandbox” design of Kubernetes pods, it is not possible to capture from a single container inside a pod: all containers in the same pod share the same network stack.

Optionally, network captures can be confined to only a subset of network interfaces of a pod, container, et cetera. Furthermore, clients can specify packet filters to apply at the sources, before sending the capture stream. Please see struct CaptureOptions for details.

Normally, packet capture streaming will go on until you stop it. See the examples for how to automatically stop a packet capture stream after a given amount of time.

Index

Constants

View Source 
const (
	// DefaultServiceTimeout specifies the time limit for completing discovery
	// service calls and for establishing a stream connection to the capture
	// service.
	DefaultServiceTimeout = 30 * time.Second
)
View Source 
const SemVersion = "0.12.2-10-g6c9c830"

SemVersion is the semantic version string of the csharg module.

Variables

View Source 
var AllNifs = Nifs{}

AllNifs will capture from all available network interfaces of a capture target, regardless of how many and which ones it will have. This is just a convenience to those situations where a programmer wants to emphasize explicitly capturing from all network interfaces of a capture target, instead of the implicit zero default.

Functions

func CaptureServiceHeaders 

func CaptureServiceHeaders(t *api.Target, opts *CaptureOptions) (header *http.Header, err error)

CaptureServiceHeaders is a convenience function that builds the set of capture service HTTP/WS headers required in order to successfully connect via the Kubernetes remote API proxy to the capture service -- where the WS service request unfortunately looses the URL parameters, so we need to resort to HTTP headers. This bug is documented, but doesn't seem to get any attention to fix it, which really is unfortunate. Anyway, we use the capture service headers also when not passing broken Kubernetes remote API servers, to keep things more uniform.

func CaptureServiceQueryParams 

func CaptureServiceQueryParams(t *api.Target, opts *CaptureOptions) (values *url.Values, err error)

CaptureServiceQueryParams is a convenience function that builds the HTTP/WS URL query parameters necessary to connect successfully to the capture service -- unless there's a broken Kubernetes remote API proxy in between where we query params get lost in transit, but we uniformly use the query params whenever we contact the SharkTank capture service, regardless of the path we'll take.

func CompleteTarget 

func CompleteTarget(t *api.Target, opts *CaptureOptions, ts *TargetCache) (*api.Target, error)

CompleteTarget completes the capture target description to the point that the SharkTank service can be successfully contacted on the service application level. If the target description needs to be modified, then CompleteTarget will return a shallow copy of the passed target description, with the necessary additional data filled in. Otherwise, if the target description is already sufficient to start the capture, then it will be returned as-is instead.

Please note that any list of network interfaces in the capture target description will be replaced by either the list specified in the capture options (which takes precedence) or the discovered complete list of network interfaces for this target.

Types

type CaptureOptions 

type CaptureOptions struct {
	// The list of network interfaces (names) to capture from; defaults to all
	// network interfaces of the capture target (that is, AllNifs) if left zero.
	Nifs Nifs
	// Packet capture filter expression, defaults to no filtering. For its
	// syntax, please refer to:
	// https://www.tcpdump.org/manpages/pcap-filter.7.html
	Filter string
	// If true, avoid switching into promiscuous mode if possible. Please note
	// that it is not possible to disable promiscuous mode: other parallel
	// captures might already have switched on promiscuous mode so we can never
	// force it off. This zero setting defaults to switching promiscuous mode
	// ON.
	AvoidPromiscuousMode bool
}

CaptureOptions describe a set of options giving more detailed control over how to capture network traffic from a capture target and an optional specific set of network interfaces to capture from.

type CaptureStreamer 

type CaptureStreamer interface {
	// Stop this capture in an orderly manner. This operation will block until
	// the capture has finally terminated. It is also idempotent.
	Stop()
	// Wait for the capture to terminate, but do not initiate the termination.
	Wait()
	// StopAfter waits the specified duration for the capture to terminate, and
	// terminates it after the duration if necessary.
	StopAfter(d time.Duration)
}

CaptureStreamer gives control over an individual network packet capture.

func StartCaptureStream 

func StartCaptureStream(w io.Writer, ws *websocket.Conn, t *api.Target, opts *CaptureOptions) (cs CaptureStreamer, err error)

StartCaptureStream is a low-level function almost all cshark package users WON'T use. Instead, csharg package users typically want to use the high-level SharkTank methods CapturePod, CaptureContainer, and Capture instead. Please see the package examples for how to use the high-level capture functions.

The low-level StartCaptureStream which needs to be given an already successfully connected websocket, a capture target specification, and capture options. It then starts the capture by issuing a capture service request via the websocket and then in the background streams the incomming network packet data into the given Writer.

type CommonClientOptions 

type CommonClientOptions struct {
	// BearerToken optionally specifies the bearer token to use when talking to
	// the cluster capture service, regardless of how we reach the service.
	BearerToken string
	// Timeout specifies a time limit for requests made to the SharkTank cluster
	// capture service. For discovery it limits the time allowed to complete a
	// discovery request and response. For capturing it limits just the
	// connection establishing phase, including the web socket handshake phase.
	Timeout time.Duration
}

CommonClientOptions defines options common to all cluster capture client types.

type Nifs 

type Nifs []string

Nifs is a list of network interface names.

type SharkTank 

type SharkTank interface {
	// Lists the available capture targets in this cluster.
	Targets() (ts api.Targets)
	// Captures network traffic from a specific pod and send the captured packet
	// stream to the writer w. The capture optionally can be restricted to only
	// a subset of the pod's network interfaces. The pod name can be prefixed by
	// a namespace in form of "namespace/podname"; if the namespace is left out
	// it defaults to the aptly-named "default" namespace.
	CapturePod(w io.Writer, podname string, opts *CaptureOptions) (cs CaptureStreamer, err error)
	// Captures network traffic from a specific container on a specific
	// kubernetes node. Of course, the capture can be restricted to only a
	// subset of the pod's network interfaces.
	CaptureContainer(w io.Writer, nodename, name string, opts *CaptureOptions) (cs CaptureStreamer, err error)
	// Captures network traffic from a capture target, such as a pod, a
	// stand-alone container, a process-less IP stack, et cetera, optionally
	// limited to a specific (set of) network interface(s) for this target. The
	// captured packets are then send to the given Writer.
	Capture(w io.Writer, t *api.Target, opts *CaptureOptions) (cs CaptureStreamer, err error)
	// Clears the cached set of capture targets: a SharkTank will fetch the set
	// of capture targets anew when it needs them, and will then cache them
	// because typically there will be multiple lookups into the cached set
	// necessary in order to start a capture.
	Clear()
}

SharkTank gives access to network captures in clusters via the SharkTank cluster capture service.

func NewSharkTankOnHost 

func NewSharkTankOnHost(hosturl string, opts *SharkTankOnHostOptions) (st SharkTank, err error)

NewSharkTankOnHost returns a new host capturer object to capture directly from host targets using a Packetflix service, and accessing it via host+port and an optional service path.

type SharkTankOnHostOptions 

type SharkTankOnHostOptions struct {
	CommonClientOptions
	InsecureSkipVerify bool
}

SharkTankOnHostOptions allows some degree of control over how to use a (SharkTank) Packetflix service reachable at a given address and port.

type TargetCache 

type TargetCache struct {
	// contains filtered or unexported fields
}

TargetCache caches and indexes a set of capture targets. It can safely be accessed simultaneously by multiple go routines.

func (*TargetCache) Clear 

func (tc *TargetCache) Clear()

Clear the cached capture target descriptions.

func (*TargetCache) IsEmpty 

func (tc *TargetCache) IsEmpty() bool

IsEmpty returns true if the cache is empty, otherwise false.

func (*TargetCache) OnNode 

func (tc *TargetCache) OnNode(nodename, prefix, name string) (*api.Target, bool)

OnNode returns the capture target with the given prefix+name and located on the specified cluster node. Use OnNode() when capturing from per-node targets, such as a kubelet, et cetera. For capturing from pods, use Pod() instead, as it doesn't need the specific nodename to be told.

func (*TargetCache) Pod 

func (tc *TargetCache) Pod(name string) (*api.Target, bool)

Pod returns the pod capture target with the specified prefix and name. For other types of targets the lookup will fail and return (nil, false). Use OnNode() instead when looking up capture targets that may occur multiple times inside a cluster on different cluster nodes.

func (*TargetCache) Set 

func (tc *TargetCache) Set(ts api.Targets)

Set the target descriptions to be cached.

func (*TargetCache) Targets 

func (tc *TargetCache) Targets() api.Targets

Targets returns the list of capture target descriptions.

Source Files

View all Source files

Directories

Path Synopsis
Package api defines the JSON data structures describing the available capture targets returned by the capture service endpoint(s).
 Package api defines the JSON data structures describing the available capture targets returned by the capture service endpoint(s).
cli
Package cli defines plugin extension points for the csharg command.
 Package cli defines plugin extension points for the csharg command.
command
Package command implements the common commands of the csharg CLI.
 Package command implements the common commands of the csharg CLI.
command/capture
sharktank
cmd
csharg
internal
gen/version
Package pcapng implements a pcapng stream editor which edits the first Section Header Block (SHB), inserting additional meta data as comments.
 Package pcapng implements a pcapng stream editor which edits the first Section Header Block (SHB), inserting additional meta data as comments.
Package pipe implements waiting for a fifo/pipe to break.
 Package pipe implements waiting for a fifo/pipe to break.
Package ws enhances Gorilla client websockets by handling graceful closing on both sides using polite close control messages.
 Package ws enhances Gorilla client websockets by handling graceful closing on both sides using polite close control messages.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.