nettrace

package module
v0.0.0-...-15d82a0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 2, 2023 License: Apache-2.0 Imports: 32 Imported by: 1

README

Network Tracing (nettrace package)

Motivation

Taking Golang's HTTP client as an example - a single HTTP request (triggered for example by calling Client.Do()) uses multiple network protocols from different layers of the network stack and performs multiple network operations, some run in sequence and others in parallel. For example, the client first has to resolve the destination hostname, unless an IP address was given directly instead. It will (most likely) use the Golang's own resolver to try every configured DNS server one by one. For every DNS server it may run DNS requests (over UDP or TCP if UDP response is truncated) for IPv4 and IPv6 in parallel (the Happy Eyeballs algorithm). The hostname itself may resolve into multiple IP addresses and the HTTP client will try one or several of them until it succeeds to open a TCP connection. For HTTPs it will then follow with the TLS handshake at the session layer of the network stack. Only then the actual HTTP request is send. If an HTTP redirect response code is returned, the whole process is repeated for the returned URL.

This can be even more complicated because:

  • TCP connection can be reused between HTTP requests (HTTP keep-alive)
  • HTTP client can be configured to use a network proxy. The proxy may listen on HTTP or HTTPs. In the latter case the client would establish TLS tunnel inside a TLS tunnel!
  • HTTP client can be configured to use a specific source IP address instead of picking one dynamically based on a routing decision.

Despite all this complexity, there is only one error value returned and available if a request fails. Even though the error may wrap multiple errors inside, there is often not enough information needed to troubleshoot a failed HTTP request. For example, quite common is to receive a timeout error of some sort from the HTTP client. Given that the client performs multiple network operations synchronously or in parallel and that there are multiple timeouts configurable for different operations (DNS resolution, TCP handshakes and the request itself can all have different timeouts), it can be challenging to determine which operation has failed to finish in time or has consumed unexpected amount of it. Furthermore, the errors returned are difficult to process and analyse programmatically. The error attributes (e.g. a destination IP address of a failed TCP handshake) are often not exported and need to be parsed from the string error message, which is cumbersome. In some cases even the error type itself is not exported (e.g. http.httpError).

The idea of this package is to hook into Golang's HTTP client and capture all kinds of information available from different network layers of the HTTP request processing (see the next section to learn what we can observe and record), which, when combined, provides the user with a clear picture of what was happening behind the scenes. When a request fails, it is much easier to backtrace to its root cause. Network traces which the package returns are all well-defined using structures with many exported and documented fields and can be used as an input for a computer-driven analysis.

For now the package supports only the Golang's HTTP client, but in the future we could add support for network tracing of some other networking-oriented clients/servers/... written in Go.

What Can Be Traced

Applying nettrace to Golang's HTTP client, the following set of network traces will be collected:

  • record of every HTTP request, including the number of the HTTP version used, HTTP headers (optional), content length, response status code and more.
  • record of every Dial (see Transport.DialContext()), with information about the destination address, static source address if configured, list of nested Dials performed by the resolver, etc.
  • record of every UDP "connection" (used for name resolution), including the src/dst IP/port 4-tuple, conntrack entry (optional), number of payload bytes sent/received, trace of every socket read/write operation (optional).
  • record of every TCP connection (established or failed to establish), including the src/dst IP/port 4-tuple, conntrack entry (optional), number of payload bytes sent/received, trace for every socket read/write operation (optional), flag informing if the connection was reused, etc.
  • record of every DNS query<->reply conversation between the resolver and a DNS server (optional)
  • record of every TLS tunnel (established or failed to establish), including summary of peer certificates (subject, issuer, validity time range), negotiated protocol and cipher suite, etc.
  • packet capture filtered to contain only packets corresponding to traced HTTP requests (optional)

These traces reference each other using trace IDs (see TraceID data type). For example, HTTP request trace references recording of the used TCP connection, which then references Dial where it has originated from.

Moreover, every trace includes one or more timestamps, used to inform when the given operation began, ended, when the context was closed, etc. These timestamps are recorded relatively in the milliseconds precision wrt. time when the tracing started for better readability.

Some of these traces are configurable and can be enabled/disabled - see the set of available options.

For a full list of available traces and their attributes, see NetTrace and its extension HTTPTrace (adding HTTP specific network traces).

How To Use It

In order to trace Golang's http.Client, it is necessary to let the nettrace package to instantiate the client (so that it can add some hooks for tracing purposes). Meaning that instead of doing client := &http.Client{}, use the constructor provided by nettrace:

import (
    "github.com/lf-edge/eve/libs/nettrace"
)

func main() {
    // Example config for the HTTP client:
    cfg := nettrace.HTTPClientCfg{
        PreferHTTP2:       false, // Prefer HTTP1/1
        DisableKeepAlives: true,  // Do not reuse connections
        ReqTimeout:        time.Minute,
    }
    // Example options for network tracing:
    opts := []nettrace.TraceOpt{
        &nettrace.WithLogging{},
        &nettrace.WithConntrack{},
        &nettrace.WithSockTrace{},
        &nettrace.WithDNSQueryTrace{},
        &nettrace.WithHTTPReqTrace{
            HeaderFields: nettrace.HdrFieldsOptDisabled,
        },
        &nettrace.WithPacketCapture{
            Interfaces:  []string{"eth0"},
            IncludeICMP: true,
            IncludeARP:  true,
        },
    }
    client, err := nettrace.NewHTTPClient(cfg, opts...)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    // ...
}

The returned client wraps the http.Client as an exported field, meaning that it exposes all the methods of the original client, like Do(), Get(), Post(), etc. If a 3rd party library expects http.Client type, simply pass the embedded client.Client.

Please DO NOT change the Client.Transport field of the embedded client (to further customize the HTTP client behaviour), otherwise tracing functionality may get broken. Instead, configure the desired behaviour of the HTTP client inside the nettrace.HTTPClientCfg argument of the nettrace.NewHTTPClient() constructor. The only allowed action is to additionally wrap the Transport with a RoundTripper implementation, which is allowed to for example modify HTTP requests/responses, but still should call the wrapped Transport for the HTTP request execution. An example of this is Transport from the oauth2 package, adding an Authorization header with a token.

With the client constructed, run one or more HTTP requests (using the embedded http.Client) and later use GetTrace() to obtain collected network traces:

ctx := context.Background()
req, err := http.NewRequestWithContext(ctx, "GET", "https://www.google.com/", nil)
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
resp, err := client.Do(req)
if err == nil && resp != nil && resp.Body != nil {
    if _, err := io.Copy(os.Stdout, resp.Body); err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}

// ...

httpTrace, pcaps, err := client.GetTrace("example")
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
traceInJson, err := json.MarshalIndent(httpTrace, "", "  ")
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
fmt.Printf("Network traces collected from the HTTP client: %s\n", string(traceInJson))

Note that communication with the HTTP server continues until the request fails or the returned body is fully read or closed. In other words, prefer getting network traces AFTER reading response body.

The returned packet captures (pcaps in the example; one for each configured interface) can be each saved to a file using PacketCapture.WriteToFile(filename) and analyzed e.g. using Wireshark.

When starting a new HTTP request, one may want to remove previously collected network traces before starting collecting new ones - use HTTPClient.ClearTrace() for that.

Lastly, before leaving HTTPClient for the garbage collector, call HTTPClient.Close() to ensure that network tracing and packet capturing are stopped and all the resources held by nettrace are freed.

Network Trace Example

Attached is an example of all network traces (except for packet capture) collected for a (NATed) GET https://www.google.com/ request. This is returned as an instance of HTTPTrace structure and can be marshalled into JSON, an example of which is shown below:

{
  "description": "GET https://www.google.com/",
  "traceBeginAt": "2022-12-23T10:17:45.260618344+01:00",
  "traceEndAt": "+1241ms",
  "dials": [
    {
      "traceID": "tid-3",
      "dialBeginAt": "+72ms",
      "dialEndAt": "+126ms",
      "ctxCloseAt": "+241ms",
      "dstAddress": "www.google.com:443",
      "resolverDials": [
        {
          "dialBeginAt": "+73ms",
          "dialEndAt": "+73ms",
          "nameserver": "8.8.8.8:53",
          "establishedConn": "tid-5"
        },
        {
          "dialBeginAt": "+73ms",
          "dialEndAt": "+73ms",
          "nameserver": "8.8.8.8:53",
          "establishedConn": "tid-7"
        }
      ],
      "sourceIP": "192.168.99.1",
      "establishedConn": "tid-8"
    }
  ],
  "tcpConns": [
    {
      "traceID": "tid-8",
      "fromDial": "tid-3",
      "handshakeBeginAt": "+115ms",
      "handshakeEndAt": "+126ms",
      "connected": true,
      "connCloseAt": "+241ms",
      "addrTuple": {
        "srcIP": "192.168.99.1",
        "srcPort": 33623,
        "dstIP": "142.251.36.132",
        "dstPort": 443
      },
      "reused": false,
      "totalSentBytes": 735,
      "totalRecvBytes": 12657,
      "conntrack": {
        "capturedAt": "+1075ms",
        "status": "assured|src-nat|confirmed|src-nat-done|dst-nat-done|seen-reply",
        "tcpState": "last-ack",
        "mark": 0,
        "addrOrig": {
          "srcIP": "192.168.99.1",
          "srcPort": 33623,
          "dstIP": "142.251.36.132",
          "dstPort": 443
        },
        "addrReply": {
          "srcIP": "142.251.36.132",
          "srcPort": 443,
          "dstIP": "192.168.88.2",
          "dstPort": 33623
        },
        "packetsSent": 24,
        "packetsRecv": 17,
        "bytesSent": 1991,
        "bytesRecv": 13549
      },
      "socketTrace": {
        "socketOps": [
          {
            "type": "write",
            "callAt": "+127ms",
            "returnAt": "+127ms",
            "dataLen": 280
          },
          {
            "type": "read",
            "callAt": "+127ms",
            "returnAt": "+152ms",
            "dataLen": 576
          },
          {
            "type": "read",
            "callAt": "+153ms",
            "returnAt": "+153ms",
            "dataLen": 3713
          },
          {
            "type": "write",
            "callAt": "+172ms",
            "returnAt": "+172ms",
            "dataLen": 64
          },
          {
            "type": "write",
            "callAt": "+172ms",
            "returnAt": "+172ms",
            "dataLen": 86
          },
          {
            "type": "write",
            "callAt": "+172ms",
            "returnAt": "+172ms",
            "dataLen": 67
          },
          {
            "type": "read",
            "callAt": "+172ms",
            "returnAt": "+183ms",
            "dataLen": 62
          },
          {
            "type": "write",
            "callAt": "+183ms",
            "returnAt": "+183ms",
            "dataLen": 31
          },
          {
            "type": "read",
            "callAt": "+183ms",
            "returnAt": "+184ms",
            "dataLen": 31
          },
          {
            "type": "read",
            "callAt": "+184ms",
            "returnAt": "+239ms",
            "dataLen": 1022
          },
          {
            "type": "read",
            "callAt": "+239ms",
            "returnAt": "+239ms",
            "dataLen": 4864
          },
          {
            "type": "read",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 736
          },
          {
            "type": "read",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 1653
          },
          {
            "type": "write",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 48
          },
          {
            "type": "write",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 48
          },
          {
            "type": "write",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 48
          },
          {
            "type": "write",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 39
          },
          {
            "type": "write",
            "callAt": "+240ms",
            "returnAt": "+240ms",
            "dataLen": 24
          },
          {
            "type": "read",
            "callAt": "+240ms",
            "returnAt": "+241ms",
            "returnErr": "read tcp 192.168.99.1:33623-\u003e142.251.36.132:443: use of closed network connection",
            "dataLen": 0
          }
        ]
      }
    }
  ],
  "udpConns": [
    {
      "traceID": "tid-5",
      "fromDial": "tid-3",
      "fromResolver": true,
      "socketCreateAt": "+73ms",
      "connCloseAt": "+115ms",
      "addrTuple": {
        "srcIP": "192.168.99.1",
        "srcPort": 40475,
        "dstIP": "8.8.8.8",
        "dstPort": 53
      },
      "totalSentBytes": 43,
      "totalRecvBytes": 71,
      "conntrack": {
        "capturedAt": "+1074ms",
        "status": "confirmed|src-nat-done|dst-nat-done|seen-reply|src-nat",
        "mark": 0,
        "addrOrig": {
          "srcIP": "192.168.99.1",
          "srcPort": 40475,
          "dstIP": "8.8.8.8",
          "dstPort": 53
        },
        "addrReply": {
          "srcIP": "8.8.8.8",
          "srcPort": 53,
          "dstIP": "192.168.88.2",
          "dstPort": 40475
        },
        "packetsSent": 1,
        "packetsRecv": 1,
        "bytesSent": 71,
        "bytesRecv": 99
      },
      "socketTrace": {
        "socketOps": [
          {
            "type": "write",
            "callAt": "+73ms",
            "returnAt": "+73ms",
            "dataLen": 43
          },
          {
            "type": "read",
            "callAt": "+73ms",
            "returnAt": "+115ms",
            "dataLen": 71
          }
        ]
      }
    },
    {
      "traceID": "tid-7",
      "fromDial": "tid-3",
      "fromResolver": true,
      "socketCreateAt": "+73ms",
      "connCloseAt": "+115ms",
      "addrTuple": {
        "srcIP": "192.168.99.1",
        "srcPort": 49789,
        "dstIP": "8.8.8.8",
        "dstPort": 53
      },
      "totalSentBytes": 43,
      "totalRecvBytes": 59,
      "conntrack": {
        "capturedAt": "+1074ms",
        "status": "seen-reply|src-nat|confirmed|src-nat-done|dst-nat-done",
        "mark": 0,
        "addrOrig": {
          "srcIP": "192.168.99.1",
          "srcPort": 49789,
          "dstIP": "8.8.8.8",
          "dstPort": 53
        },
        "addrReply": {
          "srcIP": "8.8.8.8",
          "srcPort": 53,
          "dstIP": "192.168.88.2",
          "dstPort": 49789
        },
        "packetsSent": 1,
        "packetsRecv": 1,
        "bytesSent": 71,
        "bytesRecv": 87
      },
      "socketTrace": {
        "socketOps": [
          {
            "type": "write",
            "callAt": "+73ms",
            "returnAt": "+74ms",
            "dataLen": 43
          },
          {
            "type": "read",
            "callAt": "+74ms",
            "returnAt": "+115ms",
            "dataLen": 59
          }
        ]
      }
    }
  ],
  "dnsQueries": [
    {
      "traceID": "tid-b",
      "fromDial": "tid-3",
      "connection": "tid-7",
      "dnsQueryMsgs": [
        {
          "sentAt": "+74ms",
          "id": 34311,
          "recursionDesired": true,
          "truncated": false,
          "size": 43,
          "questions": [
            {
              "name": "www.google.com.",
              "type": "A",
              "class": 1
            }
          ],
          "optUDPPayloadSize": 1232
        }
      ],
      "dnsReplyMsgs": [
        {
          "recvAt": "+115ms",
          "id": 34311,
          "authoritative": false,
          "recursionAvailable": true,
          "truncated": false,
          "size": 59,
          "rCode": "no-error",
          "answers": [
            {
              "name": "www.google.com.",
              "type": "A",
              "class": 1,
              "ttl": 227,
              "resolvedVal": "142.251.36.132"
            }
          ]
        }
      ]
    },
    {
      "traceID": "tid-a",
      "fromDial": "tid-3",
      "connection": "tid-5",
      "dnsQueryMsgs": [
        {
          "sentAt": "+73ms",
          "id": 54063,
          "recursionDesired": true,
          "truncated": false,
          "size": 43,
          "questions": [
            {
              "name": "www.google.com.",
              "type": "AAAA",
              "class": 1
            }
          ],
          "optUDPPayloadSize": 1232
        }
      ],
      "dnsReplyMsgs": [
        {
          "recvAt": "+115ms",
          "id": 54063,
          "authoritative": false,
          "recursionAvailable": true,
          "truncated": false,
          "size": 71,
          "rCode": "no-error",
          "answers": [
            {
              "name": "www.google.com.",
              "type": "AAAA",
              "class": 1,
              "ttl": 133,
              "resolvedVal": "2a00:1450:4014:80e::2004"
            }
          ]
        }
      ]
    }
  ],
  "tlsTunnels": [
    {
      "traceID": "tid-9",
      "tcpConn": "tid-8",
      "handshakeBeginAt": "+126ms",
      "handshakeEndAt": "+172ms",
      "didResume": false,
      "peerCerts": [
        {
          "subject": "CN=www.google.com",
          "issuer": "CN=GTS CA 1C3,O=Google Trust Services LLC,C=US",
          "notBefore": "2022-11-28T08:19:01Z",
          "notAfter": "2023-02-20T08:19:00Z",
          "isCA": false
        },
        {
          "subject": "CN=GTS CA 1C3,O=Google Trust Services LLC,C=US",
          "issuer": "CN=GTS Root R1,O=Google Trust Services LLC,C=US",
          "notBefore": "2020-08-13T00:00:42Z",
          "notAfter": "2027-09-30T00:00:42Z",
          "isCA": true
        },
        {
          "subject": "CN=GTS Root R1,O=Google Trust Services LLC,C=US",
          "issuer": "CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE",
          "notBefore": "2020-06-19T00:00:42Z",
          "notAfter": "2028-01-28T00:00:42Z",
          "isCA": true
        }
      ],
      "cipherSuite": 4865,
      "negotiatedProto": "h2",
      "serverName": "www.google.com"
    }
  ],
  "httpRequests": [
    {
      "traceID": "tid-2",
      "tcpConn": "tid-8",
      "protoMajor": 2,
      "protoMinor": 0,
      "reqSentAt": "+72ms",
      "reqMethod": "GET",
      "reqURL": "https://www.google.com/",
      "reqContentLen": 0,
      "respRecvAt": "+239ms",
      "respStatusCode": 200,
      "RespContentLen": 15381
    }
  ]
}

Documentation

Overview

Package nettrace allows to trace (monitor and record a summary of) network operations that happen behind the scenes during e.g. an HTTP request processing as executed by http.Client.

Index

Constants

This section is empty.

Variables

View Source 
var ConntrackFlags = map[string]uint32{

	"expected": 1,

	"seen-reply": 1 << 1,

	"assured": 1 << 2,

	"confirmed": 1 << 3,

	"src-nat": 1 << 4,

	"dst-nat": 1 << 5,

	"seq-adjust": 1 << 6,

	"src-nat-done": 1 << 7,

	"dst-nat-done": 1 << 8,

	"dying": 1 << 9,

	"fixed-timeout": 1 << 10,

	"template": 1 << 11,

	"untracked": 1 << 12,

	"helper": 1 << 13,

	"offload": 1 << 14,
}

ConntrackFlags : conntrack connection's status flags, from enum ip_conntrack_status. See uapi/linux/netfilter/nf_conntrack_common.h

View Source 
var DNSRCodeFromString = map[string]DNSRCode{
	"":                    DNSRCodeUnrecognized,
	"unrecognized-rcode":  DNSRCodeUnrecognized,
	"no-error":            DNSRCodeNoError,
	"format-error":        DNSRCodeFormatErr,
	"server-fail":         DNSRCodeServFail,
	"non-existent-domain": DNSRCodeNXDomain,
	"not-implemented":     DNSRCodeNotImp,
	"query-refused":       DNSRCodeRefused,
}

DNSRCodeFromString : get DNSRCode from a string representation.

View Source 
var DNSRCodeToString = map[DNSRCode]string{
	DNSRCodeUnrecognized: "unrecognized-rcode",
	DNSRCodeNoError:      "no-error",
	DNSRCodeFormatErr:    "format-error",
	DNSRCodeServFail:     "server-fail",
	DNSRCodeNXDomain:     "non-existent-domain",
	DNSRCodeNotImp:       "not-implemented",
	DNSRCodeRefused:      "query-refused",
}

DNSRCodeToString : convert DNSRCode to string representation used in JSON.

View Source 
var DNSResTypeFromString = map[string]DNSResType{
	"":                  DNSResTypeUnrecognized,
	"unrecognized-type": DNSResTypeUnrecognized,
	"A":                 DNSResTypeA,
	"NS":                DNSResTypeNS,
	"CNAME":             DNSResTypeCNAME,
	"SOA":               DNSResTypeSOA,
	"WKS":               DNSResTypeWKS,
	"PTR":               DNSResTypePTR,
	"HINFO":             DNSResTypeHINFO,
	"MINFO":             DNSResTypeMINFO,
	"MX":                DNSResTypeMX,
	"TXT":               DNSResTypeTXT,
	"AAAA":              DNSResTypeAAAA,
	"SRV":               DNSResTypeSRV,
	"OPT":               DNSResTypeOPT,
	"AXFR":              DNSResTypeAXFR,
	"ALL":               DNSResTypeALL,
}

DNSResTypeFromString : get DNSResType from a string representation.

View Source 
var DNSResTypeToString = map[DNSResType]string{
	DNSResTypeUnrecognized: "unrecognized-type",
	DNSResTypeA:            "A",
	DNSResTypeNS:           "NS",
	DNSResTypeCNAME:        "CNAME",
	DNSResTypeSOA:          "SOA",
	DNSResTypeWKS:          "WKS",
	DNSResTypePTR:          "PTR",
	DNSResTypeHINFO:        "HINFO",
	DNSResTypeMINFO:        "MINFO",
	DNSResTypeMX:           "MX",
	DNSResTypeTXT:          "TXT",
	DNSResTypeAAAA:         "AAAA",
	DNSResTypeSRV:          "SRV",
	DNSResTypeOPT:          "OPT",
	DNSResTypeAXFR:         "AXFR",
	DNSResTypeALL:          "ALL",
}

DNSResTypeToString : convert DNSResType to string representation used in JSON.

View Source 
var SocketOpTypeFromString = map[string]SocketOpType{
	"":                SocketOpTypeUnrecognized,
	"unrecognized-op": SocketOpTypeUnrecognized,
	"read":            SocketOpTypeRead,
	"read-from":       SocketOpTypeReadFrom,
	"write":           SocketOpTypeWrite,
	"write-to":        SocketOpTypeWriteTo,
}

SocketOpTypeFromString : get SocketOpType from a string representation.

View Source 
var SocketOpTypeToString = map[SocketOpType]string{
	SocketOpTypeUnrecognized: "unrecognized-op",
	SocketOpTypeRead:         "read",
	SocketOpTypeReadFrom:     "read-from",
	SocketOpTypeWrite:        "write",
	SocketOpTypeWriteTo:      "write-to",
}

SocketOpTypeToString : convert SocketOpType to string representation used in JSON.

View Source 
var TCPStateFromString = map[string]TCPState{
	"":            TCPStateNone,
	"none":        TCPStateNone,
	"syn-sent":    TCPStateSynSent,
	"syn-recv":    TCPStateSynRecv,
	"established": TCPStateEstablished,
	"fin-wait":    TCPStateFinWait,
	"close-wait":  TCPStateCloseWait,
	"last-ack":    TCPStateLastAck,
	"time-wait":   TCPStateTimeWait,
	"close":       TCPStateClose,
	"syn-sent2":   TCPStateSynSent2,
}

TCPStateFromString : get TCPState from a string representation.

View Source 
var TCPStateToString = map[TCPState]string{
	TCPStateNone:        "none",
	TCPStateSynSent:     "syn-sent",
	TCPStateSynRecv:     "sync-recv",
	TCPStateEstablished: "established",
	TCPStateFinWait:     "fin-wait",
	TCPStateCloseWait:   "close-wait",
	TCPStateLastAck:     "last-ack",
	TCPStateTimeWait:    "time-wait",
	TCPStateClose:       "close",
	TCPStateSynSent2:    "syn-sent2",
}

TCPStateToString : convert TCPState to string representation used in JSON.

Functions

This section is empty.

Types

type AddrTuple 

type AddrTuple struct {
	// SrcIP : source IP address.
	SrcIP string `json:"srcIP"`
	// SrcPort : source port.
	SrcPort uint16 `json:"srcPort"`
	// DstIP : destination IP address.
	DstIP string `json:"dstIP"`
	// DstPort : destination port.
	DstPort uint16 `json:"dstPort"`
}

AddrTuple : source + destination addresses fully identifying a network connection. Whether this is from our side or a remote side, before or after NAT, depends on the context.

type AnyNetTrace 

type AnyNetTrace interface {
	// contains filtered or unexported methods
}

AnyNetTrace is implemented by NetTrace and all its extensions (like HTTPTrace). Can be used as a data type for methods that accept any kind network trace as an input.

type ConntrackStatus 

type ConntrackStatus uint32

ConntrackStatus : status of a conntrack entry (combination of flags, not enum).

func (ConntrackStatus) MarshalJSON 

func (s ConntrackStatus) MarshalJSON() ([]byte, error)

MarshalJSON marshals ConntrackStatus (flags) as a quoted json string.

func (*ConntrackStatus) UnmarshalJSON 

func (s *ConntrackStatus) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to ConntrackStatus (flags).

type ConntractEntry 

type ConntractEntry struct {
	// CapturedAt : time when this conntrack entry was obtained.
	CapturedAt Timestamp `json:"capturedAt"`
	// Status : conntrack connection's status flags.
	Status ConntrackStatus `json:"status"`
	// TCPState : state of the TCP connection.
	// TCPStateNone if this is a non-TCP (UDP) conntrack.
	TCPState TCPState `json:"tcpState,omitempty"`
	// Mark assigned to the connection by conntrack (CONNMARK).
	Mark uint32 `json:"mark"`
	// AddrOrig : source+dest addresses in the direction from the origin,
	// i.e. client->server, before NAT.
	AddrOrig AddrTuple `json:"addrOrig"`
	// AddrReply : source+dest addresses in the reply direction,
	// i.e. server->client, after NAT.
	AddrReply AddrTuple `json:"addrReply"`
	// PacketsSent : number of packets sent out towards the remote endpoint.
	PacketsSent uint64 `json:"packetsSent"`
	// PacketsRecv : number of packets received from the remote endpoint.
	PacketsRecv uint64 `json:"packetsRecv"`
	// BytesSent : number of bytes sent out towards the remote endpoint.
	BytesSent uint64 `json:"bytesSent"`
	// BytesRecv : number of bytes received from the remote endpoint.
	BytesRecv uint64 `json:"bytesRecv"`
}

ConntractEntry : single conntrack entry (one tracked connection). L4 protocol depends on the context, i.e. whether it is under TCPConnTrace or UDPConnTrace.

type DNSAnswer 

type DNSAnswer struct {
	// Name of the resource to which this record pertains.
	Name string `json:"name"`
	// Type of RR (A, AAAA, MX, TXT, etc.)
	Type DNSResType `json:"type"`
	// Class is the class of network to which this DNS resource record pertains.
	Class uint16 `json:"class"`
	// TTL is the length of time (measured in seconds) which this resource
	// record is valid for (time to live).
	TTL uint32 `json:"ttl"`
	// ResolvedVal content depends on the resource type. It can be an IP address
	// (A/AAAA), CNAME, NS, PTR, or MX (for others we do not include type-specific
	// answer attributes).
	ResolvedVal string `json:"resolvedVal,omitempty"`
}

DNSAnswer : single answer from DNS reply message.

type DNSQueryMsg 

type DNSQueryMsg struct {
	// SentAt : time when the message was sent (wrote into the socket).
	SentAt Timestamp `json:"sentAt"`
	// ID : identifier used to match DNS query with DNS reply.
	ID uint16 `json:"id"`
	// RecursionDesired : indicates if the client means a recursive query.
	RecursionDesired bool `json:"recursionDesired"`
	// Truncated : indicates that this message was truncated due to excessive length.
	Truncated bool `json:"truncated"`
	// Size of the message in bytes.
	Size uint32 `json:"size"`
	// Questions : DNS questions.
	Questions []DNSQuestion `json:"questions"`
	// OptUDPPayloadSize : the maximum UDP payload size that the requestor accepts.
	// It is specified inside the query message using EDNS (RFC 6891).
	OptUDPPayloadSize uint16 `json:"optUDPPayloadSize,omitempty"`
}

DNSQueryMsg : a single DNS query message.

type DNSQueryTrace 

type DNSQueryTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// FromDial : Reference to Dial where this originated from.
	FromDial TraceID `json:"fromDial"`
	// Connection : Reference to the trace record of the underlying UDP or TCP connection,
	// which was used to carry DNS request(s)/response(s).
	Connection TraceID `json:"connection"`
	// DNSQueryMsgs : all DNS query messages sent within this connection.
	DNSQueryMsgs []DNSQueryMsg `json:"dnsQueryMsgs"`
	// DNSReplyMsgs : all DNS reply messages received within this connection.
	DNSReplyMsgs []DNSReplyMsg `json:"dnsReplyMsgs"`
}

DNSQueryTrace : recording of a DNS query.

type DNSQueryTraces 

type DNSQueryTraces []DNSQueryTrace

DNSQueryTraces is a list of DNS query traces.

func (DNSQueryTraces) Get 

func (traces DNSQueryTraces) Get(id TraceID) *DNSQueryTrace

Get pointer to the DNS query trace with the given ID.

type DNSQuestion 

type DNSQuestion struct {
	// Name of the requested resource.
	Name string `json:"name"`
	// Type of RR (A, AAAA, MX, TXT, etc.)
	Type DNSResType `json:"type"`
	// Class code.
	Class uint16 `json:"class"`
}

DNSQuestion : single question from DNS query message.

type DNSRCode 

type DNSRCode uint16

DNSRCode : https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-6 

const (
	// DNSRCodeNoError : No error.
	DNSRCodeNoError DNSRCode = iota
	// DNSRCodeFormatErr : Format Error.
	DNSRCodeFormatErr
	// DNSRCodeServFail : Server Failure.
	DNSRCodeServFail
	// DNSRCodeNXDomain : Non-Existent Domain.
	DNSRCodeNXDomain
	// DNSRCodeNotImp : Not Implemented.
	DNSRCodeNotImp
	// DNSRCodeRefused : Query Refused.
	DNSRCodeRefused
	// DNSRCodeUnrecognized : used for every other RCode.
	// Note that other types of errors are unlikely to be encountered from a client
	// (and are not recognized by the DNS message parser that we use anyway).
	DNSRCodeUnrecognized = 65534 // not assigned by IANA
)

func (DNSRCode) MarshalJSON 

func (s DNSRCode) MarshalJSON() ([]byte, error)

MarshalJSON marshals the enum as a quoted json string.

func (*DNSRCode) UnmarshalJSON 

func (s *DNSRCode) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to the enum value.

type DNSReplyMsg 

type DNSReplyMsg struct {
	// RecvAt : time when the message was received (read from the socket).
	RecvAt Timestamp `json:"recvAt"`
	// ID : identifier used to match DNS query with DNS reply.
	ID uint16 `json:"id"`
	// Authoritative : indicates if the DNS server is authoritative for the queried hostname.
	Authoritative bool `json:"authoritative"`
	// RecursionAvailable : indicates if the replying DNS server supports recursion.
	RecursionAvailable bool `json:"recursionAvailable"`
	// Truncated : indicates that this message was truncated due to excessive length.
	Truncated bool `json:"truncated"`
	// Size of the message in bytes.
	Size uint32 `json:"size"`
	// RCode : Response code.
	RCode DNSRCode `json:"rCode"`
	// Answers : DNS answers.
	Answers []DNSAnswer `json:"answers"`
}

DNSReplyMsg : a single DNS reply message.

type DNSResType 

type DNSResType uint16

DNSResType : https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-4 

const (
	// DNSResTypeUnrecognized : unrecognized Resource Record (RR) type.
	// Note that RR types not listed here are unlikely to be encountered from a client
	// (and are not recognized by the DNS message parser that we use anyway).
	DNSResTypeUnrecognized DNSResType = iota // 0 is reserved

	// DNSResTypeA : 32-bit IPv4 address.
	DNSResTypeA DNSResType = 1
	// DNSResTypeNS : Name server record.
	DNSResTypeNS DNSResType = 2
	// DNSResTypeCNAME : Canonical name record.
	DNSResTypeCNAME DNSResType = 5
	// DNSResTypeSOA : Start of [a zone of] authority record.
	DNSResTypeSOA DNSResType = 6
	// DNSResTypeWKS : Well-known services supported by a host (obsolete record type).
	DNSResTypeWKS DNSResType = 11
	// DNSResTypePTR : Pointer to a canonical name.
	DNSResTypePTR DNSResType = 12
	// DNSResTypeHINFO : Host Information.
	DNSResTypeHINFO DNSResType = 13
	// DNSResTypeMINFO : Subscriber mailing lists (record type unlikely to be ever adopted).
	DNSResTypeMINFO DNSResType = 14
	// DNSResTypeMX : Mail exchange record.
	DNSResTypeMX DNSResType = 15
	// DNSResTypeTXT : Text record.
	DNSResTypeTXT DNSResType = 16
	// DNSResTypeAAAA : IPv6 address record.
	DNSResTypeAAAA DNSResType = 28
	// DNSResTypeSRV : Service locator.
	DNSResTypeSRV DNSResType = 33
	// DNSResTypeOPT : Pseudo-record type needed to support EDNS.
	DNSResTypeOPT DNSResType = 41
	// DNSResTypeAXFR : Authoritative Zone Transfer.
	DNSResTypeAXFR DNSResType = 252
	// DNSResTypeALL : All cached records.
	DNSResTypeALL DNSResType = 255
)

func (DNSResType) MarshalJSON 

func (s DNSResType) MarshalJSON() ([]byte, error)

MarshalJSON marshals the enum as a quoted json string.

func (*DNSResType) UnmarshalJSON 

func (s *DNSResType) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to the enum value.

type DialTrace 

type DialTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// DialBeginAt : time when the dial attempt started.
	DialBeginAt Timestamp `json:"dialBeginAt"`
	// DialEndAt : time when the dial attempt ended - either successfully with an established
	// connection or when it failed and gave up.
	DialEndAt Timestamp `json:"dialEndAt"`
	// DialErr : if dial failed, here is the reason.
	DialErr string `json:"dialErr,omitempty"`
	// CtxCloseAt : time when the context assigned to the dial attempt was closed/canceled
	// by the caller.
	CtxCloseAt Timestamp `json:"ctxCloseAt"`
	// DstAddress : address of the remote endpoint in the format <host>:<port>
	// where <host> is either IP address or a domain name.
	DstAddress string `json:"dstAddress"`
	// ResolverDials : connection attempts made by the resolver towards nameservers with
	// the aim of resolving <host> from DstAddress.
	ResolverDials []ResolverDialTrace `json:"resolverDials,omitempty"`
	// SkippedNameservers : nameservers which were configured in the OS but got skipped
	// (i.e. not used for DstAddress resolution) based on the user config
	// (for example using HTTPClientCfg.SkipNameserver).
	SkippedNameservers []string `json:"skippedNameservers,omitempty"`
	// SourceIP : source IP address statically configured for the dial request.
	// Empty if the source IP was not selected statically.
	SourceIP string `json:"sourceIP,omitempty"`
	// EstablishedConn : reference to an established TCP connection.
	EstablishedConn TraceID `json:"establishedConn,omitempty"`
}

DialTrace : recording of an attempt to establish TCP connection with a remote endpoint. The endpoint can be addressed using an IP address or a domain name.

type DialTraces 

type DialTraces []DialTrace

DialTraces is a list of Dial traces.

func (DialTraces) Get 

func (traces DialTraces) Get(id TraceID) *DialTrace

Get pointer to the Dial trace with the given ID.

type HTTPClient 

type HTTPClient struct {
	// This lock protects all attributes of the HTTPClient except for lockfree Queues
	// which do not require locking.
	sync.Mutex

	// The standard HTTP client is embedded and can be accessed simply as .Client
	// DO NOT change the Client.Transport field (to customize the HTTP client
	// behaviour), otherwise tracing functionality may get broken. Instead, configure
	// the desired behaviour of the HTTP client inside the HTTPClientCfg argument
	// of the HTTPClient constructor.
	*http.Client
	// contains filtered or unexported fields
}

HTTPClient wraps and enhances the standard HTTP client with tracing capabilities, i.e. monitoring and recording of network events related to the operations of the HTTP client, including HTTP requests made, TCP connections opened/attempted, TLS tunnels established/attempted, DNS queries sent, DNS answers received, etc.

func NewHTTPClient 

func NewHTTPClient(config HTTPClientCfg, traceOpts ...TraceOpt) (*HTTPClient, error)

NewHTTPClient creates a new instance of HTTPClient, enhancing the standard http.Client with tracing capabilities. Tracing starts immediately:

  • a background Go routine collecting traces is started
  • packet capture starts on selected interfaces if WithPacketCapture option was passed

func (*HTTPClient) ClearTrace 

func (c *HTTPClient) ClearTrace() error

ClearTrace effectively restarts tracing by removing all traces collected up to this point. If packet capture is enabled (WithPacketCapture), packets captured so far are deleted. However, note that if TCP connection is reused from a previous run, it will reappear in the HTTPTrace (returned by GetTrace()) with some attributes restored to their previously recorded values (like .HandshakeBeginAt) and some updated (for example .Reused will be set to true).

func (*HTTPClient) Close 

func (c *HTTPClient) Close() error

Close stops tracing of the embedded HTTP client, including packet capture if it was enabled. After this, it would be invalid to call GetTrace(), ClearTrace() or even to keep using the embedded HTTP Client.

func (*HTTPClient) GetTrace 

func (c *HTTPClient) GetTrace(description string) (HTTPTrace, []PacketCapture, error)

GetTrace returns a summary of all network and HTTP trace records (aka HTTPTrace), collected since the tracing last (re)started (either when the client was created or when the last ClearTrace() was called). This will include packet capture for every selected interface if it was enabled. The method allows to insert some description into the returned HTTPTrace (e.g. “download image XYZ”). Note that .TraceEndAt of the returned HTTPTrace is set to the current time. Also note that this does not stop tracing or clears the collected traces - use Close() or ClearTrace() for that.

type HTTPClientCfg 

type HTTPClientCfg struct {
	// PreferHTTP2, if true, will make the HTTP client to chose HTTP/2 as the preferred
	// HTTP version during the Application-Layer Protocol Negotiation (ALPN).
	PreferHTTP2 bool
	// SourceIP : source IP address to use for all connections and packets sent.
	// This includes all TCP connections opened for HTTP requests and UDP
	// packets sent with DNS requests.
	// Leave as nil to not bind sockets to any source IP address and instead let
	// the kernel to select the source IP address for each connection based on
	// the routing decision.
	SourceIP net.IP
	// SkipNameserver can be optionally provided as a callback to exclude some
	// of the system-wide configured DNS server(s) that would be otherwise used
	// for DNS queries.
	// The callback is called for every configured DNS server just before it is
	// queried. If the callback returns true, the server is skipped and the resolver
	// moves to the next one.
	// Every skipped nameserver is recorded in DialTrace.SkippedNameservers.
	SkipNameserver NameserverSelector
	// Proxy specifies a callback to return an address of a network proxy that
	// should be used for the given HTTP request.
	// If Proxy is nil or returns a nil *URL, no proxy is used.
	Proxy func(*http.Request) (*url.URL, error)
	// TLSClientConfig specifies the TLS configuration to use for TLS tunnels.
	// If nil, the default configuration is used.
	TLSClientConfig *tls.Config
	// ReqTimeout specifies a time limit for requests made by the HTTP client.
	// The timeout includes connection time, any redirects, and reading the response body.
	// The timer remains running after Get, Head, Post, or Do return and will interrupt
	// reading of the Response.Body.
	ReqTimeout time.Duration
	// TCPHandshakeTimeout specifies the maximum amount of time to wait for a TCP handshake
	// to complete. Zero means no timeout.
	TCPHandshakeTimeout time.Duration
	// TCPKeepAliveInterval specifies the interval between keep-alive probes for an active
	// TCP connection. If zero, keep-alive probes are sent with a default value (15 seconds),
	// if supported by the operating system.
	// If negative, keep-alive probes are disabled.
	TCPKeepAliveInterval time.Duration
	// TLSHandshakeTimeout specifies the maximum amount of time to wait for a TLS handshake
	// to complete. Zero means no timeout.
	TLSHandshakeTimeout time.Duration
	// DisableKeepAlive, if true, disables HTTP keep-alive and will only use the connection
	// to the server for a single HTTP request.
	DisableKeepAlive bool
	// DisableCompression, if true, prevents the Transport from requesting compression with
	// an "Accept-Encoding: gzip" request header when the Request contains no existing
	// Accept-Encoding value.
	DisableCompression bool
	// MaxIdleConns controls the maximum number of idle (keep-alive) connections across
	// all hosts. Zero means no limit.
	MaxIdleConns int
	// MaxIdleConnsPerHost, if non-zero, controls the maximum idle (keep-alive) connections
	// to keep per-host. If zero, DefaultMaxIdleConnsPerHost from the http package is used.
	MaxIdleConnsPerHost int
	// MaxConnsPerHost optionally limits the total number of connections per host,
	// including connections in the dialing, active, and idle states. On limit violation,
	// dials will block.
	// Zero means no limit.
	MaxConnsPerHost int
	// IdleConnTimeout is the maximum amount of time an idle (keep-alive) connection will
	// remain idle before closing itself.
	// Zero means no limit.
	IdleConnTimeout time.Duration
	// ResponseHeaderTimeout, if non-zero, specifies the amount of time to wait for a server's
	// response headers after fully writing the request (including its body, if any).
	// This time does not include the time to read the response body.
	ResponseHeaderTimeout time.Duration
	// ExpectContinueTimeout, if non-zero, specifies the amount of time to wait for a server's
	// first response headers after fully writing the request headers if the request has an
	// "Expect: 100-continue" header. Zero means no timeout and causes the body to be sent
	// immediately, without waiting for the server to approve.
	// This time does not include the time to send the request header.
	ExpectContinueTimeout time.Duration
}

HTTPClientCfg : configuration for the embedded HTTP client. This is not related to tracing but how the standard HTTP client itself should behave. Normally, HTTP client is configured by customizing the client's Transport (see https://pkg.go.dev/net/http#Transport). However, for the HTTP client tracing to function properly, Client.Transport, as installed and customized by the NewHTTPClient() constructor, should not be modified. The only allowed action is to additionally wrap the Transport with a RoundTripper implementation, which is allowed to for example modify HTTP requests/responses, but still should call the wrapped Transport for the HTTP request execution. An example of this is Transport from the oauth2 package, adding an Authorization header with a token: https://pkg.go.dev/golang.org/x/oauth2#Transport

type HTTPHeader 

type HTTPHeader []HTTPHeaderKV

HTTPHeader represents the key-value pairs in an HTTP header.

func (HTTPHeader) Get 

func (header HTTPHeader) Get(name string) *HTTPHeaderKV

Get pointer to the HTTP header field with the given name.

type HTTPHeaderKV 

type HTTPHeaderKV struct {
	// FieldName : Field name.
	FieldName string `json:"fieldName"`
	// FieldVal : Field value.
	// This can be hidden (returned as empty string even if the actual value is not empty)
	// by tracing options (can contain sensitive data).
	FieldVal string `json:"fieldVal,omitempty"`
	// FieldValLen : Length of the (actual, possibly hidden) field value (in characters).
	// Just like field value, this can be also hidden (returned as zero) using tracing
	// options (e.g. if knowing value length is enough to raise security concern).
	FieldValLen uint32 `json:"fieldValLen,omitempty"`
}

HTTPHeaderKV : a single HTTP message header field (a key-value pair).

type HTTPReqTrace 

type HTTPReqTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// TCPConn : reference to the underlying TCP connection used by the HTTP request.
	TCPConn TraceID `json:"tcpConn"`
	// TLSTunnel : TLS tunnel opened with the destination HTTPS server.
	TLSTunnel TraceID `json:"tlsTunnel,omitempty"`
	// ProxyTLSTunnel : TLS tunnel opened with an HTTPS proxy (if used, see NetworkProxy).
	ProxyTLSTunnel TraceID `json:"proxyTLSTunnel,omitempty"`
	// ProtoMajor : major number of the HTTP protocol version used for
	// request & response.
	ProtoMajor uint8 `json:"protoMajor"`
	// ProtoMinor : minor number of the HTTP protocol version used for
	// request & response.
	ProtoMinor uint8 `json:"protoMinor"`
	// NetworkProxy : address of a network proxy in the format scheme://host:port
	// Empty string if the HTTP request was not (explicitly) proxied.
	NetworkProxy string `json:"networkProxy,omitempty"`

	// ReqSentAt : time when the HTTP request was sent.
	ReqSentAt Timestamp `json:"reqSentAt"`
	// ReqMethod specifies the HTTP method of the request (GET, POST, PUT, etc.).
	// List of all standardized methods: https://www.iana.org/assignments/http-methods/http-methods.xhtml
	ReqMethod string `json:"reqMethod"`
	// ReqURL specifies the resource addressed by the request.
	ReqURL string `json:"reqURL"`
	// ReqHeader : request header.
	// If tracing of HTTP header fields is disabled (which it is by default), then this is
	// an empty slice.
	ReqHeader HTTPHeader `json:"reqHeader,omitempty"`
	// ReqContentLen : size of the HTTP request body content.
	// This may be available even if Content-Length header field is not.
	// But note that this only counts the part of the content that was actually loaded
	// by the HTTP client (if it was interrupted or the content transport failed and
	// the client gave up, this would not count the whole message body).
	// This is before transfer encoding is applied on the message body.
	ReqContentLen uint64 `json:"reqContentLen"`
	// ReqError : if the HTTP request failed, this is the reason.
	ReqError string `json:"reqError,omitempty"`

	// RespRecvAt : time when the HTTP response was received.
	RespRecvAt Timestamp `json:"respRecvAt"`
	// RespRecvAt : response status code.
	RespStatusCode int `json:"respStatusCode"`
	// RespHeader : response header.
	// If tracing of HTTP header fields is disabled (which it is by default), then this is
	// an empty slice.
	RespHeader HTTPHeader `json:"respHeader,omitempty"`
	// RespContentLen : number of bytes of the HTTP response body received and read
	// by the caller. This may be available even if Content-Length header field is not.
	// But note that if the caller didn't read all bytes until EOF and didn't close
	// the response body, this will not count the whole message body.
	// This is after the received content is decoded (HTTP transfer encoding).
	RespContentLen uint64 `json:"RespContentLen"`
}

HTTPReqTrace : recording of an HTTP request.

type HTTPReqTraces 

type HTTPReqTraces []HTTPReqTrace

HTTPReqTraces is a list of HTTP request traces.

func (HTTPReqTraces) Get 

func (traces HTTPReqTraces) Get(id TraceID) *HTTPReqTrace

Get pointer to the HTTP request trace with the given ID.

type HTTPTrace 

type HTTPTrace struct {
	NetTrace
	// HTTPRequests : all executed HTTP requests.
	HTTPRequests HTTPReqTraces `json:"httpRequests"`
}

HTTPTrace : recording of network operations performed by an HTTP client.

type HdrFieldsOpt 

type HdrFieldsOpt uint8

HdrFieldsOpt : options for capturing of HTTP header fields. 

const (
	// HdrFieldsOptDisabled : do not capture and include HTTP header fields
	// in HTTPReqTrace (may contain sensitive data).
	HdrFieldsOptDisabled HdrFieldsOpt = iota
	// HdrFieldsOptNamesOnly : record only header field names without values
	// and their length.
	// ExcludeHeaderField may still be used to completely filter out some header
	// fields.
	HdrFieldsOptNamesOnly
	// HdrFieldsOptValueLenOnly : for each header field record the name and
	// the value length, but not the value itself.
	// ExcludeHeaderField may still be used to completely filter out some header
	// fields.
	HdrFieldsOptValueLenOnly
	// HdrFieldsOptWithValues : record every header field including values.
	// ExcludeHeaderField may still be used to completely filter out some header
	// fields.
	HdrFieldsOptWithValues
)

type Logger 

type Logger interface {
	// Tracef : formatted log message with info useful for finer-grained debugging.
	Tracef(format string, args ...interface{})
	// Debugf : formatted log message with info useful for debugging.
	Debugf(format string, args ...interface{})
	// Infof : formatted log message with a general info about what's going on
	// inside the application.
	Infof(format string, args ...interface{})
	// Warningf : formatted log message with a warning.
	Warningf(format string, args ...interface{})
	// Errorf : formatted log message with an error.
	Errorf(format string, args ...interface{})
	// Fatalf : formatted log message with an error, ending with a call to os.Exit()
	// with a non-zero return value.
	Fatalf(format string, args ...interface{})
	// Panicf : formatted log message with an error, raising a panic.
	Panicf(format string, args ...interface{})
}

Logger is used to log noteworthy events happening inside the network tracing engine.

type NameserverSelector 

type NameserverSelector func(ipAddr net.IP, port uint16) (skip bool, reason string)

NameserverSelector is a function that for a given nameserver decides whether it should be used for name resolution or skipped.

type NetTrace 

type NetTrace struct {
	// Description provided by the caller.
	Description string `json:"description"`
	// TraceBeginAt : (absolute) timestamp of the moment when the tracing started.
	TraceBeginAt Timestamp `json:"traceBeginAt"`
	// TraceEndAt : time (relative to TraceBeginAt) when the tracing ended.
	TraceEndAt Timestamp `json:"traceEndAt"`
	// Dials : all attempts to establish connection with a remote endpoint.
	Dials DialTraces `json:"dials"`
	// TCPConns : all established or failed TCP connections.
	TCPConns TCPConnTraces `json:"tcpConns"`
	// UDPConns : all UDP connections (successful or failed exchanges of UDP datagrams).
	UDPConns UDPConnTraces `json:"udpConns"`
	// DNSQueries : all performed DNS queries.
	// Empty if WithDNSQueryTrace is not enabled.
	DNSQueries DNSQueryTraces `json:"dnsQueries"`
	// TLSTunnels : all opened (or attempted to open) TLS tunnels.
	TLSTunnels TLSTunnelTraces `json:"tlsTunnels"`
}

NetTrace : recording of network operations performed by a client program (e.g. HTTP client).

type PacketCapture 

type PacketCapture struct {
	// InterfaceName : name of the interface on which the packets were captured
	// (on either direction).
	InterfaceName string
	// SnapLen is the maximum number of bytes captured for each packet.
	// Larger packets are (silently) returned truncated.
	SnapLen uint32
	// Packets : captured packets.
	Packets []gopacket.Packet
	// Truncated is returned as true if the capture does not contain all packets
	// because the maximum allowed total size would be exceeded otherwise.
	Truncated bool
	// WithTCPPayload : true if packet capture was configured to include also
	// TCP packets with non empty payload.
	WithTCPPayload bool
}

PacketCapture is a recording of all/some packets that arrived or left through a given interface. This is typically included alongside NetTrace and captured packets are filtered to contain only those that correspond with the traced connections.

func (PacketCapture) WriteTo 

func (pc PacketCapture) WriteTo(w io.Writer) error

WriteTo writes packet capture to a file or a buffer or whatever w represents.

func (PacketCapture) WriteToFile 

func (pc PacketCapture) WriteToFile(filename string) error

WriteToFile saves packet capture to a given file.

type PeerCert 

type PeerCert struct {
	// Subject describes the certificate subject (roughly following
	// the RFC 2253 Distinguished Names syntax).
	Subject string `json:"subject"`
	// Issuer describes the certificate issuer (roughly following
	// the RFC 2253 Distinguished Names syntax).
	Issuer string `json:"issuer"`
	// NotBefore : date and time on which the certificate becomes valid.
	NotBefore Timestamp `json:"notBefore"`
	// NotAfter : date and time after which the certificate is no longer valid.
	NotAfter Timestamp `json:"notAfter"`
	// IsCA : true if this certificate corresponds to a certificate authority.
	IsCA bool `json:"isCA"`
}

PeerCert : description of a peer certificate.

type ResolverDialTrace 

type ResolverDialTrace struct {
	// DialBeginAt : time when the dial attempt started.
	DialBeginAt Timestamp `json:"dialBeginAt"`
	// DialEndAt : time when the dial attempt ended - either successfully with an established
	// connection or when it failed and gave up.
	DialEndAt Timestamp `json:"dialEndAt"`
	// DialErr : if dial failed, here is the reason.
	DialErr string `json:"dialErr,omitempty"`
	// Nameserver : destination nameserver address in the format <host>:<port>.
	Nameserver string `json:"nameserver"`
	// EstablishedConn : reference to an established UDP or TCP connection.
	EstablishedConn TraceID `json:"establishedConn,omitempty"`
}

ResolverDialTrace : recording of a resolver's attempt to establish UDP or TCP connection with a nameserver.

type SocketOp 

type SocketOp struct {
	// Type of the operation.
	Type SocketOpType `json:"type"`
	// CallAt : Time when the socket operation was initiated by the caller.
	CallAt Timestamp `json:"callAt"`
	// ReturnAt : Time when the socket operation returned.
	ReturnAt Timestamp `json:"returnAt"`
	// ReturnErr : error returned by the operation (if any).
	ReturnErr string `json:"returnErr,omitempty"`
	// RemoteAddr : with packet-oriented operation (readFrom, writeTo), this field
	// will contain address of the remote endpoint from which the packet was received
	// or to which it was sent. The address is in the format host:port.
	RemoteAddr string `json:"remoteAddr,omitempty"`
	// DataLen : number of read/written bytes.
	DataLen uint32 `json:"dataLen"`
}

SocketOp : single I/O operation performed over AF_INET(6) socket.

type SocketOpType 

type SocketOpType uint8

SocketOpType : operations that can be performed over AF_INET(6) socket. 

const (
	// SocketOpTypeUnrecognized : operation is not recognized.
	SocketOpTypeUnrecognized SocketOpType = iota
	// SocketOpTypeRead : read bytes from connected socket.
	SocketOpTypeRead
	// SocketOpTypeReadFrom : read packet from connected socket.
	// (also see SocketOp.RemoteAddr)
	SocketOpTypeReadFrom
	// SocketOpTypeWrite : write bytes to connected socket.
	SocketOpTypeWrite
	// SocketOpTypeWriteTo : write packet destined to a given address
	// (see SocketOp.RemoteAddr).
	SocketOpTypeWriteTo
)

func (SocketOpType) MarshalJSON 

func (s SocketOpType) MarshalJSON() ([]byte, error)

MarshalJSON marshals the enum as a quoted json string.

func (*SocketOpType) UnmarshalJSON 

func (s *SocketOpType) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to the enum value.

type SocketTrace 

type SocketTrace struct {
	SocketOps []SocketOp `json:"socketOps"`
}

SocketTrace : recording of I/O operations performed over AF_INET(6) socket.

type TCPConnTrace 

type TCPConnTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// FromDial : Reference to Dial where this originated from.
	FromDial TraceID `json:"fromDial"`
	// FromResolver : true if this connection was opened from the resolver
	// and towards a nameserver.
	FromResolver bool `json:"fromResolver,omitempty"`
	// HandshakeBeginAt : time when the TCP handshake process started (SYN packet was sent).
	HandshakeBeginAt Timestamp `json:"handshakeBeginAt"`
	// HandshakeEndAt : time when the handshake process ended - either successfully with
	// an established TCP connection or with a failure (canceled, timeouted, refused, ...).
	HandshakeEndAt Timestamp `json:"handshakeEndAt"`
	// Connected is true if the handshake succeeded to establish connection.
	// If this is false, reason of the failure can be available as part of DialTrace (.DialErr).
	Connected bool `json:"connected"`
	// ConnCloseAt : time when the connection was closed (from our side).
	ConnCloseAt Timestamp `json:"connCloseAt"`
	// AddrTuple : 4-tuple with source + destination addresses identifying the TCP connection.
	AddrTuple AddrTuple `json:"addrTuple"`
	// Reused : was this TCP connection reused between separately recorded NetTrace records?
	// For example, if two HTTP requests are separately traced (producing two NetTrace instances),
	// the first one will have recording of a new TCP connection, while the second one will
	// repeat the same TCPConnTrace, with some updates for the second request and Reused=true.
	Reused bool `json:"reused"`
	// TotalSentBytes : total number of bytes sent as a TCP payload through this connection.
	// (i.e. TCP header and lower-layer headers are not included)
	TotalSentBytes uint64 `json:"totalSentBytes"`
	// TotalRecvBytes : total number of bytes received as a TCP payload through this connection.
	// (i.e. TCP header and lower-layer headers are not included)
	TotalRecvBytes uint64 `json:"totalRecvBytes"`
	// Conntract : conntrack entry (provided by Netfilter connection tracking system) corresponding
	// to this connection.
	// Nil if not available or if conntrack tracing was disabled.
	// Conntrack entry is taken as late as possible, i.e. preferably after the connection closes
	// but before the conntrack entry timeouts and is removed. This is to ensure that packet/byte
	// counters and conntrack/TCP states cover the entirety of the connection.
	Conntract *ConntractEntry `json:"conntrack,omitempty"`
	// SocketTrace : recording of socket operations (read, write).
	// Nil if socket tracing was not enabled.
	SocketTrace *SocketTrace `json:"socketTrace,omitempty"`
}

TCPConnTrace : recording of an established or even just attempted but not completed TCP connection.

type TCPConnTraces 

type TCPConnTraces []TCPConnTrace

TCPConnTraces is a list of TCP connection traces.

func (TCPConnTraces) Get 

func (traces TCPConnTraces) Get(id TraceID) *TCPConnTrace

Get pointer to the TCP connection trace with the given ID.

type TCPState 

type TCPState uint8

TCPState : TCP connection states as observed by conntrack. See tcp_conntrack_names in netfilter/nf_conntrack_proto_tcp.c 

const (
	// TCPStateNone : TCP state is not defined/available.
	TCPStateNone TCPState = iota
	// TCPStateSynSent : SYN-only packet seen (TCP establishment 3-way handshake)
	TCPStateSynSent
	// TCPStateSynRecv : SYN-ACK packet seen (TCP establishment 3-way handshake)
	TCPStateSynRecv
	// TCPStateEstablished : ACK packet seen (TCP establishment 3-way handshake)
	TCPStateEstablished
	// TCPStateFinWait : FIN packet seen (TCP termination 4-way handshake)
	TCPStateFinWait
	// TCPStateCloseWait : ACK seen (after FIN) (TCP termination 4-way handshake)
	TCPStateCloseWait
	// TCPStateLastAck :  FIN seen (after FIN) (TCP termination 4-way handshake)
	TCPStateLastAck
	// TCPStateTimeWait : last ACK seen (TCP termination 4-way handshake)
	TCPStateTimeWait
	// TCPStateClose : closed connection (RST)
	TCPStateClose
	// TCPStateSynSent2 : SYN-only packet seen from reply dir, simultaneous open.
	TCPStateSynSent2
)

func (TCPState) MarshalJSON 

func (s TCPState) MarshalJSON() ([]byte, error)

MarshalJSON marshals the enum as a quoted json string.

func (*TCPState) UnmarshalJSON 

func (s *TCPState) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to the enum value.

type TIDGenerator 

type TIDGenerator func() TraceID

TIDGenerator is a function that generates unique IDs for network traces. 

var IDGenerator TIDGenerator = AtomicCounterID

IDGenerator by default uses AtomicCounterID to generate network trace IDs. It is exported and can be changed.

type TLSTunnelTrace 

type TLSTunnelTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// TCPConn : reference to TCP connection over which the tunnel was established
	// (or attempted to be established).
	TCPConn TraceID `json:"tcpConn"`
	// HandshakeBeginAt : time when the TLS handshake process started (ClientHello was sent).
	HandshakeBeginAt Timestamp `json:"handshakeBeginAt"`
	// HandshakeEndAt : time when the handshake process ended - either successfully with
	// an established TLS tunnel or with a failure (canceled, timeouted, refused, ...).
	HandshakeEndAt Timestamp `json:"handshakeEndAt"`
	// HandshakeErr : if handshake failed to establish, here is the reason.
	HandshakeErr string `json:"handshakeErr,omitempty"`
	// DidResume is true if this connection was successfully resumed from a
	// previous session with a session ticket or similar mechanism.
	DidResume bool `json:"didResume"`
	// PeerCerts are the certificates sent by the peer, in the order in which they were sent.
	// (When TLS handshake succeeds) The first element is the leaf certificate that
	// the connection is verified against.
	// However, when TLS handshake fails it might not be possible to obtain all certificates
	// and typically only one will be included (e.g. the problematic one).
	PeerCerts []PeerCert `json:"peerCerts"`
	// CipherSuite is the cipher suite negotiated for the connection (e.g.
	// TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_AES_128_GCM_SHA256).
	// See: https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-4
	CipherSuite uint16 `json:"cipherSuite"`
	// NegotiatedProtocol is the application protocol negotiated with ALPN.
	// (e.g. HTTP/1.1, h2)
	NegotiatedProto string `json:"negotiatedProto,omitempty"`
	// ServerName is the value of the Server name Indication (SNI) extension sent by
	// the client. It's available both on the server and on the client side.
	ServerName string `json:"serverName"`
}

TLSTunnelTrace : recording of a TLS tunnel establishment (successful or a failed attempt).

type TLSTunnelTraces 

type TLSTunnelTraces []TLSTunnelTrace

TLSTunnelTraces is a list of TLS tunnel traces.

func (TLSTunnelTraces) Get 

func (traces TLSTunnelTraces) Get(id TraceID) *TLSTunnelTrace

Get pointer to the TLS tunnel trace with the given ID.

type Timestamp 

type Timestamp struct {
	// Abs : Absolute time. Used when absolute time is needed (e.g. start of tracing)
	// or when relative time is not appropriate (e.g. reused connection would have
	// negative Rel time).
	// Ignore if IsRel=true.
	Abs time.Time
	// IsRel : true if this timestamp is relative (Rel should be read instead of Abs)
	IsRel bool
	// Number of milliseconds elapsed since NetTrace.TraceBeginAt.
	Rel uint32
}

Timestamp : absolute or relative timestamp for a traced event. Zero value (IsRel is False && Abs.IsZero() is true) represents undefined timestamp.

func (Timestamp) Add 

func (t Timestamp) Add(relT Timestamp) Timestamp

Add relative timestamp to absolute timestamp and get absolute timestamp.

func (Timestamp) Elapsed 

func (t Timestamp) Elapsed() Timestamp

Elapsed returns how much time elapsed since t. t must be absolute timestamp. Returned timestamp is relative.

func (Timestamp) MarshalJSON 

func (t Timestamp) MarshalJSON() ([]byte, error)

MarshalJSON marshals Timestamp as a quoted json string.

func (Timestamp) Sub 

func (t Timestamp) Sub(t2 Timestamp) time.Duration

Sub returns the duration t-t2. It is required that timestamps are of the same type - either both relative or both absolute.

func (Timestamp) Undefined 

func (t Timestamp) Undefined() bool

Undefined returns true when timestamp is not defined.

func (*Timestamp) UnmarshalJSON 

func (t *Timestamp) UnmarshalJSON(b []byte) error

UnmarshalJSON un-marshals a quoted json string to Timestamp.

type TraceID 

type TraceID string

TraceID : identifier for a trace record (of any type - can be DialTrace, TCPConnTrace, etc.).

func AtomicCounterID 

func AtomicCounterID() TraceID

AtomicCounterID atomically increments integer and returns it as the trace ID in the decimal format and prefixed with "tid-". Generated IDs are very concise but guarantee uniqueness only within a single execution of one process (which is the minimum requirement for TraceID).

func ShortUUID 

func ShortUUID() TraceID

ShortUUID can be used as IDGenerator to produce a shorter variant of universally unique identifiers for network traces. To use this instead of the default AtomicCounterID, add to your code: 

func init() {
    nettrace.IDGenerator = nettrace.ShortUUID
}

func (TraceID) Undefined 

func (t TraceID) Undefined() bool

Undefined returns true if the ID is not defined (empty).

type TraceOpt 

type TraceOpt interface {
	// contains filtered or unexported methods
}

TraceOpt allows to customize tracing of network events.

type TraceOptWithDefaults 

type TraceOptWithDefaults interface {
	TraceOpt
	// contains filtered or unexported methods
}

TraceOptWithDefaults is implemented by options that have some non-zero default values.

type UDPConnTrace 

type UDPConnTrace struct {
	// TraceID : trace identifier for reference.
	TraceID TraceID `json:"traceID"`
	// FromDial : Reference to Dial where this originated from.
	FromDial TraceID `json:"fromDial"`
	// FromResolver : true if this connection was opened from the resolver
	// and towards a nameserver.
	FromResolver bool `json:"fromResolver,omitempty"`
	// SocketCreateAt : time when the UDP socket was created.
	SocketCreateAt Timestamp `json:"socketCreateAt"`
	// ConnCloseAt : time when the connection was closed (from our side).
	ConnCloseAt Timestamp `json:"connCloseAt"`
	// AddrTuple : 4-tuple with source + destination addresses identifying the UDP connection.
	AddrTuple AddrTuple `json:"addrTuple"`
	// TotalSentBytes : total number of bytes sent as a UDP payload through this connection.
	// (i.e. UDP header and lower-layer headers are not included)
	TotalSentBytes uint64 `json:"totalSentBytes"`
	// TotalRecvBytes : total number of bytes received as a UDP payload through this connection.
	// (i.e. UDP header and lower-layer headers are not included)
	TotalRecvBytes uint64 `json:"totalRecvBytes"`
	// Conntract : conntrack entry (provided by Netfilter connection tracking system) corresponding
	// to this connection.
	// Nil if not available or if conntrack tracing was disabled.
	// Conntrack entry is taken as late as possible, i.e. preferably after the connection closes
	// but before the conntrack entry timeouts and is removed. This is to ensure that packet/byte
	// counters and conntrack/UDP states cover the entirety of the connection.
	Conntract *ConntractEntry `json:"conntrack,omitempty"`
	// SocketTrace : recording of socket operations (read, write).
	// Nil if socket tracing was not enabled.
	SocketTrace *SocketTrace `json:"socketTrace,omitempty"`
}

UDPConnTrace : recording of a UDP connection (unreliable exchange of UDP datagrams between our UDP client and a remote UDP peer).

type UDPConnTraces 

type UDPConnTraces []UDPConnTrace

UDPConnTraces is a list of UDP connection traces.

func (UDPConnTraces) Get 

func (traces UDPConnTraces) Get(id TraceID) *UDPConnTrace

Get pointer to the UDP connection trace with the given ID.

type WithConntrack 

type WithConntrack struct {
}

WithConntrack : obtain and include conntrack entries (provided by netfilter) inside the networkTrace records of TCP and UDP connections (TCPConnTrace.Conntrack and UDPConnTrace.Conntrack).

type WithDNSQueryTrace 

type WithDNSQueryTrace struct {
}

WithDNSQueryTrace : enable tracing of DNS queries and their responses (requires to parse DNS messages sent over a socket). DNSQueryTrace-s are stored under NetTrace.DNSQueries.

type WithHTTPReqTrace 

type WithHTTPReqTrace struct {
	// HeaderFields : specify how HTTP header fields should be recorded.
	HeaderFields HdrFieldsOpt
	// ExcludeHeaderField is a callback that can be optionally specified to filter
	// out some HTTP header fields from being recorded (by returning true).
	ExcludeHeaderField func(key string) bool
}

WithHTTPReqTrace : enable tracing of HTTP requests and their responses. This requires to put a custom RoundTripper implementation under http.Client.Transport. However, some libraries that take HTTP client as an argument may expect that Transport is of type http.Transport (the standard implementation). In such cases, it is necessary to disable HTTP request tracing. As an unfortunate side effect, HTTPTrace returned by HTTPClient will also miss TLSTunnels, which it has no way of capturing them anymore.

type WithLogging 

type WithLogging struct {
	CustomLogger Logger
}

WithLogging : enable logging inside the network tracing engine. When enabled then by default Logrus from Sirupsen will be used (see github.com/sirupsen/logrus), but a custom logger can be provided instead.

type WithPacketCapture 

type WithPacketCapture struct {
	// Interfaces to capture packets from.
	Interfaces []string
	// PacketSnaplen : maximum size in bytes to read for each packet.
	// Larger packets will be (silently) returned truncated.
	// Default snaplen is 1518 bytes.
	PacketSnaplen uint32
	// TotalSizeLimit : total limit in bytes for all captured packets.
	// Once the limit is reached, further captured packets are dropped or the pcap process
	// is completely stopped/paused. To indicate that this happened, the returned
	// PacketCapture will have .Truncated set to true.
	// Default upper limit for pcap size is 1MiB.
	TotalSizeLimit uint32
	// IncludeICMP : if enabled, all sent/received ICMP packets will be captured as well.
	// This can be useful for troubleshooting purposes.
	IncludeICMP bool
	// IncludeARP : if enabled, all sent/received ARP packets will be captured as well.
	// This can be useful for troubleshooting purposes.
	IncludeARP bool
	// TCPWithoutPayload : if enabled, then for TCP only packets that do not carry any
	// payload will be captured. For example, this will include SYN, RST and FIN packets
	// as well as ACK packets without data piggybacking.
	TCPWithoutPayload bool
}

WithPacketCapture : run packet capture on selected interfaces (in both directions). Captured packets are typically filtered to contain only those that correspond to traced connections. Packet capture is returned as PacketCapture - one per each interface.

type WithSockTrace 

type WithSockTrace struct {
}

WithSockTrace : record read/write operations performed over AF_INET sockets of traced TCP and UDP connections (stored under TCPConnTrace.SocketTrace and UDPConnTrace.SocketTrace).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.