measurexlite

package
v0.27.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2023 License: GPL-3.0 Imports: 20 Imported by: 2

Documentation

Overview

Package measurexlite contains measurement extensions. This package is named "measurex lite" because it implements a lightweight approach compared to a previous package named "measurex".

measurexlite implements the dd-003-step-by-step.md design document. The fundamental data type is the *Trace, which saves events in buffered channels. The NewTrace constructor creates channels with sufficient capacity for tracing all the events we expect to see for a single use connection or for a DNS round trip. If you are not draining the channels, the *Trace will eventually stop collecting events, though.

As mentioned above, the expectation is that a *Trace will only trace a single use connection or a DNS round trip. Typically, you create a distinct trace for each TCP-TLS-HTTP or TCP-HTTP or QUIC-HTTP or DNS-lookup-with-getaddrinfo or DNS-lookup-with-UDP sequence of operations. There is a "trace ID" for each trace, which you provide to NewTrace. This ID is copied into the "transaction_id" field of the archival network events. Therefore, by using distinct trace IDs for distinct operations, you enable ooni/data to group related events together.

The *Trace features methods that mirror existing netxlite methods but implement support for collecting network events using the *Trace. For example, *Trace.NewStdlibResolver is like netxlite.NewStdlibResolver but the DNS lookups performed with the resolved returned by *Trace.NewStdlibResolver generate events that you can collect using the *Trace.

As mentioned above, internally, the *Trace uses buffered channels on which the underlying network objects attempt to write when there is an interesting event. As a user of the measurexlite package, you have methods to extract the events from the *Trace channels, such as, for example:

- *Trace.DNSLookupsFromRoundTrip

- *Trace.NetworkEvents

- *Trace.TCPConnects

- *Trace.QUICHandshakes

- *Trace.TLSHandshakes

These methods already return data structures using the archival data format implemented by the model package and specified in the ooni/spec repository. Hence, these structures are ready to be added to OONI measurements.

Index

Constants

View Source 
const DNSLookupBufferSize = 8

DNSLookupBufferSize is the *Trace buffer size for DNS lookup events.

View Source 
const DelayedDNSResponseBufferSize = 8

DNSResponseBufferSize is the *Trace buffer size for delayed DNS responses events.

View Source 
const NetworkEventBufferSize = 64

NetworkEventBufferSize is the *Trace buffer size for network I/O events.

View Source 
const QUICHandshakeBufferSize = 8

QUICHandshakeBufferSize is the *Trace buffer size for QUIC handshake events.

View Source 
const TCPConnectBufferSize = 8

TCPConnectBufferSize is the *Trace buffer size for TCP connect events.

View Source 
const TLSHandshakeBufferSize = 8

TLSHandshakeBufferSize is the *Trace buffer size for TLS handshake events.

Variables

View Source 
var ErrDelayedDNSResponseBufferFull = errors.New(
	"measurexlite: the delayed-DNS-response channel buffer is full")

ErrDelayedDNSResponseBufferFull indicates that the delayed-DNS-response channel buffer is full.

Functions

func MaybeClose 

func MaybeClose(conn net.Conn) (err error)

MaybeClose is a convenience function for closing a net.Conn when it is not nil.

func MaybeCloseQUICConn 

func MaybeCloseQUICConn(conn quic.EarlyConnection) (err error)

MaybeCloseQUICConn is a convenience function for closing a quic.EarlyConnection when it is not nil.

func MaybeCloseUDPLikeConn 

func MaybeCloseUDPLikeConn(conn model.UDPLikeConn) (err error)

MaybeCloseUDPLikeConn is a convenience function for closing a model.UDPLikeConn when it is not nil.

func NewAnnotationArchivalNetworkEvent 

func NewAnnotationArchivalNetworkEvent(
	index int64, time time.Duration, operation string, tags ...string) *model.ArchivalNetworkEvent

NewAnnotationArchivalNetworkEvent is a simplified NewArchivalNetworkEvent where we create a simple annotation without attached I/O info.

func NewArchivalDNSLookupResultFromRoundTrip 

func NewArchivalDNSLookupResultFromRoundTrip(index int64, started time.Duration,
	reso DNSNetworkAddresser, query model.DNSQuery, response model.DNSResponse,
	addrs []string, err error, finished time.Duration, tags ...string) *model.ArchivalDNSLookupResult

NewArchivalDNSLookupResultFromRoundTrip generates a model.ArchivalDNSLookupResultFromRoundTrip from the available information right after the DNS RoundTrip

func NewArchivalHTTPRequestResult 

func NewArchivalHTTPRequestResult(index int64, started time.Duration, network, address, alpn string,
	transport string, req *http.Request, resp *http.Response, maxRespBodySize int64, body []byte,
	err error, finished time.Duration, tags ...string) *model.ArchivalHTTPRequestResult

NewArchivalHTTPRequestResult creates a new model.ArchivalHTTPRequestResult.

Arguments:

- index is the index of the trace;

- started is when we started sending the request;

- network is the underlying network in use ("tcp" or "udp");

- address is the remote endpoint's address;

- alpn is the negotiated ALPN or an empty string when not applicable;

- transport is the HTTP transport's protocol we're using ("udp" or "tcp"): this field was introduced a long time ago to support QUIC measurements and we keep it for backwards compatibility but network, address, and alpn are much more informative;

- req is the certainly-non-nil HTTP request;

- resp is the possibly-nil HTTP response;

- maxRespBodySize is the maximum body snapshot size;

- body is the possibly-nil HTTP response body;

- err is the possibly-nil error that occurred during the transaction;

- finished is when we finished reading the response's body;

- tags contains optional tags to fill the .Tags field in the archival format.

func NewArchivalNetworkEvent 

func NewArchivalNetworkEvent(index int64, started time.Duration, operation string,
	network string, address string, count int, err error, finished time.Duration,
	tags ...string) *model.ArchivalNetworkEvent

NewArchivalNetworkEvent creates a new model.ArchivalNetworkEvent.

func NewArchivalTCPConnectResult 

func NewArchivalTCPConnectResult(index int64, started time.Duration, address string,
	err error, finished time.Duration, tags ...string) *model.ArchivalTCPConnectResult

NewArchivalTCPConnectResult generates a model.ArchivalTCPConnectResult from the available information right after connect returns.

func NewArchivalTLSOrQUICHandshakeResult 

func NewArchivalTLSOrQUICHandshakeResult(
	index int64, started time.Duration, network string, address string, config *tls.Config,
	state tls.ConnectionState, err error, finished time.Duration,
	tags ...string) *model.ArchivalTLSOrQUICHandshakeResult

NewArchivalTLSOrQUICHandshakeResult generates a model.ArchivalTLSOrQUICHandshakeResult from the available information right after the TLS handshake returns.

func NewFailure 

func NewFailure(err error) *string

NewFailure creates an OONI failure from an error. If the error is nil, we return nil. If the error is not already an ErrWrapper, it's converted to an ErrWrapper. If the ErrWrapper's Failure is not empty, we return its string representation. Otherwise we return a string indicating that an ErrWrapper has an empty failure (should not happen).

See https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md for more information about OONI failures.

func TLSPeerCerts 

func TLSPeerCerts(
	state tls.ConnectionState, err error) (out []model.ArchivalBinaryData)

TLSPeerCerts extracts the certificates either from the list of certificates in the connection state or from the error that occurred.

func WebGetTitle 

func WebGetTitle(measurementBody string) string

WebGetTitle returns the title or an empty string.

Types

type DNSNetworkAddresser 

type DNSNetworkAddresser interface {
	// Address is like model.DNSTransport.Address
	Address() string

	// Network is like model.DNSTransport.Network
	Network() string
}

DNSNetworkAddresser is the type of something we just used to perform a DNS round trip (e.g., model.DNSTransport, model.Resolver) that allows us to get the network and the address of the underlying resolver/transport.

type Trace 

type Trace struct {

	// Netx is the network to use for measuring. The constructor inits this
	// field using a [*netxlite.Netx]. You MAY override this field for testing. Make
	// sure you do that before you start measuring to avoid data races.
	Netx model.MeasuringNetwork
	// contains filtered or unexported fields
}

Trace implements model.Trace. We use a context.Context to register ourselves as the model.Trace and we implement the model.Trace callbacks to route events into internal buffered channels as explained in by the measurexlite package documentation. The zero-value of this struct is invalid. To construct use NewTrace.

NewTrace uses reasonable buffer sizes for the channels used for collecting events. You should drain the channels used by this implementation after each operation you perform (that is, we expect you to peform step-by-step measurements). As mentioned in the measurexlite package documentation, there are several methods for extracting events from the *Trace.

func NewTrace 

func NewTrace(index int64, zeroTime time.Time, tags ...string) *Trace

NewTrace creates a new instance of Trace using default settings.

We create buffered channels using as buffer sizes the constants that are also defined by this package.

Arguments:

- index is the unique index of this trace within the current measurement (use zero if you don't care about giving this trace a unique ID);

- zeroTime is the time when we started the current measurement;

- tags contains optional tags to mark the archival data formats specially (e.g., to identify that some traces belong to some submeasurements).

func (*Trace) CloneBytesReceivedMap added in v0.26.0 

func (tx *Trace) CloneBytesReceivedMap() (out map[string]int64)

CloneBytesReceivedMap returns a clone of the internal bytes received map. The key of the map is a string following the "EPNT_ADDRESS PROTO" pattern where the "EPNT_ADDRESS" contains the endpoint address and "PROTO" is "tcp" or "udp".

func (*Trace) DNSLookupsFromRoundTrip 

func (tx *Trace) DNSLookupsFromRoundTrip() (out []*model.ArchivalDNSLookupResult)

DNSLookupsFromRoundTrip drains the network events buffered inside the DNSLookup channel

func (*Trace) DelayedDNSResponseWithTimeout 

func (tx *Trace) DelayedDNSResponseWithTimeout(ctx context.Context,
	timeout time.Duration) (out []*model.ArchivalDNSLookupResult)

DelayedDNSResponseWithTimeout drains the network events buffered inside the delayedDNSResponse channel. We construct a child context based on [ctx] and the given [timeout] and we stop reading when original [ctx] has been cancelled or the given [timeout] expires, whatever happens first. Once the timeout expired, we drain the chan as much as possible before returning.

func (*Trace) FirstDNSLookup 

func (tx *Trace) FirstDNSLookup() *model.ArchivalDNSLookupResult

FirstDNSLookupOrNil drains the network events buffered inside the DNSLookup channel and returns the first DNSLookup, if any. Otherwise, it returns nil.

func (*Trace) FirstNetworkEventOrNil 

func (tx *Trace) FirstNetworkEventOrNil() *model.ArchivalNetworkEvent

FirstNetworkEventOrNil drains the network events buffered inside the NetworkEvents channel and returns the first NetworkEvent, if any. Otherwise, it returns nil.

func (*Trace) FirstQUICHandshakeOrNil 

func (tx *Trace) FirstQUICHandshakeOrNil() *model.ArchivalTLSOrQUICHandshakeResult

FirstQUICHandshakeOrNil drains the network events buffered inside the QUICHandshake channel and returns the first QUICHandshake, if any. Otherwise, it returns nil.

func (*Trace) FirstTCPConnectOrNil 

func (tx *Trace) FirstTCPConnectOrNil() *model.ArchivalTCPConnectResult

FirstTCPConnectOrNil drains the network events buffered inside the TCPConnect channel and returns the first TCPConnect, if any. Otherwise, it returns nil.

func (*Trace) FirstTLSHandshakeOrNil 

func (tx *Trace) FirstTLSHandshakeOrNil() *model.ArchivalTLSOrQUICHandshakeResult

FirstTLSHandshakeOrNil drains the network events buffered inside the TLSHandshake channel and returns the first TLSHandshake, if any. Otherwise, it returns nil.

func (*Trace) Index 

func (tx *Trace) Index() int64

Index returns the trace index.

func (*Trace) MaybeWrapNetConn 

func (tx *Trace) MaybeWrapNetConn(conn net.Conn) net.Conn

MaybeWrapNetConn implements model.Trace.MaybeWrapNetConn.

func (*Trace) MaybeWrapUDPLikeConn 

func (tx *Trace) MaybeWrapUDPLikeConn(conn model.UDPLikeConn) model.UDPLikeConn

MaybeWrapUDPLikeConn implements model.Trace.MaybeWrapUDPLikeConn.

func (*Trace) NetworkEvents 

func (tx *Trace) NetworkEvents() (out []*model.ArchivalNetworkEvent)

NetworkEvents drains the network events buffered inside the NetworkEvent channel.

func (*Trace) NewDialerWithoutResolver 

func (tx *Trace) NewDialerWithoutResolver(dl model.DebugLogger, wrappers ...model.DialerWrapper) model.Dialer

NewDialerWithoutResolver is equivalent to netxlite.NewDialerWithoutResolver except that it returns a model.Dialer that uses this trace.

Caveat: the dialer wrappers are there to implement the model.MeasuringNetwork interface, but they're not used by this function.

func (*Trace) NewParallelDNSOverHTTPSResolver 

func (tx *Trace) NewParallelDNSOverHTTPSResolver(logger model.DebugLogger, URL string) model.Resolver

NewParallelDNSOverHTTPSResolver returns a trace-aware parallel DoH resolver

func (*Trace) NewParallelUDPResolver 

func (tx *Trace) NewParallelUDPResolver(logger model.DebugLogger, dialer model.Dialer, address string) model.Resolver

NewParallelUDPResolver returns a trace-ware parallel UDP resolver

func (*Trace) NewQUICDialerWithoutResolver 

func (tx *Trace) NewQUICDialerWithoutResolver(
	listener model.UDPListener, dl model.DebugLogger, wrappers ...model.QUICDialerWrapper) model.QUICDialer

NewQUICDialerWithoutResolver is equivalent to netxlite.NewQUICDialerWithoutResolver except that it returns a model.QUICDialer that uses this trace.

Caveat: the dialer wrappers are there to implement the model.MeasuringNetwork interface, but they're not used by this function.

func (*Trace) NewStdlibResolver 

func (tx *Trace) NewStdlibResolver(logger model.DebugLogger) model.Resolver

NewStdlibResolver returns a trace-ware system resolver

func (*Trace) NewTLSHandshakerStdlib 

func (tx *Trace) NewTLSHandshakerStdlib(dl model.DebugLogger) model.TLSHandshaker

NewTLSHandshakerStdlib is equivalent to netxlite.NewTLSHandshakerStdlib except that it returns a model.TLSHandshaker that uses this trace.

func (*Trace) NewTLSHandshakerUTLS 

func (tx *Trace) NewTLSHandshakerUTLS(dl model.DebugLogger, id *utls.ClientHelloID) model.TLSHandshaker

NewTLSHandshakerUTLS is equivalent to netxlite.NewTLSHandshakerUTLS except that it returns a model.TLSHandshaker that uses this trace.

func (*Trace) NewUDPListener added in v0.26.0 

func (tx *Trace) NewUDPListener() model.UDPListener

NewUDPListener implements model.Measuring Network.

func (*Trace) OnConnectDone 

func (tx *Trace) OnConnectDone(
	started time.Time, network, domain, remoteAddr string, err error, finished time.Time)

OnTCPConnectDone implements model.Trace.OnTCPConnectDone.

func (*Trace) OnDNSRoundTripForLookupHost 

func (tx *Trace) OnDNSRoundTripForLookupHost(started time.Time, reso model.Resolver, query model.DNSQuery,
	response model.DNSResponse, addrs []string, err error, finished time.Time)

OnDNSRoundTripForLookupHost implements model.Trace.OnDNSRoundTripForLookupHost

func (*Trace) OnDelayedDNSResponse 

func (tx *Trace) OnDelayedDNSResponse(started time.Time, txp model.DNSTransport, query model.DNSQuery,
	response model.DNSResponse, addrs []string, err error, finished time.Time) error

OnDelayedDNSResponse implements model.Trace.OnDelayedDNSResponse

func (*Trace) OnQUICHandshakeDone 

func (tx *Trace) OnQUICHandshakeDone(started time.Time, remoteAddr string, qconn quic.EarlyConnection,
	config *tls.Config, err error, finished time.Time)

OnQUICHandshakeDone implements model.Trace.OnQUICHandshakeDone

func (*Trace) OnQUICHandshakeStart 

func (tx *Trace) OnQUICHandshakeStart(now time.Time, remoteAddr string, config *quic.Config)

OnQUICHandshakeStart implements model.Trace.OnQUICHandshakeStart

func (*Trace) OnTLSHandshakeDone 

func (tx *Trace) OnTLSHandshakeDone(started time.Time, remoteAddr string, config *tls.Config,
	state tls.ConnectionState, err error, finished time.Time)

OnTLSHandshakeDone implements model.Trace.OnTLSHandshakeDone.

func (*Trace) OnTLSHandshakeStart 

func (tx *Trace) OnTLSHandshakeStart(now time.Time, remoteAddr string, config *tls.Config)

OnTLSHandshakeStart implements model.Trace.OnTLSHandshakeStart.

func (*Trace) QUICHandshakes 

func (tx *Trace) QUICHandshakes() (out []*model.ArchivalTLSOrQUICHandshakeResult)

QUICHandshakes drains the network events buffered inside the QUICHandshake channel.

func (*Trace) TCPConnects 

func (tx *Trace) TCPConnects() (out []*model.ArchivalTCPConnectResult)

TCPConnects drains the network events buffered inside the TCPConnect channel.

func (*Trace) TLSHandshakes 

func (tx *Trace) TLSHandshakes() (out []*model.ArchivalTLSOrQUICHandshakeResult)

TLSHandshakes drains the network events buffered inside the TLSHandshake channel.

func (*Trace) Tags added in v0.25.0 

func (tx *Trace) Tags() []string

Tags returns a copy of the tags configured for this trace.

func (*Trace) TimeNow 

func (tx *Trace) TimeNow() time.Time

TimeNow implements model.Trace.TimeNow.

func (*Trace) TimeSince 

func (tx *Trace) TimeSince(t0 time.Time) time.Duration

TimeSince is equivalent to Trace.TimeNow().Sub(t0).

func (*Trace) ZeroTime 

func (tx *Trace) ZeroTime() time.Time

ZeroTime returns trace's zero time.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.