netxlite

package
v3.22.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 22, 2024 License: GPL-3.0 Imports: 35 Imported by: 0

Documentation

Overview

Package netxlite contains network extensions.

This package is the basic networking building block that you should be using every time you need networking.

It implements interfaces defined in internal/model/netx.go.

You should consider checking the tutorial explaining how to use this package for network measurements: https://github.com/ooni/probe-cli/tree/master/internal/tutorial/netxlite.

Naming and history

Previous versions of this package were called netx. Compared to such versions this package is lightweight because it does not contain code to perform the measurements, hence its name.

Design

We want to potentially be able to observe each low-level operation separately, even though this is not done by this package. This is the use case where we are performing measurements.

We also want to be able to use this package in a more casual way without having to compose each operation separately. This, instead, is the use case where we're communicating with the OONI backend.

We want to optionally provide detailed logging of every operation, thus users can use `-v` to obtain OONI logs.

We also want to mock any underlying dependency for testing.

We also want to map errors to OONI failures, which are described by https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md.

We want to have reasonable watchdog timeouts for each operation.

We also want lightweight support for tracing network events. To this end, we use context.WithValue and context.Value to inject, and retrieve, a model.Trace implementation OPTIONALLY configured by the user.

See also the design document at docs/design/dd-003-step-by-step.md, which provides an overview of netxlite's main concerns.

To implement integration testing, we support hijacking the core network primitives used by this package, that is:

1. connecting a new TCP/UDP connection;

2. creating listening UDP sockets;

3. resolving domain names with getaddrinfo.

By overriding the TProxy variable, you can control these operations and route traffic to, e.g., a wireguard peer where you implement censorship.

Operations

This package implements the following operations:

1. establishing a TCP connection;

2. performing a domain name resolution with the "stdlib" resolver (i.e., getaddrinfo on Unix) or custom DNS transports (e.g., DoT, DoH);

3. performing the TLS handshake;

4. performing the QUIC handshake;

5. dialing with TCP, TLS, and QUIC (where in this context dialing means combining domain name resolution and "connecting");

6. performing HTTP, HTTP2, and HTTP3 round trips.

Operations 1, 2, 3, and 4 are used when we perform measurements, while 5 and 6 are mostly used when speaking with our backend.

Getaddrinfo usage

When compiled with CGO_ENABLED=1, this package will link with libc and call getaddrinfo directly. While this design choice means we will need to maintain more code, it also allows us to save the correct getaddrinfo return value, which is hidden by the Go resolver. Also, this strategy allows us to deal with the Android EAI_NODATA implementation quirk (see https://github.com/ooni/probe/issues/2029).

We currently use net.Resolver when CGO_ENABLED=0. A future version of netxlite MIGHT change this and use a custom UDP resolver in such a case, to avoid depending on the assumption that /etc/resolver.conf is present on the target system. See https://github.com/ooni/probe/issues/2118 for more details regarding ongoing plans to bypass net.Resolver when CGO_ENABLED=0. (If you're reading this piece of documentation and notice it's not updated, please submit a pull request to update it :-).

Index

Constants

View Source 
const (
	DNSNoSuchHostSuffix        = "no such host"
	DNSServerMisbehavingSuffix = "server misbehaving"
	DNSNoAnswerSuffix          = "no answer from DNS server"
)

We use these strings to string-match errors in the standard library and map such errors to OONI failures.

View Source 
const (
	FailureAddressFamilyNotSupported       = "address_family_not_supported"
	FailureAddressInUse                    = "address_in_use"
	FailureAddressNotAvailable             = "address_not_available"
	FailureAlreadyConnected                = "already_connected"
	FailureAndroidDNSCacheNoData           = "android_dns_cache_no_data"
	FailureBadAddress                      = "bad_address"
	FailureBadFileDescriptor               = "bad_file_descriptor"
	FailureConnectionAborted               = "connection_aborted"
	FailureConnectionAlreadyClosed         = "connection_already_closed"
	FailureConnectionAlreadyInProgress     = "connection_already_in_progress"
	FailureConnectionRefused               = "connection_refused"
	FailureConnectionReset                 = "connection_reset"
	FailureDNSBogonError                   = "dns_bogon_error"
	FailureDNSNXDOMAINError                = "dns_nxdomain_error"
	FailureDNSNoAnswer                     = "dns_no_answer"
	FailureDNSNonRecoverableFailure        = "dns_non_recoverable_failure"
	FailureDNSRefusedError                 = "dns_refused_error"
	FailureDNSReplyWithWrongQueryID        = "dns_reply_with_wrong_query_id"
	FailureDNSServerMisbehaving            = "dns_server_misbehaving"
	FailureDNSServfailError                = "dns_servfail_error"
	FailureDNSTemporaryFailure             = "dns_temporary_failure"
	FailureDestinationAddressRequired      = "destination_address_required"
	FailureEOFError                        = "eof_error"
	FailureGenericTimeoutError             = "generic_timeout_error"
	FailureHTTPInvalidRedirectLocationHost = "http_invalid_redirect_location_host"
	FailureHostUnreachable                 = "host_unreachable"
	FailureInterrupted                     = "interrupted"
	FailureInvalidArgument                 = "invalid_argument"
	FailureJSONParseError                  = "json_parse_error"
	FailureMessageSize                     = "message_size"
	FailureNetworkDown                     = "network_down"
	FailureNetworkReset                    = "network_reset"
	FailureNetworkUnreachable              = "network_unreachable"
	FailureNoBufferSpace                   = "no_buffer_space"
	FailureNoProtocolOption                = "no_protocol_option"
	FailureNotASocket                      = "not_a_socket"
	FailureNotConnected                    = "not_connected"
	FailureOperationWouldBlock             = "operation_would_block"
	FailurePermissionDenied                = "permission_denied"
	FailureProtocolNotSupported            = "protocol_not_supported"
	FailureQUICIncompatibleVersion         = "quic_incompatible_version"
	FailureSSLFailedHandshake              = "ssl_failed_handshake"
	FailureSSLInvalidCertificate           = "ssl_invalid_certificate"
	FailureSSLInvalidHostname              = "ssl_invalid_hostname"
	FailureSSLUnknownAuthority             = "ssl_unknown_authority"
	FailureTimedOut                        = "timed_out"
	FailureWrongProtocolType               = "wrong_protocol_type"
)

This enumeration lists the failures defined at https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md. Please, refer to that document for more information.

View Source 
const (
	ECONNREFUSED    = unix.ECONNREFUSED
	ECONNRESET      = unix.ECONNRESET
	EHOSTUNREACH    = unix.EHOSTUNREACH
	ETIMEDOUT       = unix.ETIMEDOUT
	EAFNOSUPPORT    = unix.EAFNOSUPPORT
	EADDRINUSE      = unix.EADDRINUSE
	EADDRNOTAVAIL   = unix.EADDRNOTAVAIL
	EISCONN         = unix.EISCONN
	EFAULT          = unix.EFAULT
	EBADF           = unix.EBADF
	ECONNABORTED    = unix.ECONNABORTED
	EALREADY        = unix.EALREADY
	EDESTADDRREQ    = unix.EDESTADDRREQ
	EINTR           = unix.EINTR
	EINVAL          = unix.EINVAL
	EMSGSIZE        = unix.EMSGSIZE
	ENETDOWN        = unix.ENETDOWN
	ENETRESET       = unix.ENETRESET
	ENETUNREACH     = unix.ENETUNREACH
	ENOBUFS         = unix.ENOBUFS
	ENOPROTOOPT     = unix.ENOPROTOOPT
	ENOTSOCK        = unix.ENOTSOCK
	ENOTCONN        = unix.ENOTCONN
	EWOULDBLOCK     = unix.EWOULDBLOCK
	EACCES          = unix.EACCES
	EPROTONOSUPPORT = unix.EPROTONOSUPPORT
	EPROTOTYPE      = unix.EPROTOTYPE
)

This enumeration provides a canonical name for every system-call error we support. Note: this list is system dependent. You're currently looking at the list of errors for linux.

View Source 
const (
	// ResolveOperation is the operation where we resolve a domain name.
	ResolveOperation = "resolve"

	// ConnectOperation is the operation where we do a TCP connect.
	ConnectOperation = "connect"

	// DNSRoundTripOperation is the DNS round trip.
	DNSRoundTripOperation = "dns_round_trip"

	// TLSHandshakeOperation is the TLS handshake.
	TLSHandshakeOperation = "tls_handshake"

	// QUICHandshakeOperation is the handshake to setup a QUIC connection.
	QUICHandshakeOperation = "quic_handshake"

	// QUICListenOperation is when we open a listening UDP conn for QUIC.
	QUICListenOperation = "quic_listen"

	// HTTPRoundTripOperation is the HTTP round trip.
	HTTPRoundTripOperation = "http_round_trip"

	// CloseOperation is when we close a socket.
	CloseOperation = "close"

	// ReadOperation is when we read from a socket.
	ReadOperation = "read"

	// WriteOperation is when we write to a socket.
	WriteOperation = "write"

	// ReadFromOperation is when we read from an UDP socket.
	ReadFromOperation = "read_from"

	// WriteToOperation is when we write to an UDP socket.
	WriteToOperation = "write_to"

	// UnknownOperation is when we cannot determine the operation.
	UnknownOperation = "unknown"

	// TopLevelOperation is used when the failure happens at top level. This
	// happens for example with urlgetter with a cancelled context.
	TopLevelOperation = "top_level"
)

Operations that we measure. They are the possible values of the ErrWrapper.Operation field.

View Source 
const FailureUnknown = "unknown_failure"

FailureUnknown is the prefix used for unknown failures

View Source 
const StdlibResolverGetaddrinfo = "getaddrinfo"

Name of the resolver we use when we link with libc and use getaddrinfo directly.

See https://github.com/ooni/spec/pull/257 for more info.

View Source 
const StdlibResolverGolangNetResolver = "golang_net_resolver"

Name of the resolver we use when we don't link with libc and use net.Resolver.

See https://github.com/ooni/spec/pull/257 for more info.

View Source 
const StdlibResolverSystem = "system"

Legacy name of the resolver we use when we're don't know whether we're using getaddrinfo, but we're using net.Resolver, and we're splitting the answer in two A and AAAA queries. Eventually will become deprecated.

See https://github.com/ooni/spec/pull/257 for more info.

Variables

View Source 
var (
	// ErrOODNSNoSuchHost means NXDOMAIN.
	ErrOODNSNoSuchHost = fmt.Errorf("ooniresolver: %s", DNSNoSuchHostSuffix)

	// ErrOODNSMisbehaving is the error typically returned by the `netgo`resolver
	// when it cannot really make sense of the error.
	ErrOODNSMisbehaving = fmt.Errorf("ooniresolver: %s", DNSServerMisbehavingSuffix)

	// ErrOODNSNoAnswer means that we've got a valid DNS response that
	// did not contain any answer for the original query. This could happen
	// when we query for AAAA and the domain only has A records.
	ErrOODNSNoAnswer = fmt.Errorf("ooniresolver: %s", DNSNoAnswerSuffix)
)

These errors are returned by custom DNSTransport instances (e.g., DNSOverHTTPSTransport and DNSOverUDPTransport). Their suffix matches the equivalent unexported errors used by the Go standard library.

View Source 
var (
	// ErrOODNSRefused indicates that the response's Rcode was "refused"
	ErrOODNSRefused = errors.New("ooniresolver: refused")

	// ErrOODNSServfail indicates that the response's Rcode was "servfail"
	ErrOODNSServfail = errors.New("ooniresolver: servfail")
)

These errors are not part of the Go standard library but we can return them in our custom resolvers.

View Source 
var (
	// ErrDNSReplyWithWrongQueryID indicates we have got a DNS reply with the wrong queryID.
	ErrDNSReplyWithWrongQueryID = errors.New(FailureDNSReplyWithWrongQueryID)

	// ErrDNSIsQuery indicates that we were passed a DNS query.
	ErrDNSIsQuery = errors.New("ooresolver: expected response but received query")
)
View Source 
var ErrAndroidDNSCacheNoData = errors.New(FailureAndroidDNSCacheNoData)

ErrAndroidDNSCacheNoData is the kind of error returned by our getaddrinfo code on Android when we see EAI_NODATA, an error condition that could mean anything as explained in getaddrinfo_linux.go.

View Source 
var ErrDNSBogon = errors.New("dns: detected bogon address")

ErrDNSBogon indicates that we found a bogon address. Code that filters for DNS bogons MUST use this error.

View Source 
var ErrDNSIPAddress = errors.New("ooresolver: expected domain, found IP address")

ErrDNSIPAddress indicates that you passed an IP address to a DNS function that only works with domain names.

View Source 
var ErrInvalidIP = errors.New("netxlite: invalid IP")

ErrInvalidIP indicates that a string is not a valid IP.

View Source 
var ErrInvalidTLSVersion = errors.New("invalid TLS version")

ErrInvalidTLSVersion indicates that you passed us a string that does not represent a valid TLS version.

View Source 
var ErrNoConnReuse = errors.New("cannot reuse connection")

ErrNoConnReuse is the type of error returned when you create a "single use" dialer or a "single use" TLS dialer and you dial more than once, which is not supported by such a dialer.

View Source 
var ErrNoDNSTransport = errors.New("operation requires a DNS transport")

ErrNoDNSTransport is the error returned when you attempt to perform a DNS operation that requires a custom DNSTransport (e.g., DNSOverHTTPSTransport) but you are using the "stdlib" resolver instead.

View Source 
var ErrNoDialer = errors.New("no configured dialer")

ErrNoDialer is the type of error returned by "null" dialers when you attempt to dial with them.

View Source 
var ErrNoResolver = errors.New("no configured resolver")

ErrNoResolver is the type of error returned by "without resolver" dialer when asked to dial for and endpoint containing a domain name, since they can only dial for endpoints containing IP addresses.

View Source 
var ErrNoTLSDialer = errors.New("no configured TLS dialer")

ErrNoTLSDialer is the type of error returned by "null" TLS dialers when you attempt to dial with them.

View Source 
var ErrNotTLSConn = errors.New("not a TLSConn")

ErrNotTLSConn occur when an interface accepts a net.Conn but internally needs a TLSConn and you pass a net.Conn that doesn't implement TLSConn to such an interface.

View Source 
var ErrProxyUnsupportedScheme = errors.New("proxy: unsupported scheme")

ErrProxyUnsupportedScheme indicates we don't support the proxy scheme.

View Source 
var ErrUTLSHandshakePanic = errors.New("utls: handshake panic")

ErrUTLSHandshakePanic indicates that there was panic handshaking when we were using the yawning/utls library for parroting. See https://github.com/ooni/probe/issues/1770 for more information.

View Source 
var ErrUnknown = errors.New(FailureUnknown)

ErrUnknown is an error you may want to return when you don't known what exactly happened. This error is automatically mapped to FailureUnknown by ClassifyGenericError.

Functions

func ClassifyGenericError 

func ClassifyGenericError(err error) string

ClassifyGenericError maps an error occurred during an operation to an OONI failure string. This specific classifier is the most generic one. You usually use it when mapping I/O errors. You should check whether there is a specific classifier for more specific operations (e.g., DNS resolution, TLS handshake).

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

We put inside this classifier:

- system call errors;

- generic errors that can occur in multiple places;

- all the errors that depend on strings.

The more specific classifiers will call this classifier if they fail to find a mapping for the input error.

If everything else fails, this classifier returns a string like "unknown_failure: XXX" where XXX has been scrubbed so to remove any network endpoints from the original error string.

func ClassifyQUICHandshakeError 

func ClassifyQUICHandshakeError(err error) string

ClassifyQUICHandshakeError maps errors during a QUIC handshake to OONI failure strings.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ClassifyResolverError 

func ClassifyResolverError(err error) string

ClassifyResolverError maps DNS resolution errors to OONI failure strings.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ClassifyTLSHandshakeError 

func ClassifyTLSHandshakeError(err error) string

ClassifyTLSHandshakeError maps an error occurred during the TLS handshake to an OONI failure string.

If the input error is an *ErrWrapper we don't perform the classification again and we return its Failure.

If this classifier fails, it calls ClassifyGenericError and returns to the caller its return value.

func ClonedTLSConfigOrNewEmptyConfig added in v3.16.0 

func ClonedTLSConfigOrNewEmptyConfig(config *tls.Config) *tls.Config

ClonedTLSConfigOrNewEmptyConfig returns a clone of the provided config, if not nil, or a fresh and completely empty *tls.Config.

func ConfigureTLSVersion 

func ConfigureTLSVersion(config *tls.Config, version string) error

ConfigureTLSVersion configures the correct TLS version into a *tls.Config or returns ErrInvalidTLSVersion.

Recognized strings: TLSv1.3, TLSv1.2, TLSv1.1, TLSv1.0.

func ContextTraceOrDefault added in v3.16.0 

func ContextTraceOrDefault(ctx context.Context) model.Trace

ContextTraceOrDefault retrieves the trace bound to the context or returns a default implementation of the trace in case no tracing was configured.

func ContextWithTrace added in v3.16.0 

func ContextWithTrace(ctx context.Context, trace model.Trace) context.Context

ContextWithTrace returns a new context that binds to the given trace. If the given trace is nil, this function will call panic.

func CopyContext 

func CopyContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, error)

CopyContext is like io.Copy but may terminate earlier when the context expires. This function has the same caveats of ReadAllContext regarding the temporary leaking of the background I/O goroutine.

As of Go 1.17.6, CopyContext additionally deals with wrapped io.EOF correctly, while io.Copy does not. See https://github.com/ooni/probe/issues/1965.

func ErrorToGetaddrinfoRetvalOrZero added in v3.16.0 

func ErrorToGetaddrinfoRetvalOrZero(err error) int64

ErrorToGetaddrinfoRetvalOrZero converts an arbitrary error to the return value of getaddrinfo. If err is nil or is not an instance of ErrGetaddrinfo, we just return zero.

func IsBogon added in v3.14.0 

func IsBogon(address string) bool

IsBogon returns whether an IP address is bogon. Passing to this function a non-IP address causes it to return true.

func IsIPv6 added in v3.15.0 

func IsIPv6(candidate string) (bool, error)

IsIPv6 returns true if the given candidate is a valid IP address representation and such representation is IPv6.

func IsLoopback added in v3.15.0 

func IsLoopback(address string) bool

IsLoopback returns whether an IP address is loopback. Passing to this function a non-IP address causes it to return true.

func MaybeNewErrWrapper added in v3.16.0 

func MaybeNewErrWrapper(c classifier, op string, err error) error

MaybeNewErrWrapper is like NewErrWrapper except that this function won't panic if passed a nil error.

func MaybeTLSConnectionState added in v3.19.0 

func MaybeTLSConnectionState(conn model.TLSConn) (state tls.ConnectionState)

MaybeTLSConnectionState is a convenience function that returns an empty tls.ConnectionState when the model.TLSConn is nil.

func MaybeWrapWithBogonResolver added in v3.16.0 

func MaybeWrapWithBogonResolver(enabled bool, reso model.Resolver) model.Resolver

MaybeWrapWithBogonResolver wraps the given resolver with a BogonResolver iff the provided boolean flag is true. Otherwise, this factory just returns the provided resolver to the caller without any wrapping.

The returned resolver returns a wrapped ErrDNSBogon if there's a bogon error.

BUG: This resolver currently only implements LookupHost. All the other lookup methods will always return ErrNoDNSTransport.

func MaybeWrapWithCachingResolver added in v3.16.0 

func MaybeWrapWithCachingResolver(enabled bool, reso model.Resolver) model.Resolver

MaybeWrapWithCachingResolver wraps the provided resolver with a resolver that remembers the result of previous successful resolutions, if the enabled argument is true. Otherwise, we return the unmodified provided resolver.

Bug: the returned resolver only applies caching to LookupHost and any other lookup operation returns ErrNoDNSTransport to the caller.

func MaybeWrapWithProxyDialer deprecated added in v3.16.0 

func MaybeWrapWithProxyDialer(dialer model.Dialer, proxyURL *url.URL) model.Dialer

MaybeWrapWithProxyDialer returns the original dialer if the proxyURL is nil and otherwise returns a wrapped dialer that implements proxying.

Deprecated: do not use this function in new code.

func MaybeWrapWithStaticDNSCache added in v3.16.0 

func MaybeWrapWithStaticDNSCache(cache map[string][]string, reso model.Resolver) model.Resolver

MaybeWrapWithStaticDNSCache wraps the provided resolver with a resolver that checks the given cache before issuing queries to the underlying DNS resolver.

Bug: the returned resolver only applies caching to LookupHost and any other lookup operation returns ErrNoDNSTransport to the caller.

func NewDNSOverGetaddrinfoTransport added in v3.16.0 

func NewDNSOverGetaddrinfoTransport() model.DNSTransport

NewDNSOverGetaddrinfoTransport creates a new dns-over-getaddrinfo transport.

func NewDNSOverHTTPSTransport added in v3.15.0 

func NewDNSOverHTTPSTransport(client model.HTTPClient, URL string) model.DNSTransport

NewDNSOverHTTPSTransport is like NewUnwrappedDNSOverHTTPSTransport but returns an already wrapped DNSTransport.

func NewDNSOverHTTPSTransportWithHTTPTransport added in v3.16.0 

func NewDNSOverHTTPSTransportWithHTTPTransport(txp model.HTTPTransport, URL string) model.DNSTransport

NewDNSOverHTTPSTransportWithHTTPTransport is like NewDNSOverHTTPSTransport but takes in input an HTTPTransport rather than an HTTPClient.

func NewDialerWithStdlibResolver added in v3.16.0 

func NewDialerWithStdlibResolver(dl model.DebugLogger) model.Dialer

NewDialerWithStdlibResolver is equivalent to creating a system resolver using NewStdlibResolver and then a dialer using Netx.NewDialerWithResolver with the default Netx where the resolver argument is the previously created resolver.

func NewHTTP3ClientWithResolver added in v3.17.0 

func NewHTTP3ClientWithResolver(netx *Netx, logger model.Logger, reso model.Resolver) model.HTTPClient

NewHTTP3ClientWithResolver creates a new HTTP3Transport using the given resolver and then from that builds an HTTPClient.

func NewHTTP3Transport 

func NewHTTP3Transport(
	logger model.DebugLogger, dialer model.QUICDialer, tlsConfig *tls.Config) model.HTTPTransport

NewHTTP3Transport creates a new HTTPTransport using http3. The dialer argument MUST NOT be nil. If the tlsConfig argument is nil, then the code will use the default TLS configuration.

func NewHTTP3TransportWithResolver added in v3.16.0 

func NewHTTP3TransportWithResolver(netx *Netx, logger model.DebugLogger, reso model.Resolver) model.HTTPTransport

NewHTTPTransportWithResolver creates a new HTTPTransport using http3 that uses the given logger and the given resolver.

func NewHTTPClient added in v3.16.0 

func NewHTTPClient(txp model.HTTPTransport) model.HTTPClient

NewHTTPClient creates a new, wrapped HTTPClient using the given transport.

This function behavior is QUIRKY because it does not configure a cookie jar, which is probably not the right thing to do in many cases, but legacy code MAY depend on this behavior. TODO(https://github.com/ooni/probe/issues/2534).

func NewHTTPClientStdlib added in v3.14.0 

func NewHTTPClientStdlib(logger model.DebugLogger) model.HTTPClient

NewHTTPClientStdlib creates a new HTTPClient that uses the standard library for TLS and DNS resolutions.

This function behavior is QUIRKY as documented in NewHTTPTransport.

func NewHTTPClientWithResolver added in v3.16.0 

func NewHTTPClientWithResolver(netx *Netx, logger model.Logger, reso model.Resolver) model.HTTPClient

NewHTTPClientWithResolver creates a new HTTPTransport using the given resolver and then from that builds an HTTPClient.

This function behavior is QUIRKY as documented in NewHTTPTransport.

func NewHTTPTransport 

func NewHTTPTransport(logger model.DebugLogger, dialer model.Dialer, tlsDialer model.TLSDialer) model.HTTPTransport

NewHTTPTransport returns a wrapped HTTP transport for HTTP2 and HTTP/1.1 using the given dialer and logger.

The returned transport will gracefully handle TLS connections created using gitlab.com/yawning/utls.git, if the TLS dialer is a dialer using such library for TLS operations.

The returned transport will not have a configured proxy, not even the proxy configurable from the environment.

QUIRK: the returned transport will disable transparent decompression of compressed response bodies (and will not automatically ask for such compression, though you can always do that manually).

The returned transport will configure TCP and TLS connections created using its dialer and TLS dialer to always have a read watchdog timeout to address https://github.com/ooni/probe/issues/1609.

QUIRK: the returned transport will always enforce 1 connection per host and we cannot get rid of this QUIRK requirement because it is necessary to perform sane measurements with tracing. We will be able to possibly relax this requirement after we change the way in which we perform measurements.

func NewHTTPTransportWithOptions added in v3.19.0 

func NewHTTPTransportWithOptions(logger model.Logger,
	dialer model.Dialer, tlsDialer model.TLSDialer, options ...HTTPTransportOption) model.HTTPTransport

NewHTTPTransport is the high-level factory to create a model.HTTPTransport using github.com/ooni/oohttp as the HTTP library with HTTP/1.1 and HTTP2 support.

This transport is suitable for HTTP2 and HTTP/1.1 using any TLS library, including, e.g., github.com/ooni/oocrypto.

This factory clones the default github.com/ooni/oohttp transport and configures the provided dialer and TLS dialer by setting the .DialContext and .DialTLSContext fields of the transport. We also wrap the provided dialers to address https://github.com/ooni/probe/issues/1609.

Apart from that, the only non-default options set by this factory are these:

1. the .Proxy field is set to nil, so by default we DO NOT honour the HTTP_PROXY and HTTPS_PROXY environment variables, which is required if we want to use this code for measuring;

2. the .ForceAttemptHTTP2 field is set to true;

3. the .DisableCompression field is set to true, again required if we want to use this code for measuring, and we should make sure the defaults we're using are suitable for measuring, since the impact of making a mistake in measuring code is a data quality issue 😅.

The returned transport supports logging and error wrapping because internally this function calls WrapHTTPTransport before we return.

This factory is the RECOMMENDED way of creating a model.HTTPTransport.

func NewHTTPTransportWithResolver added in v3.16.0 

func NewHTTPTransportWithResolver(netx *Netx, logger model.DebugLogger, reso model.Resolver) model.HTTPTransport

NewHTTPTransportWithResolver creates a new HTTP transport using the stdlib for everything but the given resolver.

This function behavior is QUIRKY as documented in NewHTTPTransport.

func NewMaybeShapingDialer added in v3.16.0 

func NewMaybeShapingDialer(dialer model.Dialer) model.Dialer

NewMaybeShapingDialer takes in input a model.Dialer and returns in output another model.Dialer that MAY dial connections with I/O shaping, depending on whether the user builds with or without the `-tags shaping` CLI flag.

We typically use `-tags shaping` when running integration tests for dash and ndt7 to avoiod hammering m-lab servers from the very-fast GitHub CI servers.

See https://github.com/ooni/probe/issues/2112 for extra context.

func NewMozillaCertPool added in v3.18.0 

func NewMozillaCertPool() *x509.CertPool

NewMozillaCertPool returns the default x509 certificate pool that we bundle from Mozilla. It's safe to modify the returned value: every invocation returns a distinct *x509.CertPool instance. You SHOULD NOT call this function every time your experiment is processing input. If you are happy with the default cert pool, just leave the RootCAs field nil. Otherwise, you should cache the cert pool you use.

func NewNullDialer 

func NewNullDialer() model.Dialer

NewNullDialer returns a dialer that always fails with ErrNoDialer.

func NewNullTLSDialer 

func NewNullTLSDialer() model.TLSDialer

NewNullTLSDialer returns a TLS dialer that always fails with ErrNoTLSDialer.

func NewSerialUDPResolver deprecated added in v3.16.0 

func NewSerialUDPResolver(logger model.DebugLogger, dialer model.Dialer, address string) model.Resolver

NewSerialUDPResolver creates a new Resolver using DNS-over-UDP that performs serial A/AAAA lookups during LookupHost.

Deprecated: use NewParallelUDPResolver.

Arguments:

- logger is the logger to use

- dialer is the dialer to create and connect UDP conns

- address is the server address (e.g., 1.1.1.1:53)

func NewSingleUseDialer 

func NewSingleUseDialer(conn net.Conn) model.Dialer

NewSingleUseDialer returns a "single use" dialer. The first dial will succed and return conn regardless of the network and address arguments passed to DialContext. Any subsequent dial returns ErrNoConnReuse.

func NewSingleUseQUICDialer 

func NewSingleUseQUICDialer(qconn quic.EarlyConnection) model.QUICDialer

NewSingleUseQUICDialer is like NewSingleUseDialer but for QUIC.

func NewSingleUseTLSDialer 

func NewSingleUseTLSDialer(conn TLSConn) model.TLSDialer

NewSingleUseTLSDialer is like NewSingleUseDialer but takes in input a TLSConn rather than a net.Conn.

func NewTLSDialer 

func NewTLSDialer(dialer model.Dialer, handshaker model.TLSHandshaker) model.TLSDialer

NewTLSDialer creates a new TLS dialer using the given dialer and handshaker.

func NewTLSDialerWithConfig 

func NewTLSDialerWithConfig(d model.Dialer, h model.TLSHandshaker, c *tls.Config) model.TLSDialer

NewTLSDialerWithConfig is like NewTLSDialer with an optional config.

func NewUnwrappedStdlibResolver deprecated added in v3.16.0 

func NewUnwrappedStdlibResolver() model.Resolver

NewUnwrappedStdlibResolver returns a new, unwrapped resolver using the standard library (i.e., getaddrinfo if possible and &net.Resolver{} otherwise). As the name implies, this function returns an unwrapped resolver.

Deprecated: do not use this function inside new networking code.

func ParseUDPAddr added in v3.15.0 

func ParseUDPAddr(address string) (*net.UDPAddr, error)

ParseUDPAddr maps the string representation of an UDP endpoint to the corresponding *net.UDPAddr representation.

func ReadAllContext 

func ReadAllContext(ctx context.Context, r io.Reader) ([]byte, error)

ReadAllContext is like io.ReadAll but reads r in a background goroutine. This function will return earlier if the context is cancelled. In which case we will continue reading from the reader in the background goroutine, and we will discard the result. To stop the long-running goroutine, close the connection bound to the reader. Until such a connection is closed, you're leaking the backround goroutine and doing I/O.

As of Go 1.17.6, ReadAllContext additionally deals with wrapped io.EOF correctly, while io.ReadAll does not. See https://github.com/ooni/probe/issues/1965.

func StreamAllContext added in v3.21.0 

func StreamAllContext(ctx context.Context, reader io.Reader) ([]byte, error)

StreamAllContext streams from the given reader [r] until interrupted by [ctx] or when [r] hits the EOF.

This function runs a background goroutine that should exit as soon as [ctx] is done or when [reader] is closed, if applicable.

This function transforms an errors.Is(err, io.EOF) to a nil error such as the standard library's ReadAll does.

The caller of this function MUST check for the context being expired when there's an error and decide what it is proper to do in such a case. Typically, when streaming the HTTP response body for Web Connectivity v0.5, the right thing to do is to report the body as being truncated.

This function might return a non-zero-length buffer along with an non-nil error in the case in which we could only read a portion of the body and then we were interrupted by the error.

func TLSCipherSuiteString 

func TLSCipherSuiteString(value uint16) string

TLSCipherSuiteString returns the TLS cipher suite as a string. If value is zero, we return the empty string. If we don't know the mapping from the value to a cipher suite name, we return `TLS_CIPHER_SUITE_UNKNOWN_ddd` where `ddd` is the numeric value passed to this function.

func TLSVersionString 

func TLSVersionString(value uint16) string

TLSVersionString returns a TLS version string. If value is zero, we return the empty string. If the value is unknown, we return `TLS_VERSION_UNKNOWN_ddd` where `ddd` is the numeric value passed to this function.

func WithCustomTProxy added in v3.18.0 

func WithCustomTProxy(tproxy model.UnderlyingNetwork, function func())

WithCustomTProxy runs the given function with a different UnderlyingNetwork and restores the previous UnderlyingNetwork before returning.

func WrapDialer 

func WrapDialer(logger model.DebugLogger, resolver model.Resolver,
	baseDialer model.Dialer, wrappers ...model.DialerWrapper) (outDialer model.Dialer)

WrapDialer wraps an existing Dialer to add extra functionality such as separting DNS lookup and connecting, error wrapping, logging, etc.

When possible use Netx.NewDialerWithResolver or NewDialerWithoutResolver instead of using this rather low-level function.

Arguments

1. logger is used to emit debug messages (MUST NOT be nil);

2. resolver is the resolver to use when dialing for endpoint addresses containing domain names (MUST NOT be nil);

3. baseDialer is the dialer to wrap (MUST NOT be nil);

4. wrappers is a list of zero or more functions allowing you to modify the behavior of the returned dialer (see below). Please note that this function will just ignore any nil wrapper.

Return value

The returned dialer is an opaque type consisting of the composition of several simple dialers. The following pseudo code illustrates the general behavior of the returned composed dialer: 

addrs, err := dnslookup()
if err != nil {
  return nil, err
}
errors := []error{}
for _, a := range addrs {
  conn, err := tcpconnect(a)
  if err != nil {
    errors = append(errors, err)
    continue
  }
  return conn, nil
}
return nil, errors[0]

The following table describes the structure of the returned dialer: 

+-------+-----------------+------------------------------------------+
| Index | Name            | Description                              |
+-------+-----------------+------------------------------------------+
| 0     | base            | the baseDialer argument                  |
+-------+-----------------+------------------------------------------+
| 1     | errWrapper      | wraps Go errors to be consistent with    |
|       |                 | OONI df-007-errors spec                  |
+-------+-----------------+------------------------------------------+
| 2     | ???             | if there are wrappers, result of calling |
|       |                 | the first one on the errWrapper dialer   |
+-------+-----------------+------------------------------------------+
| ...   | ...             | ...                                      |
+-------+-----------------+------------------------------------------+
| N     | ???             | if there are wrappers, result of calling |
|       |                 | the last one on the N-1 dialer           |
+-------+-----------------+------------------------------------------+
| N+1   | logger (inner)  | logs TCP connect operations              |
+-------+-----------------+------------------------------------------+
| N+2   | resolver        | DNS lookup and try connect each IP in    |
|       |                 | sequence until one of them succeeds      |
+-------+-----------------+------------------------------------------+
| N+3   | logger (outer)  | logs the overall dial operation          |
+-------+-----------------+------------------------------------------+

The list of wrappers allows to insert modified dialers in the correct place for observing and saving I/O events (connect, read, etc.).

Remarks

When the resolver is &NullResolver{} any attempt to perform DNS resolutions in the dialer at index N+2 will fail with ErrNoResolver.

Otherwise, the dialer at index N+2 will try each resolved IP address sequentially. In case of failure, such a resolver will return the first error that occurred. This implementation strategy is a QUIRK that is documented at TODO(https://github.com/ooni/probe/issues/1779).

If the baseDialer is &DialerSystem{}, there will be a fixed TCP connect timeout for each connect operation. Because there may be multiple IP addresses per dial, the overall timeout would be a multiple of the timeout of a single connect operation. You may want to use the context to reduce the overall time spent trying all addresses and timing out.

func WrapHTTPClient 

func WrapHTTPClient(clnt model.HTTPClient) model.HTTPClient

WrapHTTPClient wraps an HTTP client to add error wrapping capabilities.

This is a low level factory. Consider not using it directly.

func WrapHTTPTransport 

func WrapHTTPTransport(logger model.DebugLogger, txp model.HTTPTransport) model.HTTPTransport

WrapHTTPTransport creates an HTTPTransport using the given logger and guarantees that returned errors are wrapped.

This is a low level factory. Consider not using it directly.

func WrapResolver 

func WrapResolver(logger model.DebugLogger, resolver model.Resolver) model.Resolver

WrapResolver creates a new resolver that wraps an existing resolver to add these properties:

1. handles IDNA;

2. performs logging;

3. short-circuits IP addresses like getaddrinfo does (i.e., resolving "1.1.1.1" yields []string{"1.1.1.1"};

4. wraps errors;

5. enforces reasonable timeouts ( see https://github.com/ooni/probe/issues/1726).

This is a low-level factory. Use only if out of alternatives.

Types

type DNSDecoderMiekg 

type DNSDecoderMiekg struct{}

DNSDecoderMiekg uses github.com/miekg/dns to implement the Decoder.

func (*DNSDecoderMiekg) DecodeResponse added in v3.16.0 

func (d *DNSDecoderMiekg) DecodeResponse(data []byte, query model.DNSQuery) (model.DNSResponse, error)

DecodeResponse implements model.DNSDecoder.DecodeResponse.

type DNSEncoderMiekg 

type DNSEncoderMiekg struct{}

DNSEncoderMiekg uses github.com/miekg/dns to implement the Encoder.

func (*DNSEncoderMiekg) Encode 

func (e *DNSEncoderMiekg) Encode(domain string, qtype uint16, padding bool) model.DNSQuery

Encoder implements model.DNSEncoder.Encode.

type DNSOverHTTPSTransport added in v3.15.0 

type DNSOverHTTPSTransport struct {
	// Client is the MANDATORY http client to use.
	Client model.HTTPClient

	// Decoder is the MANDATORY DNSDecoder.
	Decoder model.DNSDecoder

	// URL is the MANDATORY URL of the DNS-over-HTTPS server.
	URL string

	// HostOverride is OPTIONAL and allows to override the
	// Host header sent in every request.
	HostOverride string
}

DNSOverHTTPSTransport is a DNS-over-HTTPS DNSTransport.

func NewUnwrappedDNSOverHTTPSTransport added in v3.16.0 

func NewUnwrappedDNSOverHTTPSTransport(client model.HTTPClient, URL string) *DNSOverHTTPSTransport

NewUnwrappedDNSOverHTTPSTransport creates a new DNSOverHTTPSTransport instance that has not been wrapped yet.

Arguments:

- client is a model.HTTPClient type;

- URL is the DoH resolver URL (e.g., https://dns.google/dns-query).

func NewUnwrappedDNSOverHTTPSTransportWithHostOverride added in v3.16.0 

func NewUnwrappedDNSOverHTTPSTransportWithHostOverride(
	client model.HTTPClient, URL, hostOverride string) *DNSOverHTTPSTransport

NewUnwrappedDNSOverHTTPSTransportWithHostOverride creates a new DNSOverHTTPSTransport with the given Host header override. This instance has not been wrapped yet.

func (*DNSOverHTTPSTransport) Address added in v3.15.0 

func (t *DNSOverHTTPSTransport) Address() string

Address returns the URL we're using for the DoH server.

func (*DNSOverHTTPSTransport) CloseIdleConnections added in v3.15.0 

func (t *DNSOverHTTPSTransport) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverHTTPSTransport) Network added in v3.15.0 

func (t *DNSOverHTTPSTransport) Network() string

Network returns the transport network, i.e., "doh".

func (*DNSOverHTTPSTransport) RequiresPadding added in v3.15.0 

func (t *DNSOverHTTPSTransport) RequiresPadding() bool

RequiresPadding returns true for DoH according to RFC8467.

func (*DNSOverHTTPSTransport) RoundTrip added in v3.15.0 

func (t *DNSOverHTTPSTransport) RoundTrip(
	ctx context.Context, query model.DNSQuery) (model.DNSResponse, error)

RoundTrip sends a query and receives a reply.

type DNSOverTCPTransport added in v3.15.0 

type DNSOverTCPTransport struct {
	// contains filtered or unexported fields
}

DNSOverTCPTransport is a DNS-over-{TCP,TLS} DNSTransport.

Note: this implementation always creates a new connection for each query. This strategy is less efficient but MAY be more robust for cleartext TCP connections when querying for a blocked domain name causes endpoint blocking.

func NewUnwrappedDNSOverTCPTransport added in v3.16.0 

func NewUnwrappedDNSOverTCPTransport(dial DialContextFunc, address string) *DNSOverTCPTransport

NewUnwrappedDNSOverTCPTransport creates a new DNSOverTCPTransport that has not been wrapped yet.

Arguments:

- dial is a function with the net.Dialer.DialContext's signature;

- address is the endpoint address (e.g., 8.8.8.8:53).

func NewUnwrappedDNSOverTLSTransport added in v3.16.0 

func NewUnwrappedDNSOverTLSTransport(dial DialContextFunc, address string) *DNSOverTCPTransport

NewUnwrappedDNSOverTLSTransport creates a new DNSOverTLS transport that has not been wrapped yet.

Arguments:

- dial is a function with the net.Dialer.DialContext's signature;

- address is the endpoint address (e.g., 8.8.8.8:853).

func (*DNSOverTCPTransport) Address added in v3.15.0 

func (t *DNSOverTCPTransport) Address() string

Address returns the upstream server endpoint (e.g., "1.1.1.1:853").

func (*DNSOverTCPTransport) CloseIdleConnections added in v3.15.0 

func (t *DNSOverTCPTransport) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverTCPTransport) Network added in v3.15.0 

func (t *DNSOverTCPTransport) Network() string

Network returns the transport network, i.e., "dot" or "tcp".

func (*DNSOverTCPTransport) RequiresPadding added in v3.15.0 

func (t *DNSOverTCPTransport) RequiresPadding() bool

RequiresPadding returns true for DoT and false for TCP according to RFC8467.

func (*DNSOverTCPTransport) RoundTrip added in v3.15.0 

func (t *DNSOverTCPTransport) RoundTrip(
	ctx context.Context, query model.DNSQuery) (model.DNSResponse, error)

RoundTrip sends a query and receives a reply.

type DNSOverUDPTransport added in v3.15.0 

type DNSOverUDPTransport struct {
	// Decoder is the MANDATORY DNSDecoder to use.
	Decoder model.DNSDecoder

	// Dialer is the MANDATORY dialer used to create the conn.
	Dialer model.Dialer

	// Endpoint is the MANDATORY server's endpoint (e.g., 1.1.1.1:53)
	Endpoint string
	// contains filtered or unexported fields
}

DNSOverUDPTransport is a DNS-over-UDP DNSTransport.

To construct this type, either manually fill the fields marked as MANDATORY or just use the NewDNSOverUDPTransport factory directly.

RoundTrip creates a new connected UDP socket for each outgoing query. Using a new socket is good because some censored environments will block the client UDP endpoint for several seconds when you query for blocked domains. We could also have used an unconnected UDP socket here, but:

1. connected UDP sockets are great because they get some ICMP errors to be translated into socket errors (among them, host_unreachable);

2. connected UDP sockets ignore responses from illegitimate IP addresses but most if not all DNS resolvers also do that, therefore this does not seem to be a realistic censorship vector. At the same time, connected sockets provide us for free with the feature that we don't need to bother with checking whether the reply comes from the expected server.

Being able to observe some ICMP errors is good because it could possibly make this code suitable to implement parasitic traceroute.

This transport by default listens for additional responses after the first one and makes them available using the context-configured trace.

func NewUnwrappedDNSOverUDPTransport added in v3.16.0 

func NewUnwrappedDNSOverUDPTransport(dialer model.Dialer, address string) *DNSOverUDPTransport

NewUnwrappedDNSOverUDPTransport creates a DNSOverUDPTransport instance that has not been wrapped yet.

Arguments:

- dialer is any type that implements the Dialer interface;

- address is the endpoint address (e.g., 8.8.8.8:53).

If the address contains a domain name rather than an IP address (e.g., dns.google:53), we will end up using the first of the IP addresses returned by the underlying DNS lookup performed using the dialer. This usage pattern is NOT RECOMMENDED because we'll have less control over which IP address is being used.

func (*DNSOverUDPTransport) Address added in v3.15.0 

func (t *DNSOverUDPTransport) Address() string

Address returns the upstream server address.

func (*DNSOverUDPTransport) CloseIdleConnections added in v3.15.0 

func (t *DNSOverUDPTransport) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*DNSOverUDPTransport) Network added in v3.15.0 

func (t *DNSOverUDPTransport) Network() string

Network returns the transport network, i.e., "udp".

func (*DNSOverUDPTransport) RequiresPadding added in v3.15.0 

func (t *DNSOverUDPTransport) RequiresPadding() bool

RequiresPadding returns false for UDP according to RFC8467.

func (*DNSOverUDPTransport) RoundTrip added in v3.15.0 

func (t *DNSOverUDPTransport) RoundTrip(
	ctx context.Context, query model.DNSQuery) (model.DNSResponse, error)

RoundTrip sends a query and receives a response.

type DefaultTProxy added in v3.17.0 

type DefaultTProxy struct{}

DefaultTProxy is the default UnderlyingNetwork implementation.

func (*DefaultTProxy) DefaultCertPool added in v3.18.0 

func (tp *DefaultTProxy) DefaultCertPool() *x509.CertPool

DefaultCertPool implements model.UnderlyingNetwork

func (*DefaultTProxy) DialContext added in v3.17.0 

func (tp *DefaultTProxy) DialContext(ctx context.Context, network, address string) (net.Conn, error)

DialContext implements UnderlyingNetwork.

func (*DefaultTProxy) DialTimeout added in v3.19.0 

func (tp *DefaultTProxy) DialTimeout() time.Duration

DialTimeout implements model.UnderlyingNetwork

func (*DefaultTProxy) GetaddrinfoLookupANY added in v3.17.0 

func (tp *DefaultTProxy) GetaddrinfoLookupANY(ctx context.Context, domain string) ([]string, string, error)

GetaddrinfoLookupANY implements UnderlyingNetwork.

func (*DefaultTProxy) GetaddrinfoResolverNetwork added in v3.17.0 

func (tp *DefaultTProxy) GetaddrinfoResolverNetwork() string

GetaddrinfoResolverNetwork implements UnderlyingNetwork.

func (*DefaultTProxy) ListenTCP added in v3.19.0 

func (tp *DefaultTProxy) ListenTCP(network string, addr *net.TCPAddr) (net.Listener, error)

ListenTCP implements UnderlyingNetwork.

func (*DefaultTProxy) ListenUDP added in v3.17.0 

func (tp *DefaultTProxy) ListenUDP(network string, addr *net.UDPAddr) (model.UDPLikeConn, error)

ListenUDP implements UnderlyingNetwork.

type DialContextFunc 

type DialContextFunc func(context.Context, string, string) (net.Conn, error)

DialContextFunc is the type of net.Dialer.DialContext.

type ErrGetaddrinfo added in v3.16.0 

type ErrGetaddrinfo struct {
	// Err is the error proper.
	Underlying error

	// Code is getaddrinfo's return code.
	Code int64
}

ErrGetaddrinfo represents a getaddrinfo failure.

func NewErrGetaddrinfo added in v3.19.0 

func NewErrGetaddrinfo(code int64, err error) *ErrGetaddrinfo

NewErrGetaddrinfo creates a new instance of the ErrGetaddrinfo type.

func (*ErrGetaddrinfo) Error added in v3.16.0 

func (err *ErrGetaddrinfo) Error() string

Error returns the underlying error's string.

func (*ErrGetaddrinfo) Unwrap added in v3.16.0 

func (err *ErrGetaddrinfo) Unwrap() error

Unwrap allows to get the underlying error value.

type ErrWrapper 

type ErrWrapper struct {
	// Failure is the OONI failure string. The failure strings are
	// loosely backward compatible with Measurement Kit.
	//
	// This is either one of the FailureXXX strings or any other
	// string like `unknown_failure: ...`. The latter represents an
	// error that we have not yet mapped to a failure.
	Failure string

	// Operation is the operation that failed.
	//
	// If possible, the Operation string SHOULD be a _major_
	// operation. Major operations are:
	//
	// - ResolveOperation: resolving a domain name failed
	// - ConnectOperation: connecting to an IP failed
	// - TLSHandshakeOperation: TLS handshaking failed
	// - QUICHandshakeOperation: QUIC handshaking failed
	// - HTTPRoundTripOperation: other errors during round trip
	//
	// Because a network connection doesn't necessarily know
	// what is the current major operation we also have the
	// following _minor_ operations:
	//
	// - CloseOperation: CLOSE failed
	// - ReadOperation: READ failed
	// - WriteOperation: WRITE failed
	//
	// If an ErrWrapper referring to a major operation is wrapping
	// another ErrWrapper and such ErrWrapper already refers to
	// a major operation, then the new ErrWrapper should use the
	// child ErrWrapper major operation. Otherwise, it should use
	// its own major operation. This way, the topmost wrapper is
	// supposed to refer to the major operation that failed.
	Operation string

	// WrappedErr is the error that we're wrapping.
	WrappedErr error
}

ErrWrapper is our error wrapper for Go errors. The key objective of this structure is to properly set Failure, which is also returned by the Error() method, to be one of the OONI failure strings.

OONI failure strings are defined in the github.com/ooni/spec repo at https://github.com/ooni/spec/blob/master/data-formats/df-007-errors.md.

func NewErrWrapper 

func NewErrWrapper(c classifier, op string, err error) *ErrWrapper

NewErrWrapper creates a new ErrWrapper using the given classifier, operation name, and underlying error.

This function panics if classifier is nil, or operation is the empty string or error is nil.

If the err argument has already been classified, the returned error wrapper will use the same classification string and will determine whether to keep the major operation as documented in the ErrWrapper.Operation documentation.

func NewTopLevelGenericErrWrapper 

func NewTopLevelGenericErrWrapper(err error) *ErrWrapper

NewTopLevelGenericErrWrapper wraps an error occurring at top level using a generic classifier as classifier. This is the function you should call when you suspect a given error hasn't already been wrapped. This function panics if err is nil.

If the err argument has already been classified, the returned error wrapper will use the same classification string and failed operation of the original error.

func (*ErrWrapper) Error 

func (e *ErrWrapper) Error() string

Error returns the OONI failure string for this error.

func (*ErrWrapper) MarshalJSON 

func (e *ErrWrapper) MarshalJSON() ([]byte, error)

MarshalJSON converts an ErrWrapper to a JSON value.

func (*ErrWrapper) Unwrap 

func (e *ErrWrapper) Unwrap() error

Unwrap allows to access the underlying error.

type HTTPTransportOption added in v3.19.0 

type HTTPTransportOption func(txp *oohttp.Transport)

HTTPTransportOption is an initialization option for NewHTTPTransport.

func HTTPTransportOptionDisableCompression added in v3.19.0 

func HTTPTransportOptionDisableCompression(value bool) HTTPTransportOption

HTTPTransportOptionDisableCompression configures the .DisableCompression field, which otherwise is set to true, so that this code is ready for measuring out of the box.

func HTTPTransportOptionMaxConnsPerHost added in v3.19.0 

func HTTPTransportOptionMaxConnsPerHost(value int) HTTPTransportOption

HTTPTransportOptionMaxConnsPerHost configures the .MaxConnPerHosts field, which otherwise uses the default set in github.com/ooni/oohttp.

func HTTPTransportOptionProxyURL added in v3.19.0 

func HTTPTransportOptionProxyURL(proxyURL *url.URL) HTTPTransportOption

HTTPTransportOptionProxyURL configures the transport to use the given proxyURL or disables proxying (already the default) if the proxyURL is nil.

func HTTPTransportOptionTLSClientConfig added in v3.19.0 

func HTTPTransportOptionTLSClientConfig(config *tls.Config) HTTPTransportOption

HTTPTransportOptionTLSClientConfig configures the .TLSClientConfig field, which otherwise is nil, to imply using the default config.

TODO(https://github.com/ooni/probe/issues/2536): using the default config breaks tests using netem and this option is the workaround we're using to address this limitation. Future releases MIGHT use a different technique and, as such, we MAY remove this option when we don't need it anymore.

type MaybeCustomUnderlyingNetwork added in v3.19.0 

type MaybeCustomUnderlyingNetwork struct {
	// contains filtered or unexported fields
}

MaybeCustomUnderlyingNetwork is a nil-safe model.UnderlyingNetwork provider. When the pointer to the MaybeCustomUnderlyingNetwork is nil or the underlying field is nil, the Get method of the MaybeCustomUnderlyingNetwork falls back to calling [tproxySingleton].

func (*MaybeCustomUnderlyingNetwork) Get added in v3.19.0 

func (p *MaybeCustomUnderlyingNetwork) Get() model.UnderlyingNetwork

Get returns the model.UnderlyingNetwork returned by [tproxySingleton] if p is nil or the underlying field is nil and otherwise returns the value of the underlying field.

type NetemUnderlyingNetworkAdapter added in v3.19.0 

type NetemUnderlyingNetworkAdapter struct {
	UNet netem.UnderlyingNetwork
}

NetemUnderlyingNetworkAdapter adapts netem.UnderlyingNetwork to model.UnderlyingNetwork.

func (*NetemUnderlyingNetworkAdapter) DefaultCertPool added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) DefaultCertPool() *x509.CertPool

DefaultCertPool implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) DialContext added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) DialContext(ctx context.Context, network string, address string) (net.Conn, error)

DialContext implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) DialTimeout added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) DialTimeout() time.Duration

DialTimeout implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) GetaddrinfoLookupANY added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) GetaddrinfoLookupANY(ctx context.Context, domain string) ([]string, string, error)

GetaddrinfoLookupANY implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) GetaddrinfoResolverNetwork added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) GetaddrinfoResolverNetwork() string

GetaddrinfoResolverNetwork implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) ListenTCP added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) ListenTCP(network string, addr *net.TCPAddr) (net.Listener, error)

ListenTCP implements model.UnderlyingNetwork

func (*NetemUnderlyingNetworkAdapter) ListenUDP added in v3.19.0 

func (a *NetemUnderlyingNetworkAdapter) ListenUDP(network string, addr *net.UDPAddr) (model.UDPLikeConn, error)

ListenUDP implements model.UnderlyingNetwork

type Netx added in v3.19.0 

type Netx struct {
	// Underlying is the OPTIONAL [model.UnderlyingNetwork] to use. Leaving this field
	// nil makes this implementation functionally equivalent to netxlite top-level functions.
	Underlying model.UnderlyingNetwork
}

Netx allows constructing netxlite data types using a specific model.UnderlyingNetwork.

func (*Netx) ListenTCP added in v3.19.0 

func (netx *Netx) ListenTCP(network string, addr *net.TCPAddr) (net.Listener, error)

ListenTCP creates a new listening TCP socket using the given address.

func (*Netx) MaybeCustomUnderlyingNetwork added in v3.19.0 

func (netx *Netx) MaybeCustomUnderlyingNetwork() *MaybeCustomUnderlyingNetwork

MaybeCustomUnderlyingNetwork wraps the model.UnderlyingNetwork using a *MaybeCustomUnderlyingNetwork.

func (*Netx) NewDialerWithResolver added in v3.19.0 

func (netx *Netx) NewDialerWithResolver(dl model.DebugLogger, r model.Resolver, w ...model.DialerWrapper) model.Dialer

NewDialerWithResolver creates a [Dialer] with error wrapping.

This dialer will try to connect to each of the resolved IP address sequentially. In case of failure, such a resolver will return the first error that occurred. This implementation strategy is a QUIRK that is documented at TODO(https://github.com/ooni/probe/issues/1779).

The model.DialerWrapper arguments wrap the returned dialer in such a way that we can implement the legacy [netx] package. New code MUST NOT use this functionality, which we'd like to remove ASAP.

func (*Netx) NewDialerWithoutResolver added in v3.19.0 

func (netx *Netx) NewDialerWithoutResolver(dl model.DebugLogger, w ...model.DialerWrapper) model.Dialer

NewDialerWithoutResolver implements model.MeasuringNetwork.

func (*Netx) NewHTTP3TransportStdlib added in v3.19.0 

func (netx *Netx) NewHTTP3TransportStdlib(logger model.DebugLogger) model.HTTPTransport

NewHTTP3TransportStdlib creates a new HTTPTransport using http3 that uses standard functionality for everything but the logger.

func (*Netx) NewHTTPTransportStdlib added in v3.19.0 

func (netx *Netx) NewHTTPTransportStdlib(logger model.DebugLogger) model.HTTPTransport

NewHTTPTransportStdlib creates a new HTTPTransport using the stdlib for DNS resolutions and TLS.

This factory calls NewHTTPTransport with suitable dialers.

This function behavior is QUIRKY as documented in NewHTTPTransport.

func (*Netx) NewParallelDNSOverHTTPSResolver added in v3.19.0 

func (netx *Netx) NewParallelDNSOverHTTPSResolver(logger model.DebugLogger, URL string) model.Resolver

NewParallelDNSOverHTTPSResolver implements model.MeasuringNetwork.

func (*Netx) NewParallelUDPResolver added in v3.19.0 

func (netx *Netx) NewParallelUDPResolver(logger model.DebugLogger, dialer model.Dialer, address string) model.Resolver

NewParallelUDPResolver implements model.MeasuringNetwork.

func (*Netx) NewQUICDialerWithResolver added in v3.19.0 

func (netx *Netx) NewQUICDialerWithResolver(listener model.UDPListener, logger model.DebugLogger,
	resolver model.Resolver, wrappers ...model.QUICDialerWrapper) (outDialer model.QUICDialer)

NewQUICDialerWithResolver creates a QUICDialer with error wrapping.

Unlike the dialer returned by Netx.NewDialerWithResolver, this dialer MAY attempt happy eyeballs, perform parallel dial attempts, and return an error that aggregates all the errors that occurred.

The model.QUICDialerWrapper arguments wraps the returned dialer in such a way that we can implement the legacy [netx] package. New code MUST NOT use this functionality, which we'd like to remove ASAP.

func (*Netx) NewQUICDialerWithoutResolver added in v3.19.0 

func (netx *Netx) NewQUICDialerWithoutResolver(listener model.UDPListener,
	logger model.DebugLogger, wrappers ...model.QUICDialerWrapper) model.QUICDialer

NewQUICDialerWithoutResolver implements model.MeasuringNetwork.

func (*Netx) NewStdlibResolver added in v3.19.0 

func (netx *Netx) NewStdlibResolver(logger model.DebugLogger) model.Resolver

NewStdlibResolver implements model.MeasuringNetwork.

func (*Netx) NewTLSHandshakerStdlib added in v3.19.0 

func (netx *Netx) NewTLSHandshakerStdlib(logger model.DebugLogger) model.TLSHandshaker

NewTLSHandshakerStdlib implements model.MeasuringNetwork.

func (*Netx) NewTLSHandshakerUTLS added in v3.19.0 

func (netx *Netx) NewTLSHandshakerUTLS(logger model.DebugLogger, id *utls.ClientHelloID) model.TLSHandshaker

NewTLSHandshakerUTLS implements model.MeasuringNetwork.

func (*Netx) NewUDPListener added in v3.19.0 

func (netx *Netx) NewUDPListener() model.UDPListener

NewUDPListener creates a new UDPListener using the underlying *Netx structure to create listening UDP sockets.

type NullResolver added in v3.16.0 

type NullResolver struct{}

NullResolver is a resolver that is not capable of resolving domain names to IP addresses and always returns ErrNoResolver.

func (*NullResolver) Address added in v3.16.0 

func (r *NullResolver) Address() string

func (*NullResolver) CloseIdleConnections added in v3.16.0 

func (r *NullResolver) CloseIdleConnections()

func (*NullResolver) LookupHTTPS added in v3.16.0 

func (r *NullResolver) LookupHTTPS(
	ctx context.Context, domain string) (*model.HTTPSSvc, error)

func (*NullResolver) LookupHost added in v3.16.0 

func (r *NullResolver) LookupHost(ctx context.Context, hostname string) (addrs []string, err error)

func (*NullResolver) LookupNS added in v3.16.0 

func (r *NullResolver) LookupNS(
	ctx context.Context, domain string) ([]*net.NS, error)

func (*NullResolver) Network added in v3.16.0 

func (r *NullResolver) Network() string

type ParallelResolver added in v3.15.0 

type ParallelResolver struct {
	// Txp is the MANDATORY underlying DNS transport.
	Txp model.DNSTransport
}

ParallelResolver uses a transport and performs a LookupHost operation in a parallel fashion, hence its name.

You should probably use NewUnwrappedParallelResolver to create a new instance of this type.

func NewUnwrappedParallelResolver added in v3.15.0 

func NewUnwrappedParallelResolver(t model.DNSTransport) *ParallelResolver

UnwrappedParallelResolver creates a new ParallelResolver instance. This instance is not wrapped and you should wrap if before using it.

func (*ParallelResolver) Address added in v3.15.0 

func (r *ParallelResolver) Address() string

Address returns the "address" of the underlying transport.

func (*ParallelResolver) CloseIdleConnections added in v3.15.0 

func (r *ParallelResolver) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*ParallelResolver) LookupHTTPS added in v3.15.0 

func (r *ParallelResolver) LookupHTTPS(
	ctx context.Context, hostname string) (*model.HTTPSSvc, error)

LookupHTTPS implements Resolver.LookupHTTPS.

func (*ParallelResolver) LookupHost added in v3.15.0 

func (r *ParallelResolver) LookupHost(ctx context.Context, hostname string) ([]string, error)

LookupHost performs an A lookup in parallel with an AAAA lookup.

func (*ParallelResolver) LookupNS added in v3.15.0 

func (r *ParallelResolver) LookupNS(
	ctx context.Context, hostname string) ([]*net.NS, error)

LookupNS implements Resolver.LookupNS.

func (*ParallelResolver) Network added in v3.15.0 

func (r *ParallelResolver) Network() string

Network returns the "network" of the underlying transport.

func (*ParallelResolver) Transport added in v3.15.0 

func (r *ParallelResolver) Transport() model.DNSTransport

Transport returns the transport being used.

type ResolverShortCircuitIPAddr added in v3.19.0 

type ResolverShortCircuitIPAddr struct {
	Resolver model.Resolver
}

ResolverShortCircuitIPAddr recognizes when the input hostname is an IP address and returns it immediately to the caller.

func (*ResolverShortCircuitIPAddr) Address added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) Address() string

func (*ResolverShortCircuitIPAddr) CloseIdleConnections added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) CloseIdleConnections()

func (*ResolverShortCircuitIPAddr) LookupHTTPS added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) LookupHTTPS(ctx context.Context, hostname string) (*model.HTTPSSvc, error)

func (*ResolverShortCircuitIPAddr) LookupHost added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) LookupHost(ctx context.Context, hostname string) ([]string, error)

func (*ResolverShortCircuitIPAddr) LookupNS added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) LookupNS(
	ctx context.Context, hostname string) ([]*net.NS, error)

func (*ResolverShortCircuitIPAddr) Network added in v3.19.0 

func (r *ResolverShortCircuitIPAddr) Network() string

type SerialResolver deprecated 

type SerialResolver struct {
	// NumTimeouts is MANDATORY and counts the number of timeouts.
	NumTimeouts *atomic.Int64

	// Txp is the MANDATORY underlying DNS transport.
	Txp model.DNSTransport
}

SerialResolver uses a transport and performs a LookupHost operation in a serial fashion (query for A first, wait for response, then query for AAAA, and wait for response), hence its name.

You should probably use NewSerialResolver to create a new instance.

Deprecated: please use ParallelResolver in new code. We cannot remove this code as long as we use tracing for measuring.

QUIRK: unlike the ParallelResolver, this resolver's LookupHost retries each query three times for soft errors.

func NewUnwrappedSerialResolver added in v3.16.0 

func NewUnwrappedSerialResolver(t model.DNSTransport) *SerialResolver

NewUnwrappedSerialResolver creates a new, and unwrapped, SerialResolver instance.

func (*SerialResolver) Address 

func (r *SerialResolver) Address() string

Address returns the "address" of the underlying transport.

func (*SerialResolver) CloseIdleConnections 

func (r *SerialResolver) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

func (*SerialResolver) LookupHTTPS 

func (r *SerialResolver) LookupHTTPS(
	ctx context.Context, hostname string) (*model.HTTPSSvc, error)

LookupHTTPS implements Resolver.LookupHTTPS.

func (*SerialResolver) LookupHost 

func (r *SerialResolver) LookupHost(ctx context.Context, hostname string) ([]string, error)

LookupHost performs an A lookup followed by an AAAA lookup for hostname.

func (*SerialResolver) LookupNS added in v3.15.0 

func (r *SerialResolver) LookupNS(
	ctx context.Context, hostname string) ([]*net.NS, error)

LookupNS implements Resolver.LookupNS.

func (*SerialResolver) Network 

func (r *SerialResolver) Network() string

Network returns the "network" of the underlying transport.

func (*SerialResolver) Transport 

func (r *SerialResolver) Transport() model.DNSTransport

Transport returns the transport being used.

type TLSConn 

type TLSConn = model.TLSConn

The TLSConn alias was originally defined here in netxlite and we want to keep it available to other packages for now.

type UTLSConn added in v3.17.0 

type UTLSConn struct {
	// We include the real UConn
	*utls.UConn
	// contains filtered or unexported fields
}

UTLSConn implements TLSConn and uses a utls UConn as its underlying connection

func NewUTLSConn added in v3.17.0 

func NewUTLSConn(conn net.Conn, config *tls.Config, cid *utls.ClientHelloID) (*UTLSConn, error)

NewUTLSConn creates a new connection with the given client hello ID.

func (*UTLSConn) ConnectionState added in v3.17.0 

func (c *UTLSConn) ConnectionState() tls.ConnectionState

func (*UTLSConn) HandshakeContext added in v3.17.0 

func (c *UTLSConn) HandshakeContext(ctx context.Context) (err error)

func (*UTLSConn) NetConn added in v3.17.0 

func (c *UTLSConn) NetConn() net.Conn

Source Files

View all Source files

Directories

Path Synopsis
internal
gencertifi
generrno

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.