chttp

Documentation

Overview

Package chttp is the HTTP client every katbyte tool talks to APIs with: a stalled server fails fast, a passing failure is tried again without ever re-sending a write whose fate is unknown, and every exchange is traced to whatever logger it is handed, with credentials blanked.

It uses the standard library alone and logs nothing unasked, so an SDK can be built on it. What is worth another try, how long to wait, and what is secret are each API's own to set (Options); the defaults are cautious.

Index

• Constants
• Variables
• func Dropped(err error) bool
• func Explain(err error) error
• func IsNotFound(err error) bool
• func IsWebPage(contentType string, body []byte) bool
• func KeepCredentialsOnHost(headers ...string) func(req *http.Request, via []*http.Request) error
• func MarkRetrySafe(req *http.Request) *http.Request
• func NewBaseTransport(o Options) http.RoundTripper
• func Preview(body []byte) string
• func RedactError(err error, also ...string) error
• func RedactJSON(text string, also ...string) string
• func RedactURL(raw string, also ...string) string
• func RefuseRedirects(advice string) func(req *http.Request, via []*http.Request) error
• func Refused(err error) bool
• func SecretName(name string, also ...string) bool
• func StatusCode(err error) int
• func Tries(resp *http.Response) int
• type Client
  • func New(o Options) *Client
  • func (c *Client) Fetch(req *http.Request, limit int64) (*http.Response, []byte, error)
• type Logger
• type Options
• type RedirectError
  • func (e *RedirectError) Error() string
• type Retry
• type RetryTransport
  • func NewRetryTransport(o Options, next http.RoundTripper) *RetryTransport
  • func (t *RetryTransport) RoundTrip(req *http.Request) (*http.Response, error)
• type StatusError
  • func (e *StatusError) Error() string
• type Transport
  • func NewTransport(o Options, next http.RoundTripper) *Transport
  • func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

Constants

const DefaultHeaderWait = 30 * time.Second

DefaultHeaderWait is how long a server has to start answering.

const DefaultTraceBody = 8 << 10

DefaultTraceBody is how much of a body a trace prints: enough to see what came back, not a whole listing.

const DefaultTries = 3

DefaultTries is how many times a request is sent before giving up: enough for a blip, not enough to turn an outage into a long hang.

const MaxRetryAfter = 60 * time.Second

MaxRetryAfter caps the wait a Retry-After header can ask for: longer is "come back later", not "sit there".

const PreviewLen = 300

PreviewLen is how much of a response body a StatusError carries.

Variables

var ErrTooLarge = errors.New("the answer is too large")

ErrTooLarge is an answer longer than Fetch was told to hold.

Functions

func Dropped

func Dropped(err error) bool

Dropped reports whether a connection that was working went away: reset, closed by the other end, or cut off mid-answer. That is the one failure worth sending the request again for (Retry); a server that could not be reached, a timeout or a cancel would meet the same again.

func Explain

func Explain(err error) error

Explain adds to a request that got no answer what the operating system may be doing, where known: "no route to host" on a Mac is usually its Local Network privacy, which no retry fixes. Transport does this itself; Explain is for a client built on another transport.

func IsNotFound

func IsNotFound(err error) bool

IsNotFound reports whether err is a 404 from the server.

func IsWebPage

func IsWebPage(contentType string, body []byte) bool

IsWebPage reports whether an answer is a web page where an API's answer was expected: a wrong address or a login page answers 200 with one. The body decides, since a proxy labels anything anything: a document that starts as HTML is a page, and one called HTML is only if it starts with a tag.

func KeepCredentialsOnHost

func KeepCredentialsOnHost(headers ...string) func(req *http.Request, via []*http.Request) error

KeepCredentialsOnHost is a CheckRedirect that follows a redirect but drops Authorization and the named headers when it leaves the first host or steps down from https. Go alone keeps a custom header for any host.

func MarkRetrySafe

func MarkRetrySafe(req *http.Request) *http.Request

MarkRetrySafe says a request may be re-sent though its method says otherwise: a GraphQL query is a read that travels as a POST.

func NewBaseTransport

func NewBaseTransport(o Options) http.RoundTripper

NewBaseTransport is the default transport with timeouts so a stalled server fails fast: ten seconds to connect, ten for TLS, HeaderWait to start answering. Exported for a caller that must build its own client.

func Preview

func Preview(body []byte) string

Preview is the start of a body for an error to carry: trimmed, cut at PreviewLen on a whole character with "..." for more, and its credentials blanked (RedactJSON), a field cut off part way included.

func RedactError

func RedactError(err error, also ...string) error

RedactError blanks the credentials in the URL a failed request prints: a request that got no answer fails with its whole URL, key included. The error keeps its type and what it wraps. Use it where the error is first seen; a message already built around it keeps its text.

func RedactJSON

func RedactJSON(text string, also ...string) string

RedactJSON blanks every string field in JSON text, whole or cut short, whose name is a credential's (SecretName, plus also). It is the rule a trace hides a body's secrets by, for whatever else writes one down.

func RedactURL

func RedactURL(raw string, also ...string) string

RedactURL blanks the password and every credential parameter in a URL, the common ones (api_key, apikey, api_token, access_token, token) and those in also, and leaves the rest as sent.

func RefuseRedirects

func RefuseRedirects(advice string) func(req *http.Request, via []*http.Request) error

RefuseRedirects is a CheckRedirect that follows none: the request fails with a RedirectError saying where it was sent, plus the advice. For an API that answers where it is asked, a redirect means the address is wrong, and following it is worse than failing: Go turns a POST into a GET on a 302, the GET answers 200, and a write reports success having done nothing.

func Refused

func Refused(err error) bool

Refused reports whether nothing was listening where a request was sent. It is not retried unless a caller whose server restarts under it asks, with Dropped(err) || Refused(err) as its Retry's Error.

func SecretName

func SecretName(name string, also ...string) bool

SecretName reports whether a name holds a credential, by the rule a trace hides one by: a common credential parameter, one of also, or a name ending password, passphrase, passkey, secret, token, cookie, apikey, api_key, privatekey or private_key, in any case.

func StatusCode

func StatusCode(err error) int

StatusCode is the status of a *StatusError in err's chain, or 0.

func Tries

func Tries(resp *http.Response) int

Tries is how many times the request behind an answer was sent: 1 for first time, 0 for a request no client of this package sent.

Types

type Client

type Client struct {
	*http.Client
	// contains filtered or unexported fields
}

Client is an http.Client whose requests are retried and traced as its Options say, with Fetch added for an answer read whole.

func New

func New(o Options) *Client

New makes a client. It sets no overall timeout and follows redirects as Go does; set Timeout and CheckRedirect for otherwise (RefuseRedirects, KeepCredentialsOnHost). A Timeout covers one Do with all its tries and waits; each time Fetch asks again is a Do of its own.

func (*Client) Fetch

func (c *Client) Fetch(req *http.Request, limit int64) (*http.Response, []byte, error)

Fetch sends a request and reads its whole answer, up to limit bytes; a longer one is ErrTooLarge, not an answer cut to fit, and math.MaxInt64 is no limit. The response comes back read and closed whatever its status. An answer that stops part way is asked for again like any dropped connection, which a transport cannot do since the body is read after it returns; the usual rules on what may be re-sent, and how often, hold.

type Logger

type Logger interface {
	Debugf(format string, args ...any)
	Tracef(format string, args ...any)
}

Logger is where a client says what it does: a retry at debug, each exchange at trace. A logrus logger is one as it stands. What is traced is built only when the logger formats it, so a level that is off costs nothing; a logger must therefore format before Tracef returns.

type Options

type Options struct {
	// Name says whose traffic this is in a log, so a tool talking to two APIs can tell them apart.
	Name string
	// Log is where retries and the trace of each exchange go. Nil logs nothing.
	Log Logger
	// Retry is when a request is sent again.
	Retry Retry
	// HeaderWait is how long the server has to start answering one try; 0 is DefaultHeaderWait. A search that runs before it writes a byte wants
	// longer. Reading the answer is not bounded by it.
	HeaderWait time.Duration
	// SecretHeaders are headers a trace must not show, beside Authorization, Proxy-Authorization, Cookie and Set-Cookie.
	SecretHeaders []string
	// SecretNames are names a trace must not show the value of wherever they appear, compared without case, beside those hidden unasked (SecretName):
	// an API's "pass" or "pin" needs naming, proxyPassword does not.
	SecretNames []string
	// TraceBody is how much of a body a trace prints: 0 is DefaultTraceBody, negative prints none. No more is ever read for it.
	TraceBody int
	// Base is what sends the requests, under the retries and the trace; nil is NewBaseTransport. A test or a proxy hands in its own.
	Base http.RoundTripper
}

Options are what a client is made with. The zero value is a client that logs nothing and retries with the defaults.

type RedirectError

type RedirectError struct {
	Method string
	Path   string
	// To is where it was sent, credentials hidden
	To string
	// Advice is what to do about it on this API: which setting to fix
	Advice string
}

RedirectError is a request that was redirected and not followed (RefuseRedirects).

func (*RedirectError) Error

func (e *RedirectError) Error() string

type Retry

type Retry struct {
	// Tries is the most a request is ever sent; 0 is DefaultTries, 1 never retries. Fetch's own re-reads count against the same number.
	Tries int
	// Wait is how long to wait after attempt n (from 0) failed; nil is 1s, 2s, 4s. A 429's Retry-After is used instead, up to MaxRetryAfter.
	Wait func(attempt int) time.Duration
	// Status says whether this status is worth another try; nil is 502, 503 and 504. A plain 500 is left out: most servers answer it for what will
	// never succeed. An API that uses 500 for "busy" adds it.
	Status func(code int) bool
	// Error says whether a request that got no answer is worth sending again; nil is Dropped. A server that cannot be reached at all, or a timeout,
	// is not: the same wait again rarely helps.
	Error func(err error) bool
}

Retry is when a request is sent again and how long after. The zero value is three tries, a second then two apart, for a 502, 503 or 504 and for a dropped connection. A request is sent again only when that cannot do the work twice: a 429 was refused before it was acted on, so anything may retry it; otherwise only a GET, HEAD, OPTIONS or a request marked MarkRetrySafe is, since a write that got no answer may still have landed.

type RetryTransport

type RetryTransport struct {
	// contains filtered or unexported fields
}

RetryTransport retries as the Options say (Retry): a 429 for any request, a dropped connection or a listed status for one safe to repeat. A cancelled context ends the wait.

func NewRetryTransport

func NewRetryTransport(o Options, next http.RoundTripper) *RetryTransport

NewRetryTransport wraps next with the retries the options ask for.

func (*RetryTransport) RoundTrip

func (t *RetryTransport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.

type StatusError

type StatusError struct {
	Method     string
	Path       string
	StatusCode int
	// Expected are the statuses taken as success, when named
	Expected []int
	// Body is the start of the answer (Preview), printed with its credentials blanked: a refused save echoes the record back, key included
	Body string
	// Tries is how often the request was sent, when more than once
	Tries int
	// Note is what the caller knows of this status on this API: which key to check for a 401, what a 403 wants
	Note string
}

StatusError is an answer with a status the request did not ask for.

func (*StatusError) Error

func (e *StatusError) Error() string

type Transport

type Transport struct {
	// contains filtered or unexported fields
}

Transport traces each request and answer to the Log with secrets blanked and JSON laid out, and explains a request that got no answer where the operating system is the cause (Explain). With no Log it only explains. While tracing, the start of a text answer is read before it is handed on.

func NewTransport

func NewTransport(o Options, next http.RoundTripper) *Transport

NewTransport wraps next with the trace the options ask for.

func (*Transport) RoundTrip

func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.