revproxy

Documentation

Overview

Package revproxy implements a minimal HTTP reverse proxy that caches files locally on disk, backed by objects in an S3 bucket.

Limitations

By default, only objects marked "immutable" by the target server are eligible to be cached. Volatile objects that specify a max-age are also cached in-memory, but are not persisted on disk or in S3. If we think it's worthwhile we can spend some time to add more elaborate cache pruning, but for now we're doing the simpler thing.

Index

• type Server
  • func (s *Server) Metrics() *expvar.Map
  • func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

Types

type Server

type Server struct {
	// Targets is the list of hosts for which the proxy should forward requests.
	// Host names should be fully-qualified ("host.example.com").
	Targets []string

	// Local is the path of a local cache directory where responses are cached.
	// It must be non-empty.
	Local string

	// S3Client is the S3 client used to read and write cache entries to the
	// backing store. It must be non-nil
	S3Client *s3util.Client

	// KeyPrefix, if non-empty, is prepended to each key stored into S3, with an
	// intervening slash.
	KeyPrefix string

	// Logf, if non-nil, is used to write log messages. If nil, logs are
	// discarded.
	Logf func(string, ...any)

	// LogRequests, if true, enables detailed (but noisy) debug logging of all
	// requests handled by the reverse proxy. Logs are written to Logf.
	//
	// Each request is presented in the format:
	//
	//     B U:"<url>" H:<digest> C:<bool>
	//     E H:<digest> <disposition> B:<bytes> (<time> elapsed)
	//     - H:<digest> miss
	//
	// The "B" line is when the request began, and "E" when it was finished.
	// The abbreviated fields are:
	//
	//     U:       -- request URL
	//     H:       -- request URL digest (cache key)
	//     C:       -- whether the request is cacheable (true/false)
	//     B:       -- body size in bytes (for hits)
	//
	// The dispositions of a request are:
	//
	//     hit mem  -- cache hit in memory (volatile)
	//     hit disk -- cache hit in local disk
	//     hit S3   -- cache hit in S3 (faulted to disk)
	//     fetch    -- fetched from the origin server
	//
	// On fetches, the "RC" tag indicates whether the response is cacheable,
	// with "no" meaning it was not cached at all, "mem" meaning it was cached
	// as a short-lived volatile response in memory, and "yes" meaning it was
	// cached on disk (and S3).
	LogRequests bool
	// contains filtered or unexported fields
}

Server is a caching reverse proxy server that caches successful responses to GET requests for certain designated domains.

The host field of the request URL must match one of the configured targets. If not, the request is rejected with HTTP 502 (Bad Gateway). Otherwise, the request is forwarded. A successful response will be cached if the server's Cache-Control does not include "no-store", and does include "immutable".

In addition, a successful response that is not immutable and specifies a max-age will be cached temporarily in-memory.

Cache Format

A cached response is a file with a header section and the body, separated by a blank line. Only a subset of response headers are saved.

Cache Responses

For requests handled by the proxy, the response includes an "X-Cache" header indicating how the response was obtained:

• "hit, memory": The response was served out of the memory cache.
• "hit, local": The response was served out of the local cache.
• "hit, remote": The response was faulted in from S3.
• "fetch, cached": The response was forwarded to the target and cached.
• "fetch, uncached": The response was forwarded to the target and not cached.

For results intersecting with the cache, it also reports a X-Cache-Id giving the storage key of the cache object.

func (*Server) Metrics

func (s *Server) Metrics() *expvar.Map

Metrics returns a map of cache server metrics for s. The caller is responsible to publish these metrics as desired.

func (*Server) ServeHTTP

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface for the proxy.