pagecache

README

pagecache

Note: This library is under active development and most pieces haven't been tested yet. Do not use in production until a tagged release is out.

Package pagecache provides a stable interface and helpers for caching HTTP responses.

Features

• Stable cache interface.
• Simple and easy-to-use API.
• Multiple helpers, making implementation easier.

pagecache.Cache implementations

The pagecace package itself only provides a cache interface and helper functions for users who wish to implement that interface. You can either use an implementation created by someone else or write your own.

Implementations

• memorycachex provides a thread-safe in-memory cache using the Mockingjay cache replacement policy.

If you wrote a pagecache.Cache implementation and wish it to be linked here, please send a patch.

Installation

To install pagecache alone, run:

go get git.sr.ht/~jamesponddotco/pagecache-go

Contributing

Anyone can help make pagecache better. Check out the contribution guidelines for more information.

Resources

The following resources are available:

• Package documentation.
• Support and general discussions.
• Patches and development related questions.
• Instructions on how to prepare patches.
• Feature requests and bug reports.

---

Released under the MIT License.

Documentation

Overview

Package cachex implements a cache of HTTP responses.

Index

• Constants
• Variables
• func Key(name string, req *http.Request, extra ...string) string
• func LoadResponse(request, response []byte) (*http.Response, error)
• func SaveResponse(resp *http.Response) (request, response []byte, err error)
• type Behavior
• type Cache
• type Entry
• type Error
  • func NewCacheError(errType ErrorType, err error) *Error
  • func (ce *Error) Error() string
  • func (ce *Error) Type() ErrorType
  • func (ce *Error) Unwrap() error
• type ErrorType
  • func (et ErrorType) String() string
• type Policy
  • func DefaultPolicy() *Policy
  • func (p *Policy) IsCacheable(resp *http.Response) bool
  • func (p *Policy) TTL(resp *http.Response) time.Duration
• type Rule
  • func (r *Rule) Match(url string) bool

Constants

const (
	// DefaultCacheName is the default name used for the default cache.
	DefaultCacheName string = "httpx"

	// DefaultCapacity is the default capacity of the memory cache when not
	// specified.
	DefaultCapacity uint64 = 128
)

const (
	// DefaultTTL is the default time-to-live for cached responses.
	DefaultTTL time.Duration = 60 * time.Minute

	// DefaultMaxBodySize is the default maximum size of a response body.
	DefaultMaxBodySize int64 = 5 * 1024 * 1024
)

const ErrInvalidResponse xerrors.Error = "invalid response"

ErrInvalidResponse is returned when a response is invalid.

Variables

var (
	// ErrCacheMiss is returned when a cache entry is not found for the given key.
	ErrCacheMiss = NewCacheError(ErrNotFound, xerrors.Error("cache miss"))

	// ErrCacheExpired is returned when a cache entry is found but has expired.
	ErrCacheExpired = NewCacheError(ErrNotFound, xerrors.Error("cache entry expired"))

	// ErrKeyNotFound is returned when a cache entry is not found for the given key.
	ErrKeyNotFound = NewCacheError(ErrNotFound, xerrors.Error("key not found"))

	// ErrCacheStoreFailed is returned when storing an item in the cache fails.
	ErrCacheStoreFailed = NewCacheError(ErrOperationFailed, xerrors.Error("failed to set cache entry"))

	// ErrCacheDeleteFailed is returned when deleting an item from the cache fails.
	ErrCacheDeleteFailed = NewCacheError(ErrOperationFailed, xerrors.Error("failed to delete cache entry"))

	// ErrCachePurgeFailed is returned when purging the entire cache fails.
	ErrCachePurgeFailed = NewCacheError(ErrOperationFailed, xerrors.Error("failed to purge cache"))
)

Common error messages for Cache implementations.

Functions

func Key

func Key(name string, req *http.Request, extra ...string) string

Key generates a cache key by concatenating information from the given *http.Request, cache name, and optional extra information. It hashes the result with the FNV-1 64-bit algorithm for fast hashing.

The generated key is of the form:

"name:NAME:method:METHOD:url:URL:extra:EXTRA:EXTRA".

This function is not used by the package itself, but is exported for use by packages implementing the Cache interface.

func LoadResponse

func LoadResponse(request, response []byte) (*http.Response, error)

LoadResponse loads an HTTP response from a saved request and response.

func SaveResponse

func SaveResponse(resp *http.Response) (request, response []byte, err error)

SaveResponse saves an HTTP response and the request that generated it.

Types

type Behavior

type Behavior int

Behavior represents the caching behavior for a specific URL pattern.

const (
	// BehaviorInclude means to include the URL in caching, even if it would be
	// excluded by default.
	BehaviorInclude Behavior = iota

	// BehaviorExclude means to exclude the URL from caching.
	BehaviorExclude
)

type Cache

type Cache interface {
	// Get retrieves an *http.Response from the cache associated with the given
	// key.
	Get(ctx context.Context, key string) (*http.Response, error)

	// Set stores an *http.Response in the cache, associated with the given
	// key, and sets the expiration duration.
	Set(ctx context.Context, key string, resp *http.Response, duration time.Duration) error

	// Delete removes the cache entry associated with the given key.
	Delete(ctx context.Context, key string) error

	// Policy returns the cache policy.
	Policy() *Policy

	// Purge clears the entire cache.
	Purge(ctx context.Context) error
}

Cache is a storage mechanism used to store and retrieve HTTP responses for improved reliability and performance.

Implementations can use various caching strategies such as in-memory, file-based, or distributed caches like Redis.

type Entry

type Entry interface {
	// Load loads the HTTP response from the cache entry.
	Load(key string) (*http.Response, error)

	// Access updates the last access time of the entry.
	Access()

	// SetTTL sets the time-to-live of the entry.
	SetTTL(ttl time.Duration)

	// SetSize sets the size of the entry.
	SetSize(size uint64)

	// IsExpired checks if the entry is expired.
	IsExpired() bool
}

Entry is an interface that represents a cache entry.

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error is a custom error type for the cache package.

func NewCacheError

func NewCacheError(errType ErrorType, err error) *Error

NewCacheError creates a new Error with the specified error type and underlying error.

func (*Error) Error

func (ce *Error) Error() string

Error returns a string representation of the CacheError.

func (*Error) Type

func (ce *Error) Type() ErrorType

Type returns the CacheErrorType of the CacheError.

func (*Error) Unwrap

func (ce *Error) Unwrap() error

Unwrap returns the underlying error of the CacheError.

type ErrorType

type ErrorType int

ErrorType represents the type of cache error.

const (
	// ErrNotFound indicates that the requested key was not found in the cache.
	ErrNotFound ErrorType = iota

	// ErrOperationFailed indicates that a cache operation, such as set or delete, has failed.
	ErrOperationFailed
)

func (ErrorType) String

func (et ErrorType) String() string

String returns a string representation of the ErrorType.

type Policy

type Policy struct {
	// AllowedStatusCodes is a list of HTTP status codes that should be cached.
	AllowedStatusCodes map[int]struct{} //nolint:revive // using int as key for performance

	// AllowedMethods is a list of HTTP methods that should be cached.
	AllowedMethods map[string]struct{} //nolint:revive // using string as key for performance

	// ExcludedHeaders is a list of HTTP headers to exclude from caching.
	ExcludedHeaders map[string]struct{} //nolint:revive // see above

	// ExcludedCookies is a list of HTTP cookies to exclude from caching.
	ExcludedCookies map[string]struct{} //nolint:revive // see above

	// Rules is a list of rules to apply to a request in order to determine if it should be cached.
	Rules []*Rule

	// MaxBodySize is the maximum size of the response body allowed to be
	// cached, in bytes. Zero or a negative value indicates no limit.
	MaxBodySize int64

	// UseCacheControl controls whether the cache takes the Cache-Control header
	// into account when deciding whether to cache a response.
	UseCacheControl bool

	// DefaultTTL is the default time-to-live of a cached response. Zero or a
	// negative value is interpreted as no expiration.
	//
	// If UseCacheControl is true, the cache will use the header's value to
	// determine the TTL instead.
	DefaultTTL time.Duration
}

Policy defines under which conditions an HTTP response may be cached.

func DefaultPolicy

func DefaultPolicy() *Policy

DefaultPolicy returns a new *Policy with opinionated but sane defaults.

func (*Policy) IsCacheable

func (p *Policy) IsCacheable(resp *http.Response) bool

IsCacheable checks if a given request and response pair is cacheable according to the policy. It evaluates status codes, methods, headers, cookies, and rules.

Returns true if the request and response should be cached, otherwise false.

func (*Policy) TTL

func (p *Policy) TTL(resp *http.Response) time.Duration

TTL returns the time-to-live (TTL) for the given response according to the policy. If the policy is configured to use the Cache-Control header and the header contains a valid max-age directive, the TTL will be based on that value. Otherwise, the policy's default TTL will be used.

type Rule

type Rule struct {
	// URL is a URL to match against URLs.
	URL string

	// Pattern is a regular expression pattern to match against URLs.
	//
	// This field is ignored if URL is set.
	Pattern string

	// PatternFlag is a control flag for the regular expression pattern.
	//
	// URL is a slice of URLs to match against URLs.
	PatternFlag recache.Flag

	// Behavior is the caching behavior to apply for the matched URLs.
	Behavior Behavior
}

Rule defines a pattern for matching URLs, a caching behavior, and an optional map of custom headers to apply when the rule is matched.

func (*Rule) Match

func (r *Rule) Match(url string) bool

Match returns true if the URL matches the rule.