cache

Documentation

Overview

Package cache provides HTTP caching interfaces and utilities.

Index

• func FetchURL(ctx context.Context, cache HTTPCache, client *http.Client, req *http.Request, ...) ([]byte, error)
• func FetchURLWithValidator(ctx context.Context, cache HTTPCache, client *http.Client, req *http.Request, ...) ([]byte, error)
• type BDCache
  • func New(ttl time.Duration) (*BDCache, error)
  • func NewWithPath(ttl time.Duration, cachePath string) (*BDCache, error)
  • func (c *BDCache) Close() error
  • func (c *BDCache) Get(ctx context.Context, url string) (data []byte, etag string, headers map[string]string, found bool)
  • func (c *BDCache) RecordHit()
  • func (c *BDCache) RecordMiss()
  • func (c *BDCache) SetAsync(ctx context.Context, url string, data []byte, etag string, ...) error
  • func (c *BDCache) SetAsyncWithTTL(ctx context.Context, url string, data []byte, etag string, ...) error
  • func (c *BDCache) Stats() Stats
• type CachedResponse
• type DomainRateLimiter
  • func NewDomainRateLimiter(minDelay time.Duration) *DomainRateLimiter
  • func (r *DomainRateLimiter) SetDomainDelay(domain string, delay time.Duration)
  • func (r *DomainRateLimiter) Wait(rawURL string)
• type HTTPCache
• type HTTPError
  • func (e *HTTPError) Error() string
• type ResponseValidator
• type Stats
  • func (s Stats) HitRate() float64

Functions

func FetchURL

func FetchURL(ctx context.Context, cache HTTPCache, client *http.Client, req *http.Request, logger *slog.Logger) ([]byte, error)

FetchURL fetches a URL with caching support. If cache is non-nil and contains the URL, returns cached data. Otherwise, executes the HTTP request, caches successful responses (HTTP 200), and returns the body. Returns an error if the HTTP status is not 200 OK. The caller must set all necessary headers on the request before calling this function.

func FetchURLWithValidator

func FetchURLWithValidator(
	ctx context.Context,
	cache HTTPCache,
	client *http.Client,
	req *http.Request,
	logger *slog.Logger,
	validator ResponseValidator,
) ([]byte, error)

FetchURLWithValidator fetches a URL with caching support and optional response validation. If validator is provided and returns false, the response is NOT cached but still returned. This is useful for avoiding caching of incomplete/shell responses.

Types

type BDCache

type BDCache struct {
	// contains filtered or unexported fields
}

BDCache wraps bdcache to implement the HTTPCache interface.

func New

func New(ttl time.Duration) (*BDCache, error)

New creates a new BDCache with disk persistence. The cache directory defaults to ~/.cache/sociopath. ttl is the default time-to-live for cached entries. Use NewWithPath to specify a custom cache directory.

func NewWithPath

func NewWithPath(ttl time.Duration, cachePath string) (*BDCache, error)

NewWithPath creates a new BDCache with disk persistence at the specified path.

func (*BDCache) Close

func (c *BDCache) Close() error

Close flushes and closes the cache.

func (*BDCache) Get

func (c *BDCache) Get(ctx context.Context, url string) (data []byte, etag string, headers map[string]string, found bool)

Get retrieves a cached response by URL.

func (*BDCache) RecordHit

func (c *BDCache) RecordHit()

RecordHit increments the cache hit counter.

func (*BDCache) RecordMiss

func (c *BDCache) RecordMiss()

RecordMiss increments the cache miss counter.

func (*BDCache) SetAsync

func (c *BDCache) SetAsync(ctx context.Context, url string, data []byte, etag string, headers map[string]string) error

SetAsync stores a response in the cache asynchronously.

func (*BDCache) SetAsyncWithTTL

func (c *BDCache) SetAsyncWithTTL(ctx context.Context, url string, data []byte, etag string, headers map[string]string, ttl time.Duration) error

SetAsyncWithTTL stores a response in the cache asynchronously with a custom TTL.

func (*BDCache) Stats

func (c *BDCache) Stats() Stats

Stats returns the current cache statistics.

type CachedResponse

type CachedResponse struct {
	Data    []byte
	Headers map[string]string
	ETag    string
}

CachedResponse holds HTTP response data.

type DomainRateLimiter

type DomainRateLimiter struct {
	// contains filtered or unexported fields
}

DomainRateLimiter enforces a minimum delay between requests to the same domain. It is safe for concurrent use from multiple goroutines.

func NewDomainRateLimiter

func NewDomainRateLimiter(minDelay time.Duration) *DomainRateLimiter

NewDomainRateLimiter creates a rate limiter that enforces minDelay between requests to the same domain. Domain-specific overrides can be set with SetDomainDelay.

func (*DomainRateLimiter) SetDomainDelay

func (r *DomainRateLimiter) SetDomainDelay(domain string, delay time.Duration)

SetDomainDelay sets a custom minimum delay for a specific domain. This overrides the default minDelay for requests to this domain.

func (*DomainRateLimiter) Wait

func (r *DomainRateLimiter) Wait(rawURL string)

Wait blocks until it's safe to make a request to the given URL's domain. It ensures at least minDelay has passed since the last request to that domain.

type HTTPCache

type HTTPCache interface {
	Get(ctx context.Context, url string) (data []byte, etag string, headers map[string]string, found bool)
	SetAsync(ctx context.Context, url string, data []byte, etag string, headers map[string]string) error
	SetAsyncWithTTL(ctx context.Context, url string, data []byte, etag string, headers map[string]string, ttl time.Duration) error
	RecordHit()
	RecordMiss()
	Stats() Stats
}

HTTPCache defines the interface for caching HTTP responses.

type HTTPError

type HTTPError struct {
	URL        string
	StatusCode int
}

HTTPError represents an HTTP error response.

func (*HTTPError) Error

func (e *HTTPError) Error() string

type ResponseValidator

type ResponseValidator func(body []byte) bool

ResponseValidator is a function that validates a response body. Returns true if the response should be cached, false otherwise.

type Stats

type Stats struct {
	Hits   int64
	Misses int64
}

Stats holds cache hit/miss statistics.

func (Stats) HitRate

func (s Stats) HitRate() float64

HitRate returns the cache hit rate as a percentage (0-100).