webcache

README

webcache

Reverse proxy as an HTTP cache and requests decorator

About

Webcache is a reverse proxy that caches http responses in order to provide them rapidly each time a same request is performed. Through its configuration file, webcache allows blocking hosts and methods (responding with 403: Forbidden or 405: Method not allowed respectively) and lets to specify new headers for all or some specific requests.

By default, Webcache uses a Redis server as a cache, and has its own configuration structure. However, all of this can be easily customized by just implementing the corresponding interfaces Config and Cache.

Configuration

By default, the server will expect to find any .yaml file in the /etc/webcache/ path, or any other defined by CONFIG_PATH environment variable. If no config file is found, or none of them follows the structure from the example down below, no request or method will be allowed by the webcache.

methods:
  - name: default   # default methods configuration for all endpoints listed in this file
    enabled: true   # since non listed methods are disabled by default, enable all them
    cached: true    # since cache is disabled by default, enable for all methods
    timeout: 1h     # for how long a cached response is valid (10 minutes by default)
    headers:        # custom headers for all the endpoints listed in this file
      X_GLOBAL_HEADER: global_header
      
  - name: DELETE
    enabled: false  # block all DELETE requests (405 - method not allowed)
  - name: POST
    enabled: false  # block all POST requests (405 - method not allowed)
  - name: PUT
    cached: false   # do not catch any PUT requests

router:
  - endpoints: # afected endpoints for the current configuration (regex)
      - https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,32}\/?$

    methods: # overwrite the default methods configuration for the endpoints above
      - name: default
        headers: # custom headers for the endpoints above
          X_A_CUSTOM_HEADER: header_value
          X_ANOTHER_HEADER: another_value
      - name: GET
        headers:
          X_JUST_IN_GET_HEADER: get_header_value

Environment variables

The application requires a set of environment variables that describes how the service must perform. These environment variables are those listed down below:

# Service endpoint and network
SERVICE_ADDR=:8000
SERVICE_NETWORK=tcp

# Configuration path 
CONFIG_PATH=.config/

# Redis datasource
REDIS_DSN=webcache-redis:6379

# In-memory cache configuration 
CACHE_SIZE=1024 # how many entries the local cache can have
CACHE_TTL=10m # for how long an entry is stored in the local cache

Documentation

Index

• Constants
• Variables
• func DigestRequest(rq *http.Request, headers []string) []byte
• func FormatHttpRequest(req *http.Request) (format string)
• type Cache
• type Config
• type ConfigFile
• type ConfigGroup
  • func NewConfigGroup(path string, logger *zap.Logger) (*ConfigGroup, error)
  • func (group *ConfigGroup) AddConfig(file *ConfigFile) (err error)
  • func (group *ConfigGroup) ReadDir(dir string) error
  • func (group *ConfigGroup) ReadFile(filepath string) error
  • func (group *ConfigGroup) RequestOptions(endpoint string, method string) (*Options, bool)
• type HttpResponse
  • func NewHttpResponse() *HttpResponse
  • func (r *HttpResponse) Echo(w http.ResponseWriter) (int, error)
  • func (r *HttpResponse) Empty() bool
  • func (r *HttpResponse) Format() (format string)
  • func (r *HttpResponse) Header() http.Header
  • func (r *HttpResponse) Write(b []byte) (int, error)
  • func (r *HttpResponse) WriteHeader(i int)
• type Method
• type Options
  • func NewOptions() *Options
• type RedisCache
  • func NewRedisCache(addr string, size int, ttl time.Duration) (*RedisCache, error)
  • func (c *RedisCache) Load(key string, value any) (err error)
  • func (c *RedisCache) Store(key string, value any, ttl time.Duration) error
• type ReverseProxy
  • func NewReverseProxy(manager Config, cache Cache, logger *zap.Logger) *ReverseProxy
  • func (reverse *ReverseProxy) ServeHTTP(w http.ResponseWriter, req *http.Request)
• type Router

Constants

const (
	DEFAULT_KEYWORD = "default"
	YAML_REGEX      = "^\\w*\\.(yaml|yml|YAML|YML)*$"
)

const (
	ETAG_SERVER_HEADER   = "ETag"
	HTTP_LOCATION_HEADER = "Location"
	HTTP_CODE_BOUNDARY   = 400
	HTTP_CODE_REDIRECT   = 300
)

Variables

var (
	ErrUnknown   = errors.New("E001")
	ErrNotFound  = errors.New("E002")
	ErrNoContent = errors.New("E003")
)

var (
	DefaultTimeout = 10 * time.Minute
)

Functions

func DigestRequest

func DigestRequest(rq *http.Request, headers []string) []byte

DigestRequest returns the md5 of the given request rq taking as input parameters the request's method, the exact host and path, all those listed query params and headers, and the body, if any

func FormatHttpRequest

func FormatHttpRequest(req *http.Request) (format string)

FormatHttpRequest returns and string descriving the content of an HttpRequest

Types

type Cache

type Cache interface {
	Store(string, interface{}, time.Duration) error
	Load(string, interface{}) error
}

A Cache represents an storage for request's responses

type Config

type Config interface {
	RequestOptions(endpoint, method string) (*Options, bool)
}

A Config represents a set of settings about how the ReverseProxy must perform

type ConfigFile

type ConfigFile struct {
	Methods []Method `yaml:"methods"`
	Router  []Router `yaml:"router"`
}

type ConfigGroup

type ConfigGroup struct {
	// contains filtered or unexported fields
}

ConfigGroup represents a set of settings to apply over http requests and responses

func NewConfigGroup

func NewConfigGroup(path string, logger *zap.Logger) (*ConfigGroup, error)

NewConfigGroup reads the content of the provided path and returns the declared configuration

func (*ConfigGroup) AddConfig

func (group *ConfigGroup) AddConfig(file *ConfigFile) (err error)

AddConfig registers the given configuration into the config group

func (*ConfigGroup) ReadDir

func (group *ConfigGroup) ReadDir(dir string) error

ReadDir applies all configuration files inside the given directory into the current configuration

func (*ConfigGroup) ReadFile

func (group *ConfigGroup) ReadFile(filepath string) error

ReadFile applies a configuration file into the current configuration

func (*ConfigGroup) RequestOptions

func (group *ConfigGroup) RequestOptions(endpoint string, method string) (*Options, bool)

RequestOptions returns the Options instance for the given endpoint and method

type HttpResponse

type HttpResponse struct {
	Body    []byte
	Headers http.Header
	Code    int
}

func NewHttpResponse

func NewHttpResponse() *HttpResponse

func (*HttpResponse) Echo

func (r *HttpResponse) Echo(w http.ResponseWriter) (int, error)

func (*HttpResponse) Empty

func (r *HttpResponse) Empty() bool

func (*HttpResponse) Format

func (r *HttpResponse) Format() (format string)

func (*HttpResponse) Header

func (r *HttpResponse) Header() http.Header

implement http.ResponseWriter https://golang.org/pkg/net/http/#ResponseWriter

func (*HttpResponse) Write

func (r *HttpResponse) Write(b []byte) (int, error)

func (*HttpResponse) WriteHeader

func (r *HttpResponse) WriteHeader(i int)

type Method

type Method struct {
	Name    string `yaml:"name"`
	Enabled *bool  `yaml:"enabled"`
	Cached  *bool  `yaml:"cached"`
	Timeout *string
	Headers map[string]string `yaml:"headers"`
}

type Options

type Options struct {
	// contains filtered or unexported fields
}

func NewOptions

func NewOptions() *Options

type RedisCache

type RedisCache struct {
	// contains filtered or unexported fields
}

func NewRedisCache

func NewRedisCache(addr string, size int, ttl time.Duration) (*RedisCache, error)

NewRedisCache returns an implementation of Cache for RedisCache

func (*RedisCache) Load

func (c *RedisCache) Load(key string, value any) (err error)

Load returns the value for a given key, if any, otherwise err != nil

func (*RedisCache) Store

func (c *RedisCache) Store(key string, value any, ttl time.Duration) error

Store stores a value under a given key with a lifetime of ttl duration so far

type ReverseProxy

type ReverseProxy struct {
	DigestRequest func(req *http.Request) (string, error)
	// contains filtered or unexported fields
}

A ReverseProxy is a cached reverse proxy that captures responses in order to provide it in the future instead of permorming the same request each time

func NewReverseProxy

func NewReverseProxy(manager Config, cache Cache, logger *zap.Logger) *ReverseProxy

NewReverseProxy returns a brand new ReverseProxy with the provided config and cache

func (*ReverseProxy) ServeHTTP

func (reverse *ReverseProxy) ServeHTTP(w http.ResponseWriter, req *http.Request)

ServeHTTP performs http requests if not cached yet or returns the chaced body instead

type Router

type Router struct {
	Endpoints []string `yaml:"endpoints"`
	Methods   []Method `yaml:"methods"`
}