httpcache

package module
v0.0.0-...-ef4fc8a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 11, 2024 License: MIT Imports: 20 Imported by: 0

README

Caddy Module: http.handlers.cache

Development repository of the caddy cache-handler module.
This is a distributed HTTP cache module for Caddy based on Souin cache.

Features

Minimal Configuration

Using the minimal configuration the responses will be cached for 120s

{
    order cache before rewrite
    cache
}

example.com {
    cache
    reverse_proxy your-app:8080
}

Global Option Syntax

Here are all the available options for the global options

{
    order cache before rewrite
    log {
        level debug
    }
    cache {
        allowed_http_verbs GET POST PATCH
        api {
            basepath /some-basepath
            prometheus
            souin {
                basepath /souin-changed-endpoint-path
            }
        }
        badger {
            path the_path_to_a_file.json
        }
        cache_keys {
            .*\.css {
                disable_body
                disable_host
                disable_method
                disable_query
                headers X-Token Authorization
                hide
            }
        }
        cache_name Another
        cdn {
            api_key XXXX
            dynamic
            email darkweak@protonmail.com
            hostname domain.com
            network your_network
            provider fastly
            strategy soft
            service_id 123456_id
            zone_id anywhere_zone
        }
        etcd {
            configuration {
                # Your etcd configuration here
            }
        }
        key {
            disable_body
            disable_host
            disable_method
            headers Content-Type Authorization
        }
        log_level debug
        mode bypass
        nuts {
            path /path/to/the/storage
        }
        olric {
            url url_to_your_cluster:3320
            path the_path_to_a_file.yaml
            configuration {
                # Your olric configuration here
            }
        }
        regex {
            exclude /test2.*
        }
        stale 200s
        ttl 1000s
        default_cache_control no-store
    }
}

:4443
respond "Hello World!"

Cache directive Syntax

Here are all the available options for the directive options

@match path /path

handle @match {
    cache {
        cache_name ChangeName
        cache_keys {
            (host1|host2).*\.css {
                disable_body
                disable_host
                disable_method
                disable_query
                headers X-Token Authorization
            }
        }
        cdn {
            api_key XXXX
            dynamic
            email darkweak@protonmail.com
            hostname domain.com
            network your_network
            provider fastly
            strategy soft
            service_id 123456_id
            zone_id anywhere_zone
        }
        key {
            disable_body
            disable_host
            disable_method
            disable_query
            headers Content-Type Authorization
        }
        log_level debug
        regex {
            exclude /test2.*
        }
        stale 200s
        ttl 1000s
        default_cache_control no-store
    }
}

Provider Syntax

Badger

The badger provider must have either the path or the configuration directive.

badger-path.com {
    cache {
        badger {
            path /tmp/badger/first-match
        }
    }
}

badger-configuration.com {
    cache {
        badger {
            configuration {
                # Required value
                ValueDir <string>

                # Optional
                SyncWrites <bool>
                NumVersionsToKeep <int>
                ReadOnly <bool>
                Compression <int>
                InMemory <bool>
                MetricsEnabled <bool>
                MemTableSize <int>
                BaseTableSize <int>
                BaseLevelSize <int>
                LevelSizeMultiplier <int>
                TableSizeMultiplier <int>
                MaxLevels <int>
                VLogPercentile <float>
                ValueThreshold <int>
                NumMemtables <int>
                BlockSize <int>
                BloomFalsePositive <float>
                BlockCacheSize <int>
                IndexCacheSize <int>
                NumLevelZeroTables <int>
                NumLevelZeroTablesStall <int>
                ValueLogFileSize <int>
                ValueLogMaxEntries <int>
                NumCompactors <int>
                CompactL0OnClose <bool>
                LmaxCompaction <bool>
                ZSTDCompressionLevel <int>
                VerifyValueChecksum <bool>
                EncryptionKey <string>
                EncryptionKeyRotationDuration <Duration>
                BypassLockGuard <bool>
                ChecksumVerificationMode <int>
                DetectConflicts <bool>
                NamespaceOffset <int>
            }
        }
    }
}
Etcd

The etcd provider must have the configuration directive.

etcd-configuration.com {
    cache {
        etcd {
            configuration {
                Endpoints etcd1:2379 etcd2:2379 etcd3:2379
                AutoSyncInterval 1s
                DialTimeout 1s
                DialKeepAliveTime 1s
                DialKeepAliveTimeout 1s
                MaxCallSendMsgSize 10000000
                MaxCallRecvMsgSize 10000000
                Username john
                Password doe
                RejectOldCluster false
                PermitWithoutStream false
            }
        }
    }
}
NutsDB

The nutsdb provider must have either the path or the configuration directive.

nuts-path.com {
    cache {
        nuts {
            path /tmp/nuts-path
        }
    }
}

nuts-configuration.com {
    cache {
        nuts {
            configuration {
                Dir /tmp/nuts-configuration
                EntryIdxMode 1
                RWMode 0
                SegmentSize 1024
                NodeNum 42
                SyncEnable true
                StartFileLoadingMode 1
            }
        }
    }
}
Olric

The olric provider must have either the url directive to work as client mode.

olric-url.com {
    cache {
        olric {
            url olric:3320
        }
    }
}

The olric provider must have either the path or the configuration directive to work as embedded mode.

olric-path.com {
    cache {
        olric {
            path /path/to/olricd.yml
        }
    }
}

olric-configuration.com {
    cache {
        nuts {
            configuration {
                Dir /tmp/nuts-configuration
                EntryIdxMode 1
                RWMode 0
                SegmentSize 1024
                NodeNum 42
                SyncEnable true
                StartFileLoadingMode 1
            }
        }
    }
}
Redis

The redis provider must have either the URL or the configuration directive.

redis-url.com {
    cache {
        redis {
            url 127.0.0.1:6379
        }
    }
}

redis-configuration.com {
    cache {
        redis {
            configuration {
                Network my-network
                Addr 127.0.0.1:6379
                Username user
                Password password
                DB 1
                MaxRetries 1
                MinRetryBackoff 5s
                MaxRetryBackoff 5s
                DialTimeout 5s
                ReadTimeout 5s
                WriteTimeout 5s
                PoolFIFO true
                PoolSize 99999
                PoolTimeout 10s
                MinIdleConns 100
                MaxIdleConns 100
                ConnMaxIdleTime 5s
                ConnMaxLifetime 5s
            }
        }
    }
}

What does these directives mean?

Key Description Value example
allowed_http_verbs The HTTP verbs allowed to be cached GET POST PATCH

(default: GET HEAD)
api The cache-handler API cache management
api.basepath BasePath for all APIs to avoid conflicts /your-non-conflict-route

(default: /souin-api)
api.prometheus Enable the Prometheus metrics
api.souin.basepath Souin API basepath /another-souin-api-route

(default: /souin)
badger Configure the Badger cache storage
badger.path Configure Badger with a file /anywhere/badger_configuration.json
badger.configuration Configure Badger directly in the Caddyfile or your JSON caddy configuration See the Badger configuration for the options
cache_name Override the cache name to use in the Cache-Status response header Another Caddy Cache-Handler Souin
cache_keys Define the key generation rules for each URI matching the key regexp
cache_keys.{your regexp} Regexp that the URI should match to override the key generation .+\.css
cache_keys.{your regexp} Regexp that the URI should match to override the key generation .+\.css
cache_keys.{your regexp}.disable_body Disable the body part in the key matching the regexp (GraphQL context) true

(default: false)
cache_keys.{your regexp}.disable_host Disable the host part in the key matching the regexp true

(default: false)
cache_keys.{your regexp}.disable_method Disable the method part in the key matching the regexp true

(default: false)
cache_keys.{your regexp}.disable_query Disable the query string part in the key matching the regexp true

(default: false)
cache_keys.{your regexp}.headers Add headers to the key matching the regexp Authorization Content-Type X-Additional-Header
cache_keys.{your regexp}.hide Prevent the key from being exposed in the Cache-Status HTTP response header true

(default: false)
cdn The CDN management, if you use any cdn to proxy your requests Souin will handle that
cdn.provider The provider placed before Souin akamai

fastly

souin
cdn.api_key The api key used to access to the provider XXXX
cdn.dynamic Enable the dynamic keys returned by your backend application (default: true)
cdn.email The api key used to access to the provider if required, depending the provider XXXX
cdn.hostname The hostname if required, depending the provider domain.com
cdn.network The network if required, depending the provider your_network
cdn.strategy The strategy to use to purge the cdn cache, soft will keep the content as a stale resource hard

(default: soft)
cdn.service_id The service id if required, depending the provider 123456_id
cdn.zone_id The zone id if required, depending the provider anywhere_zone
default_cache_control Set the default value of Cache-Control response header if not set by upstream (Souin treats empty Cache-Control as public if omitted) no-store
max_cachable_body_bytes Set the maximum size (in bytes) for a response body to be cached (unlimited if omited) 1048576 (1MB)
key Override the key generation with the ability to disable unecessary parts
key.disable_body Disable the body part in the key (GraphQL context) true

(default: false)
key.disable_host Disable the host part in the key true

(default: false)
key.disable_method Disable the method part in the key true

(default: false)
key.disable_query Disable the query string part in the key true

(default: false)
key.headers Add headers to the key matching the regexp Authorization Content-Type X-Additional-Header
key.hide Prevent the key from being exposed in the Cache-Status HTTP response header true

(default: false)
mode Bypass the RFC respect One of bypass bypass_request bypass_response strict (default strict)
nuts Configure the Nuts cache storage
nuts.path Set the Nuts file path storage /anywhere/nuts/storage
nuts.configuration Configure Nuts directly in the Caddyfile or your JSON caddy configuration See the Nuts configuration for the options
etcd Configure the Etcd cache storage
etcd.configuration Configure Etcd directly in the Caddyfile or your JSON caddy configuration See the Etcd configuration for the options
olric Configure the Olric cache storage
olric.path Configure Olric with a file /anywhere/olric_configuration.json
olric.configuration Configure Olric directly in the Caddyfile or your JSON caddy configuration See the Olric configuration for the options
redis Configure the Redis cache storage
redis.url Set the Redis url storage localhost:6379
redis.configuration Configure Redis directly in the Caddyfile or your JSON caddy configuration See the Nuts configuration for the options
regex.exclude The regex used to prevent paths being cached ^[A-z]+.*$
stale The stale duration 25m
timeout The timeout configuration
timeout.backend The timeout duration to consider the backend as unreachable 10s
timeout.cache The timeout duration to consider the cache provider as unreachable 10ms
ttl The TTL duration 120s
log_level The log level One of DEBUG, INFO, WARN, ERROR, DPANIC, PANIC, FATAL it's case insensitive

Other resources

You can find an example for the Caddyfile or the JSON file.
See the Souin configuration for the full configuration, and its associated Caddyfile

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Configuration 

type Configuration struct {
	// Default cache to fallback on when none are redefined.
	DefaultCache DefaultCache
	// API endpoints enablers.
	API configurationtypes.API
	// Cache keys configuration.
	CacheKeys configurationtypes.CacheKeys `json:"cache_keys"`
	// Override the ttl depending the cases.
	URLs map[string]configurationtypes.URL
	// Logger level, fallback on caddy's one when not redefined.
	LogLevel string
	// SurrogateKeys contains the surrogate keys to use with a predefined mapping
	SurrogateKeys map[string]configurationtypes.SurrogateKeys
	// contains filtered or unexported fields
}

Configuration holder

func (*Configuration) GetAPI 

func (c *Configuration) GetAPI() configurationtypes.API

GetAPI get the default cache

func (*Configuration) GetCacheKeys 

func (c *Configuration) GetCacheKeys() configurationtypes.CacheKeys

GetCacheKeys get the cache keys rules to override

func (*Configuration) GetDefaultCache 

func (c *Configuration) GetDefaultCache() configurationtypes.DefaultCacheInterface

GetDefaultCache get the default cache

func (*Configuration) GetLogLevel 

func (c *Configuration) GetLogLevel() string

GetLogLevel get the log level

func (*Configuration) GetLogger 

func (c *Configuration) GetLogger() *zap.Logger

GetLogger get the logger

func (*Configuration) GetSurrogateKeys 

func (c *Configuration) GetSurrogateKeys() map[string]configurationtypes.SurrogateKeys

GetSurrogateKeys get the surrogate keys list

func (*Configuration) GetUrls 

func (c *Configuration) GetUrls() map[string]configurationtypes.URL

GetUrls get the urls list in the configuration

func (*Configuration) GetYkeys 

func (c *Configuration) GetYkeys() map[string]configurationtypes.SurrogateKeys

GetYkeys get the ykeys list

func (*Configuration) SetLogger 

func (c *Configuration) SetLogger(l *zap.Logger)

SetLogger set the logger

type DefaultCache 

type DefaultCache struct {
	// Allowed HTTP verbs to be cached by the system.
	AllowedHTTPVerbs []string `json:"allowed_http_verbs"`
	// Badger provider configuration.
	Badger configurationtypes.CacheProvider `json:"badger"`
	// The cache name to use in the Cache-Status response header.
	CacheName string                 `json:"cache_name"`
	CDN       configurationtypes.CDN `json:"cdn"`
	// The default Cache-Control header value if none set by the upstream server.
	DefaultCacheControl string `json:"default_cache_control"`
	// The maximum body size (in bytes) to be stored into cache.
	MaxBodyBytes uint64 `json:"max_cachable_body_bytes"`
	// Redis provider configuration.
	Distributed bool `json:"distributed"`
	// Headers to add to the cache key if they are present.
	Headers []string `json:"headers"`
	// Configure the global key generation.
	Key configurationtypes.Key `json:"key"`
	// Mode defines if strict or bypass.
	Mode string `json:"mode"`
	// Olric provider configuration.
	Olric configurationtypes.CacheProvider `json:"olric"`
	// Redis provider configuration.
	Redis configurationtypes.CacheProvider `json:"redis"`
	// Etcd provider configuration.
	Etcd configurationtypes.CacheProvider `json:"etcd"`
	// NutsDB provider configuration.
	Nuts configurationtypes.CacheProvider `json:"nuts"`
	// Regex to exclude cache.
	Regex configurationtypes.Regex `json:"regex"`
	// Storage providers chaining and order.
	Storers []string `json:"storers"`
	// Time before cache or backend access timeout.
	Timeout configurationtypes.Timeout `json:"timeout"`
	// Time to live.
	TTL configurationtypes.Duration `json:"ttl"`
	// Stale time to live.
	Stale configurationtypes.Duration `json:"stale"`
}

DefaultCache the struct

func (*DefaultCache) GetAllowedHTTPVerbs 

func (d *DefaultCache) GetAllowedHTTPVerbs() []string

GetAllowedHTTPVerbs returns the allowed verbs to cache

func (*DefaultCache) GetBadger 

func (d *DefaultCache) GetBadger() configurationtypes.CacheProvider

GetBadger returns the Badger configuration

func (*DefaultCache) GetCDN 

func (d *DefaultCache) GetCDN() configurationtypes.CDN

GetCDN returns the CDN configuration

func (*DefaultCache) GetCacheName 

func (d *DefaultCache) GetCacheName() string

GetCacheName returns the cache name to use in the Cache-Status response header

func (*DefaultCache) GetDefaultCacheControl 

func (d *DefaultCache) GetDefaultCacheControl() string

GetDefaultCacheControl returns the configured default cache control value

func (*DefaultCache) GetDistributed 

func (d *DefaultCache) GetDistributed() bool

GetDistributed returns if it uses Olric or not as provider

func (*DefaultCache) GetEtcd 

func (d *DefaultCache) GetEtcd() configurationtypes.CacheProvider

GetEtcd returns etcd configuration

func (*DefaultCache) GetHeaders 

func (d *DefaultCache) GetHeaders() []string

GetHeaders returns the default headers that should be cached

func (*DefaultCache) GetKey 

func (d *DefaultCache) GetKey() configurationtypes.Key

GetKey returns the default Key generation strategy

func (*DefaultCache) GetMaxBodyBytes 

func (d *DefaultCache) GetMaxBodyBytes() uint64

GetMaxBodyBytes returns the maximum body size (in bytes) to be cached

func (*DefaultCache) GetMode 

func (d *DefaultCache) GetMode() string

GetMode returns mdoe configuration

func (*DefaultCache) GetNuts 

func (d *DefaultCache) GetNuts() configurationtypes.CacheProvider

GetNuts returns nuts configuration

func (*DefaultCache) GetOlric 

func (d *DefaultCache) GetOlric() configurationtypes.CacheProvider

GetOlric returns olric configuration

func (*DefaultCache) GetRedis 

func (d *DefaultCache) GetRedis() configurationtypes.CacheProvider

GetRedis returns redis configuration

func (*DefaultCache) GetRegex 

func (d *DefaultCache) GetRegex() configurationtypes.Regex

GetRegex returns the regex that shouldn't be cached

func (*DefaultCache) GetStale 

func (d *DefaultCache) GetStale() time.Duration

GetStale returns the stale duration

func (*DefaultCache) GetStorers 

func (d *DefaultCache) GetStorers() []string

GetStorers returns the chianed storers

func (*DefaultCache) GetTTL 

func (d *DefaultCache) GetTTL() time.Duration

GetTTL returns the default TTL

func (*DefaultCache) GetTimeout 

func (d *DefaultCache) GetTimeout() configurationtypes.Timeout

GetTimeout returns the backend and cache timeouts

type SouinApp 

type SouinApp struct {
	DefaultCache
	// The provider to use.
	Storers []types.Storer
	// Surrogate storage to support th econfiguration reload without surrogate-key data loss.
	SurrogateStorage providers.SurrogateInterface
	// Cache-key tweaking.
	CacheKeys configurationtypes.CacheKeys `json:"cache_keys,omitempty"`
	// API endpoints enablers.
	API configurationtypes.API `json:"api,omitempty"`
	// Logger level, fallback on caddy's one when not redefined.
	LogLevel string `json:"log_level,omitempty"`
}

SouinApp contains the whole Souin necessary items

func (SouinApp) CaddyModule 

func (s SouinApp) CaddyModule() caddy.ModuleInfo

CaddyModule implements caddy.ModuleInfo

func (SouinApp) Provision 

func (s SouinApp) Provision(_ caddy.Context) error

Provision implements caddy.Provisioner

func (SouinApp) Start 

func (s SouinApp) Start() error

Start will start the App

func (SouinApp) Stop 

func (s SouinApp) Stop() error

Stop will stop the App

type SouinCaddyMiddleware 

type SouinCaddyMiddleware struct {
	*middleware.SouinBaseHandler

	Configuration Configuration
	// Logger level, fallback on caddy's one when not redefined.
	LogLevel string `json:"log_level,omitempty"`
	// Allowed HTTP verbs to be cached by the system.
	AllowedHTTPVerbs []string `json:"allowed_http_verbs,omitempty"`
	// Headers to add to the cache key if they are present.
	Headers []string `json:"headers,omitempty"`
	// Configure the Badger cache storage.
	Badger configurationtypes.CacheProvider `json:"badger,omitempty"`
	// Configure the global key generation.
	Key configurationtypes.Key `json:"key,omitempty"`
	// Override the cache key generation matching the pattern.
	CacheKeys configurationtypes.CacheKeys `json:"cache_keys,omitempty"`
	// Configure the Badger cache storage.
	Nuts configurationtypes.CacheProvider `json:"nuts,omitempty"`
	// Enable the Etcd distributed cache storage.
	Etcd configurationtypes.CacheProvider `json:"etcd,omitempty"`
	// Enable the Redis distributed cache storage.
	Redis configurationtypes.CacheProvider `json:"redis,omitempty"`
	// Enable the Olric distributed cache storage.
	Olric configurationtypes.CacheProvider `json:"olric,omitempty"`
	// Time to live for a key, using time.duration.
	Timeout configurationtypes.Timeout `json:"timeout,omitempty"`
	// Time to live for a key, using time.duration.
	TTL configurationtypes.Duration `json:"ttl,omitempty"`
	// Time to live for a stale key, using time.duration.
	Stale configurationtypes.Duration `json:"stale,omitempty"`
	// Storage providers chaining and order.
	Storers []string `json:"storers,omitempty"`
	// The default Cache-Control header value if none set by the upstream server.
	DefaultCacheControl string `json:"default_cache_control,omitempty"`
	// The cache name to use in the Cache-Status response header.
	CacheName string `json:"cache_name,omitempty"`
	// contains filtered or unexported fields
}

SouinCaddyMiddleware development repository of the cache handler, allows the user to set up an HTTP cache system, RFC-7234 compliant and supports the tag based cache purge, distributed and not-distributed storage, key generation tweaking.

func (SouinCaddyMiddleware) CaddyModule 

func (SouinCaddyMiddleware) CaddyModule() caddy.ModuleInfo

CaddyModule returns the Caddy module information.

func (*SouinCaddyMiddleware) Cleanup 

func (s *SouinCaddyMiddleware) Cleanup() error

func (*SouinCaddyMiddleware) FromApp 

func (s *SouinCaddyMiddleware) FromApp(app *SouinApp) error

FromApp to initialize configuration from App structure.

func (*SouinCaddyMiddleware) Provision 

func (s *SouinCaddyMiddleware) Provision(ctx caddy.Context) error

Provision to do the provisioning part.

func (*SouinCaddyMiddleware) ServeHTTP 

func (s *SouinCaddyMiddleware) ServeHTTP(rw http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error

ServeHTTP implements caddyhttp.MiddlewareHandler.

func (*SouinCaddyMiddleware) UnmarshalCaddyfile 

func (s *SouinCaddyMiddleware) UnmarshalCaddyfile(h *caddyfile.Dispenser) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.