statigz

README

statigz

statigz serves pre-compressed embedded files with http in Go 1.16 and later.

Why?

Since version 1.16 Go provides standard way to embed static assets. This API has advantages over previous solutions:

• assets are processed during build, so there is no need for manual generation step,
• embedded data does not need to be kept in residential memory (as opposed to previous solutions that kept data in regular byte slices).

A common case for embedding is to serve static assets of a web application. In order to save bandwidth and improve latency, those assets are often served compressed. Compression concerns are out of embed responsibilities, yet they are quite important. Previous solutions (for example vfsgen with httpgzip) can optimize performance by storing compressed assets and serving them directly to capable user agents. This library implements such functionality for embedded file systems.

NOTE: Guarding new api (embed) with build tags is not a viable option, since it imposes issue in older versions of Go.

Example

package main

import (
	"embed"
	"log"
	"net/http"

	"github.com/vearutop/statigz"
	"github.com/vearutop/statigz/brotli"
)

// Declare your embedded assets.

//go:embed static/*
var st embed.FS

func main() {
	// Plug static assets handler to your server or router.
	err := http.ListenAndServe(":80", statigz.FileServer(st, brotli.AddEncoding))
	if err != nil {
		log.Fatal(err)
	}
}

Usage

Behavior is based on nginx gzip static module and github.com/lpar/gzipped.

Static assets have to be manually compressed with additional file extension, e.g. bundle.js would become bundle.js.gz (compressed with gzip) or index.html would become index.html.br (compressed with brotli).

NOTE: zopfli provides better compression than gzip while being backwards compatible with it.

Upon request server checks if there is a compressed file matching Accept-Encoding and serves it directly.

If user agent does not support available compressed data, server uses an uncompressed file if it is available ( e.g. bundle.js). If uncompressed file is not available, then server would decompress a compressed file into response.

Responses have ETag headers (64-bit FNV-1 hash of file contents) to enable caching. Responses that are not dynamically decompressed are served with http.ServeContent for ranges support.

Brotli support

Support for brotli is optional. Using brotli adds about 260 KB to binary size, that's why it is moved to a separate package.

NOTE: Although brotli has better compression than gzip and already has wide support in browsers, it has limitations for non-https servers, see this and this.

Runtime encoding

Recommended way of embedding assets is to compress assets before the build, so that binary includes *.gz or *.br files. This can be inconvenient in some cases, there is EncodeOnInit option to compress assets in runtime when creating file server. Once compressed, assets will be served directly without additional dynamic compression.

Files with extensions ".gz", ".br", ".gif", ".jpg", ".png", ".webp" are excluded from runtime encoding by default.

NOTE: Compressing assets in runtime can degrade startup performance and increase memory usage to prepare and store compressed data.

Documentation

Overview

Package statigz serves pre-compressed embedded files with http.

Index

• Variables
• func EncodeOnInit(server *Server)
• func OnError(onErr func(rw http.ResponseWriter, r *http.Request, err error)) func(server *Server)
• type Encoding
  • func GzipEncoding() Encoding
• type Server
  • func FileServer(fs fs.ReadDirFS, options ...func(server *Server)) *Server
  • func (s *Server) ServeHTTP(rw http.ResponseWriter, req *http.Request)

Examples

• FileServer

Variables

var SkipCompressionExt = []string{".gz", ".br", ".gif", ".jpg", ".png", ".webp"}

SkipCompressionExt lists file extensions of data that is already compressed.

Functions

func EncodeOnInit

func EncodeOnInit(server *Server)

EncodeOnInit enables runtime encoding for unencoded files to allow compression for uncompressed embedded files.

Enabling this option can degrade startup performance and memory usage in case of large embeddings, use with caution.

func OnError

func OnError(onErr func(rw http.ResponseWriter, r *http.Request, err error)) func(server *Server)

OnError is an option to customize error handling in Server.

Types

type Encoding

type Encoding struct {
	// FileExt is an extension of file with compressed content, for example ".gz".
	FileExt string

	// ContentEncoding is encoding name that is used in Accept-Encoding and Content-Encoding
	// headers, for example "gzip".
	ContentEncoding string

	// Decoder is a function that can decode data for an agent that does not accept encoding,
	// can be nil to disable dynamic decompression.
	Decoder func(r io.Reader) (io.Reader, error)

	// Encoder is a function that can encode data
	Encoder func(r io.Reader) ([]byte, error)
}

Encoding describes content encoding.

func GzipEncoding

func GzipEncoding() Encoding

GzipEncoding provides gzip Encoding.

type Server

type Server struct {
	// OnError controls error handling during Serve.
	OnError func(rw http.ResponseWriter, r *http.Request, err error)

	// Encodings contains supported encodings, default GzipEncoding.
	Encodings []Encoding

	// EncodeOnInit encodes files that does not have encoded version on Server init.
	// This allows embedding uncompressed files and still leverage one time compression
	// for multiple requests.
	// Enabling this option can degrade startup performance and memory usage in case
	// of large embeddings, use with caution.
	EncodeOnInit bool
	// contains filtered or unexported fields
}

Server is a http.Handler that directly serves compressed files from file system to capable agents.

Please use FileServer to create an instance of Server.

If agent does not accept encoding and uncompressed file is not available in file system, it would decompress the file before serving.

Compressed files should have an additional extension to indicate their encoding, for example "style.css.gz" or "bundle.js.br".

Caching is implemented with ETag and If-None-Match headers. Range requests are supported with help of http.ServeContent.

Behavior is similar to http://nginx.org/en/docs/http/ngx_http_gzip_static_module.html and https://github.com/lpar/gzipped, except compressed data can be decompressed for an incapable agent.

func FileServer

func FileServer(fs fs.ReadDirFS, options ...func(server *Server)) *Server

FileServer creates an instance of Server from file system.

Typically file system would be an embed.FS.

  //go:embed *.png *.br
	 var FS embed.FS

Brotli support is optionally available with brotli.AddEncoding.

Example

package main

import (
	"embed"
	"log"
	"net/http"

	"github.com/vearutop/statigz"
	"github.com/vearutop/statigz/brotli"
)

// Declare your embedded assets.

//go:embed _testdata/*
var st embed.FS

func main() {
	// Plug static assets handler to your server or router.
	err := http.ListenAndServe(":80", statigz.FileServer(st, brotli.AddEncoding))
	if err != nil {
		log.Fatal(err)
	}
}

func (*Server) ServeHTTP

func (s *Server) ServeHTTP(rw http.ResponseWriter, req *http.Request)

ServeHTTP serves static files.