stuffbin

package module
Version: v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 14, 2020 License: MIT Imports: 11 Imported by: 8

README

stuffbin

stuffbin is a utility + package to compress and embed static files and assets into Go binaries for distribution. It supports falling back to the local file system when no embedded assets are available, for instance, in development mode. stuffbin is inspired by zgok but is much cleaner and leaner.

stuffbin

How does it work?

stuffbin compresses and embeds arbitrary files to the end of Go binaries. This does not affect the normal execution of the binary by the operating system as it is aware of its original size. The compressed data that is appended beyond its original size is simply ignored. When a stuffed application is executed, stuffbin reads the compressed bytes from self (the executable), uncompresses the files on the fly into an in-memory filesystem and provides a simple FileSystem interface to access them. This enables complex Go applications that have external file dependencies to be shipped a single fat binary, commonly, web applications that have static file and template dependencies.

  • Built in ZIP compression
  • A virtual filesystem abstraction to access embedded files
  • Add static assets from nested directories recursively
  • Re-path files and whole directories with the :suffix format, eg: ../my/original/file.txt:/my/virtual/file.txt and /my/nested/dir:/virtual/dir
  • Template parsing helper similar to template.ParseGlob() to parse templates from the virtual filesystem
  • Launch an http.FileServer for serving static files
  • Gracefully failover to the local file system in the absence of embedded assets
  • CLI to stuff, unstuff and extract, and list stuffed files in binaries

Installation

go get -u github.com/knadh/stuffbin/...

Usage

Stuffing and embedding
# -a, -in, and -out params followed by the paths of files to embed.
# To normalize paths, aliases can be suffixed with a colon.
stuffbin -a stuff -in /path/to/exe -out /path/to/new.exe \
    static/file1.css static/file2.pdf /somewhere/else/file3.txt:/static/file3.txt
List files in a stuffed binary
stuffbin -a id -in /path/to/new/exe
Extract stuffed files from a binary
stuffbin -a unstuff -in /path/to/new/exe -out assets.zip

In the application

To test this, cd into ./mock and run go run mock.go

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/knadh/stuffbin"
)

func main() {
	// Get self executable path.
	path, err := os.Executable()
	if err != nil {
		log.Fatalf("error getting executable path: %v", err)
	}
	// Read stuffed data from self.
	fs, err := stuffbin.UnStuff(path)
	if err != nil {
		// Binary is unstuffed or is running in dev mode.
		// Can halt here or fall back to the local filesystem.
		if err == stuffbin.ErrNoID {
			// First argument is to the root to mount the files in the FileSystem
			// and the rest of the arguments are paths to embed.
			fs, err = stuffbin.NewLocalFS("/",
				"./", "bar.txt:/virtual/path/bar.txt")
			if err != nil {
				log.Fatalf("error falling back to local filesystem: %v", err)
			}
		} else {
			log.Fatalf("error reading stuffed binary: %v", err)
		}
	}

	fmt.Println("loaded files", fs.List())
	// Read the file 'foo'.
	f, err := fs.Get("foo.txt")
	if err != nil {
		log.Fatalf("error reading foo.txt: %v", err)
	}
	log.Println("foo.txt =", string(f.ReadBytes()))

	// Read the file 'bar'.
	f, err = fs.Get("/virtual/path/bar.txt")
	if err != nil {
		log.Fatalf("error reading /virtual/path/bar.txt: %v", err)
	}
	log.Println("/virtual/path/bar.txt =", string(f.ReadBytes()))

	fmt.Println("stuffed files:")
	for _, f := range fs.List() {
		fmt.Println("\t", f)
	}

	// Compile templates with the helpers:
	// err, tpl := stuffbin.ParseTemplatesGlob(nil, fs, "/templates/*.html")
	//
	// Template func map.
	// mp := map[string]interface{}{
	// 	"Foo": func() string {
	// 		return "func"
	// 	},
	// }
	// err, tpl := stuffbin.ParseTemplates(mp, fs, "/templates/index.html", "/templates/hello.html")

	// Expose an HTTP file server.
	// Try http://localhost:8000/static/foo.txt
	// Try http://localhost:8000/static/virtual/path/bar.txt
	// Try http://localhost:8000/static/subdir/baz.txt
	http.Handle("/static/", http.StripPrefix("/static/", fs.FileServer()))
	log.Println("listening on :8000")
	http.ListenAndServe(":8000", nil)
}
License

Licensed under the MIT License.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrNoID = errors.New("no ID found in the file")

ErrNoID is used to indicate if an ID was found in a file or not.

View Source
var ErrNotSupported = errors.New("this method is not supported")

ErrNotSupported indicates interface methods that are implemented but not supported.

Functions

func GetStuff

func GetStuff(in string) ([]byte, error)

GetStuff takes the path to a stuffed binary and extracts the packed data.

func MergeFS

func MergeFS(dest FileSystem, src FileSystem) error

MergeFS merges FileSystem b into a, overwriting conflicting paths.

func ParseTemplates

func ParseTemplates(f template.FuncMap, fs FileSystem, path ...string) (*template.Template, error)

ParseTemplates takes a file system, a list of file paths, and parses them into a template.Template.

func ParseTemplatesGlob

func ParseTemplatesGlob(f template.FuncMap, fs FileSystem, pattern string) (*template.Template, error)

ParseTemplatesGlob takes a file system, a file path pattern, and parses matching files into a template.Template with an optional template.FuncMap that will be applied to the compiled templates.

func Stuff

func Stuff(in, out, rootPath string, files ...string) (int64, int64, error)

Stuff takes the path to a binary, a list of file paths to stuff, and compresses the files and appends them to the end of the binary's body and writes everything to a new binary.

Types

type File

type File struct {
	// contains filtered or unexported fields
}

File represents an abstraction over http.File.

func NewFile

func NewFile(path string, info os.FileInfo, b []byte) *File

NewFile creates and returns a new instance of File.

func (*File) Close

func (f *File) Close() error

Close emulates http.File's Close but internally, it simply seeks the File's reader to 0.

func (*File) Path

func (f *File) Path() string

Path returns the path of the file.

func (*File) Read

func (f *File) Read(b []byte) (int, error)

Read reads the file contents.

func (*File) ReadBytes

func (f *File) ReadBytes() []byte

ReadBytes returns the bytes of the given file.

func (*File) Readdir

func (f *File) Readdir(count int) ([]os.FileInfo, error)

Readdir is a dud.

func (*File) Seek

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek seeks the given offset in the file.

func (*File) Stat

func (f *File) Stat() (os.FileInfo, error)

Stat returns the file's os.FileInfo.

type FileSystem

type FileSystem interface {
	Add(f *File) error
	List() []string
	Len() int
	Size() int64
	Get(path string) (*File, error)
	Glob(pattern string) ([]string, error)
	Read(path string) ([]byte, error)
	Open(path string) (http.File, error)
	Delete(path string) error
	Merge(f FileSystem) error
	FileServer() http.Handler
}

FileSystem represents a simple filesystem abstraction that implements the http.fileSystem interface.

func NewFS

func NewFS() (FileSystem, error)

NewFS returns a new instance of FileSystem.

func NewLocalFS

func NewLocalFS(rootPath string, paths ...string) (FileSystem, error)

NewLocalFS returns a new instance of FileSystem with the given list of local files and directories mapped to it.

func UnStuff

func UnStuff(path string) (FileSystem, error)

UnStuff takes the path to a stuffed binary, unstuffs it, and returns a FileSystem.

func UnZip

func UnZip(b []byte) (FileSystem, error)

UnZip unzips zipped bytes and returns a FileSystem with the files mapped to it.

type ID

type ID struct {
	Name    [8]byte
	BinSize uint64
	ZipSize uint64
}

ID represents an identifier that is appended to binaries for identifying stuffbin binaries. The fields are appended as bytes totalling 8 + 12 + 8 + 8 = 36 bytes in the order Name BinSize ZipSize.

func GetFileID

func GetFileID(fName string) (ID, error)

GetFileID attempts to get the stuffbin identifier from the end of the file and returns the identifier name and file sizes.

type WalkFunc

type WalkFunc func(srcPath, targetPath string, fInfo os.FileInfo) error

WalkFunc is an abstraction over filepath.WalkFunc that's used as a callback to receive the real file path and their corresponding target (alias) paths from a real filepath.Walk() traversal of a list of file and directory paths.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto