binclude

README

New Projects should use the official embed package instead, which was added in go 1.16.

binclude

binclude is a tool for including static files into Go binaries.

• focuses on ease of use
• the bincluded files add no more than the filesize to the binary
• uses go/ast for typesafe parsing
• each package can have its own binclude.FileSystem
• binclude.FileSystem implements the http.FileSystem interface
• ioutil like functions FileSystem.ReadFile, FileSystem.ReadDir
• include all files/ directories under a given path by calling binclude.Include("./path")
• include files based on a glob pattern binclude.IncludeGlob("./path/*.txt")
• add file paths from a textfile binclude.IncludeFromFile("includefile.txt")
• high test coverage
• supports execution of executables directly from a binclude.FileSystem via binexec (os/exec wrapper)
• optional compression of files with gzip binclude -gzip
• debug mode to read files from disk binclude.Debug = true

Install

GO111MODULE=on go get -u github.com/lu4p/binclude/cmd/binclude

Usage

package main

//go:generate binclude

import (
	"io/ioutil"
	"log"
	"path/filepath"

	"github.com/lu4p/binclude"
)

var assetPath = binclude.Include("./assets") // include ./assets with all files and subdirectories

func main() {
	binclude.Include("file.txt") // include file.txt

	// like os.Open
	f, err := BinFS.Open("file.txt")
	if err != nil {
		log.Fatalln(err)
	}

	out, err := ioutil.ReadAll(f)
	if err != nil {
		log.Fatalln(err)
	}

	log.Println(string(out))

	// like ioutil.Readfile
	content, err := BinFS.ReadFile(filepath.Join(assetPath, "asset1.txt"))
	if err != nil {
		log.Fatalln(err)
	}

	log.Println(string(content))

	// like ioutil.ReadDir
	infos, err := BinFS.ReadDir(assetPath)
	if err != nil {
		log.Fatalln(err)
	}

	for _, info := range infos {
		log.Println(info.Name())
	}
}

To build use:

go generate
go build

A more detailed example can be found here.

Binary size

The resulting binary, with the included files can get quite large.

You can reduce the final binary size by building without debug info (go build -ldflags "-s -w") and compressing the resulting binary with upx (upx binname).

Note: If you don't need to access the compressed form of the files I would advise to just use upx and don't add seperate compression to the files.

You can add compression to the included files with -gzip

Note: decompression is optional to allow for the scenario where you want to serve compressed files for a webapp directly.

OS / Arch Specific Includes

binclude supports including files/binaries only on specific architectures and operating systems. binclude follows the same pattern as Go's implicit Build Constraints. It will generate files for the specific platforms like binclude_windows.go which contains all windows specific files.

If a file's name, matches any of the following patterns:

*_GOOS.go
*_GOARCH.go
*_GOOS_GOARCH.go

binclude will consider all files included by binclude.Include in this file as files which should only be included on a specific GOOS and/ or GOARCH.

For example, if you want to include a binary only on Windows you could have a file static_windows.go and reference the static file:

package main

import "github.com/lu4p/binclude"

func bar() {
  binclude.Include("./windows-file.dll")
}

OS / Arch Specific Includes are used in the binexec example.

Note: explicit build tags like // +build debug are not supported for including files conditionally.

Advanced Usage

The generator can also be included into your package to allow for the code generator to run after all module dependencies are installed. Without installing the binclude generator to the PATH seperatly.

main.go:

// +build !gen

//go:generate go run -tags=gen .
package main

import (
	"github.com/lu4p/binclude"
)

func main() {
	binclude.Include("./assets")
}

main_gen.go:

// +build gen

package main

import (
	"github.com/lu4p/binclude"
	"github.com/lu4p/binclude/bincludegen"
)

func main() {
	bincludegen.Generate(binclude.None, ".")
	// binclude.None == no compression 
	// binclude.Gzip == gzip compression
}

To build use:

go generate
go build

Contributing

If you find a feature missing, which you consider a useful addition please open an issue.

If you plan on implementing a new feature please open an issue so we can discuss it first (I don't want you to waste your time).

Pull requests are welcome, and will be timely reviewed and merged, please provide tests for your code, if you don't know how to test your code open a pull request and I will suggest the right testing method.

Documentation

Index

• Variables
• func Include(name string) string
• func IncludeFromFile(name string)
• func IncludeGlob(pattern string) string
• type Compression
• type File
  • func (f *File) Close() error
  • func (f *File) Name() string
  • func (f *File) Read(p []byte) (n int, err error)
  • func (f *File) Readdir(count int) (infos []os.FileInfo, err error)
  • func (f *File) Seek(offset int64, whence int) (int64, error)
  • func (f *File) Size() int64
  • func (f *File) Stat() (os.FileInfo, error)
• type FileInfo
  • func (info *FileInfo) IsDir() bool
  • func (info *FileInfo) ModTime() time.Time
  • func (info *FileInfo) Mode() os.FileMode
  • func (info *FileInfo) Name() string
  • func (info *FileInfo) Size() int64
  • func (info *FileInfo) Sys() interface{}
• type FileSystem
  • func (fs *FileSystem) Compress(algo Compression) error
  • func (fs *FileSystem) CopyFile(bincludePath, hostPath string) error
  • func (fs *FileSystem) Decompress() (err error)
  • func (fs *FileSystem) Open(name string) (http.File, error)
  • func (fs *FileSystem) ReadDir(dirname string) ([]os.FileInfo, error)
  • func (fs *FileSystem) ReadFile(filename string) ([]byte, error)
  • func (fs *FileSystem) Stat(name string) (os.FileInfo, error)
• type Files

Examples

• FileSystem.CopyFile
• FileSystem.Open
• FileSystem.ReadDir
• FileSystem.ReadFile
• IncludeGlob

Variables

var Debug = false

Debug if set to true files are read via os.Open() and the bincluded files are ignored, use when developing.

Functions

func Include

func Include(name string) string

Include this file/ directory (including subdirectories) relative to the package path (noop) The path is walked via filepath.Walk and all files found are included This function returns the name to make it usable in global variable definitions.

func IncludeFromFile

func IncludeFromFile(name string)

IncludeFromFile like include but reads paths from a textfile. Paths are separated by a newline (noop)

func IncludeGlob

func IncludeGlob(pattern string) string

IncludeGlob include all files matching the given pattern same syntax as filepath.Glob This function returns an empty string to make it usable in global variable definitions.

Example

package main

import (
	"fmt"
	"io/ioutil"

	"github.com/lu4p/binclude"
	"github.com/lu4p/binclude/example"
)

var BinFS = example.BinFS

func main() {
	binclude.IncludeGlob("./assets/*.txt")
	f, _ := BinFS.Open("./assets/asset1.txt")
	data, _ := ioutil.ReadAll(f)
	fmt.Println(string(data))
}

Output:

asset1

Types

type Compression

type Compression int

Compression the compression algorithm to use

const (
	// None dont compress
	None Compression = iota
	// Gzip use gzip compression
	Gzip
)

type File

type File struct {
	Filename string
	Mode     os.FileMode
	ModTime  time.Time
	Content  []byte
	Compression
	// contains filtered or unexported fields
}

File implements the http.File interface

func (*File) Close

func (f *File) Close() error

Close closes the File, rendering it unusable for I/O.

func (*File) Name

func (f *File) Name() string

Name returns the name of the file as presented to Open.

func (*File) Read

func (f *File) Read(p []byte) (n int, err error)

Read implements the io.Reader interface.

func (*File) Readdir

func (f *File) Readdir(count int) (infos []os.FileInfo, err error)

Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.

func (*File) Seek

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface.

func (*File) Size

func (f *File) Size() int64

Size returns the original length of the underlying byte slice. Size is the number of bytes available for reading via ReadAt. The returned value is always the same and is not affected by calls to any other method.

func (*File) Stat

func (f *File) Stat() (os.FileInfo, error)

Stat returns the FileInfo structure describing file. Error is always nil

type FileInfo

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo implements the os.FileInfo interface.

func (*FileInfo) IsDir

func (info *FileInfo) IsDir() bool

IsDir abbreviation for Mode().IsDir()

func (*FileInfo) ModTime

func (info *FileInfo) ModTime() time.Time

ModTime returns the modification time (returns current time)

func (*FileInfo) Mode

func (info *FileInfo) Mode() os.FileMode

Mode returns the file mode bits

func (*FileInfo) Name

func (info *FileInfo) Name() string

Name returns the base name of the file

func (*FileInfo) Size

func (info *FileInfo) Size() int64

Size returns the length in bytes

func (*FileInfo) Sys

func (info *FileInfo) Sys() interface{}

Sys underlying data source (returns nil)

type FileSystem

type FileSystem struct {
	Files
	sync.RWMutex
}

FileSystem implements access to a collection of named files.

func (*FileSystem) Compress

func (fs *FileSystem) Compress(algo Compression) error

Compress turns a FileSystem without compressed files into a filesystem with compressed files

func (*FileSystem) CopyFile

func (fs *FileSystem) CopyFile(bincludePath, hostPath string) error

CopyFile copies a specific file from a binclude FileSystem to the hosts FileSystem. Permissions are copied from the included file.

Example

package main

import (
	"fmt"
	"io/ioutil"

	"github.com/lu4p/binclude/example"
)

var BinFS = example.BinFS

func main() {
	BinFS.CopyFile("./assets/asset1.txt", "asset1.txt")

	c, _ := ioutil.ReadFile("asset1.txt")

	fmt.Println(string(c))

}

Output:

asset1

func (*FileSystem) Decompress

func (fs *FileSystem) Decompress() (err error)

Decompress turns a FileSystem with compressed files into a filesystem without compressed files

func (*FileSystem) Open

func (fs *FileSystem) Open(name string) (http.File, error)

Open returns a File using the File interface

Example

package main

import (
	"fmt"
	"io/ioutil"

	"github.com/lu4p/binclude"
	"github.com/lu4p/binclude/example"
)

var BinFS = example.BinFS

func main() {
	binclude.Include("./assets")
	f, _ := BinFS.Open("./assets/asset1.txt")
	data, _ := ioutil.ReadAll(f)
	fmt.Println(string(data))
}

Output:

asset1

func (*FileSystem) ReadDir

func (fs *FileSystem) ReadDir(dirname string) ([]os.FileInfo, error)

ReadDir reads the directory named by dirname and returns a list of directory entries sorted by filename.

Example

package main

import (
	"fmt"

	"github.com/lu4p/binclude"
	"github.com/lu4p/binclude/example"
)

var BinFS = example.BinFS

func main() {
	binclude.Include("./assets")
	infos, _ := BinFS.ReadDir("./assets")
	for _, info := range infos {
		fmt.Println(info.Name())
	}
}

Output:

asset1.txt
asset2.txt
logo_nocompress.png
subdir

func (*FileSystem) ReadFile

func (fs *FileSystem) ReadFile(filename string) ([]byte, error)

ReadFile reads the file named by filename and returns the contents. A successful call returns err == nil, not err == EOF. Because ReadFile reads the whole file, it does not treat an EOF from Read as an error to be reported.

Example

package main

import (
	"fmt"

	"github.com/lu4p/binclude"
	"github.com/lu4p/binclude/example"
)

var BinFS = example.BinFS

func main() {
	binclude.Include("file.txt")
	data, _ := BinFS.ReadFile("file.txt")
	fmt.Println(string(data))
}

Output:

file.txt

func (*FileSystem) Stat

func (fs *FileSystem) Stat(name string) (os.FileInfo, error)

Stat returns a FileInfo describing the named file. If there is an error, it will be of type *PathError.

type Files

type Files map[string]*File

Files a map from the filepath to the files