resources

README

Heads up!
This library is in maintenance mode. Please use the official embed library going forward.
https://pkg.go.dev/embed

Resources

Unfancy resources embedding with Go.

• No blings.
• No runtime dependency.
• Idiomatic Library First design.

Dude, Why?

Yes, there is quite a lot of projects that handles resource embedding but they come with more bling than you ever need and you often end up with having dependencies for your end project. Not this time.

Installing

Just go get it!

$ go get github.com/omeid/go-resources/cmd/resources

Usage

$ resources -h
Usage resources:
  -declare
        whether to declare the -var (default false)
  -fmt
        run output through gofmt, this is slow for huge files (default false)
  -output filename
        filename to write the output to
  -package name
        name of the package to generate (default "main")
  -tag tag
        tag to use for the generated package (default no tag)
  -trim prefix
        path prefix to remove from the resulting file path in the virtual filesystem
  -var name
        name of the variable to assign the virtual filesystem to (default "FS")
  -width number
        number of content bytes per line in generetated file (default 12)

Optimization

Generating resources result in a very high number of lines of code, 1MB of resources result about 5MB of code at over 87,000 lines of code. This is caused by the chosen representation of the file contents within the generated file.

Instead of a (binary) string, resources transforms each file into an actual byte slice. For example, a file with content Hello, world! will be represented as follows:

FS = &FileSystem{
  "/hello.txt": File{
    data: []byte{
      0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x2c, 0x20, 0x57, 0x6f, 0x72, 0x6c, 0x64,
      0x21,
    },
    fi: FileInfo{
      name:    "hello.txt",
      size:    13,
      modTime: time.Unix(0, 1504640959536230658),
      isDir:   false,
    },
  },
}

While this seems wasteful, the compiled binary is not really affected. If you add 1MB of resources, your binary will increase 1MB as well.

However, compiling this many lines of code takes time and slows down the compiler. To avoid recompiling the resources every time and leverage the compiler cache, generate your resources into a standalone package and then import it, this will allow for faster iteration as you don't have to wait for the resources to be compiled with every change.

mkdir -p assets
resources -declare -var=FS -package=assets -output=assets/assets.go your/files/here

package main

import "importpath/to/assets"

func main() {
  data, err := assets.FS.Open("your/files/here")
  // ...
}

"Live" development of resources

For fast iteration and improvement of your resources, you can work around the compile with the following technique:

First, create a normal main.go:

package main

import "net/http"

var Assets http.FileSystem

func main() {
  if Assets == nil {
    panic("No Assets. Have you generated the resources?")
  }

  // use Assets here
}

Then, add a second file in the same package (main here), with the following content:

// +build !embed

package main

import (
	"net/http"

	"github.com/omeid/go-resources/live"
)

var Assets = live.Dir("./public")

Now when you build or run your project, you will have files directly served from ./public directory.

To create a production build, i.e. one with the embedded files, build the resouces with -tag=embed and add the embed tag to go build:

$ resources -output=public_resources.go -var=Assets -tag=embed public/*
$ go build -tags=embed

Now your resources should be embedded with your program! Of course, you may use any var or tag name you please.

Go Generate

There is a few reasons to avoid resource embedding in go generate.

First go generate is for generating Go source code from your code, generally the resources you want to embed aren't effected by the Go source directly and as such generating resources are slightly out of the scope of go generate.

Second, you're unnecessarily slowing down code iterations by blocking go generate for resource generation.

Resources, The Library

The resource generator is written as a library and isn't bound to filesystem by the way of accepting files in the form

type File interface {
      io.Reader
      Stat() (os.FileInfo, error)
}

along with a helper method that adds files from filesystem.

This allows to integrate resources with ease in your workflow when the when the provided command doesn't fit well, for an example see the Gonzo binding resources.

Please refer to the GoDoc for complete documentation.

Strings

The generated FileSystem also implements an String(string) (string, bool) method that allows you to read the content of a file as string, to use that instead of defining your file Assets variable as simply an http.FileSystem, do the following:

type Resources interface {
	http.FileSystem
	String(string) (string, bool)
}

var Assets Resources

Now you can call Assets.String(someFile) and get the content as string with a boolean value indicating whatever the file was found or not.

---

Contributing

Please consider opening an issue first, or just send a pull request. :)

Credits

See Contributors.

LICENSE

MIT.

Documentation

Overview

Package resources provides unfancy resources embedding with Go.

Index

• Variables
• type Config
• type File
• type Package
  • func New() *Package
  • func (p *Package) Add(path string, file File) error
  • func (p *Package) AddFile(path string, file string) error
  • func (p *Package) AddFiles(trimPath string, path string) (int, error)
  • func (p *Package) Build(out io.Writer) error
  • func (p *Package) Write(path string) error

Variables

var (

	// BlockWidth allows to adjust the number of bytes per line in the generated file
	BlockWidth = 12
)

Types

type Config

type Config struct {
	Pkg     string // Pkg holds the package name
	Var     string // Var holds the variable name for the virtual filesystem
	Tag     string // Tag may hold an optional build tag, unless empty
	Declare bool   // Declare controls if the Var should be declared as well
	Format  bool   // Format controls, whether gofmt should be applied to the output
}

Config defines some details about the output file

type File

type File interface {
	io.Reader
	Stat() (os.FileInfo, error)
}

File mimicks the os.File and http.File interface.

type Package

type Package struct {
	Config
	Files map[string]File
}

A Package describes a collection of files and how they should be tranformed to an output.

func New

func New() *Package

New creates a new Package.

func (*Package) Add

func (p *Package) Add(path string, file File) error

Add a file to the package at the give path.

func (*Package) AddFile

func (p *Package) AddFile(path string, file string) error

AddFile is a helper function that adds a file into the package.

func (*Package) AddFiles

func (p *Package) AddFiles(trimPath string, path string) (int, error)

AddFiles is a helper function that recursively adds files to the package.

func (*Package) Build

func (p *Package) Build(out io.Writer) error

Build compiles the package and writes it into an io.Writer.

func (*Package) Write

func (p *Package) Write(path string) error

Write builds the package (via Build) and writes the output the the file given by the path argument.