README

ember

Build Status Go Report Card GoDoc GoDev

Ember is a lightweight library and tool for embedding arbitrary resources into a go executable at runtime. The resources don't need to exist at compile time.

Embedding binary files (eg. zip-archives and executables) is supported.

Use case

Applications often require runtime- or user-defined configuration and resources to be stored alongside the executable.
This forces the end-user to deal with multiple files when copying, moving or distributing the application. It also allows end users to manipulate those attachments, which is not always desirable.

The main use-case of ember is to bundle such configuration files and other resources with the application at runtime. There is no need for setting up a go toolchain to (re-)build the application every time there is a new configuration.

Cross platform

Ember is truly cross-platform. It supports any OS, and embedding resources can also be done cross-platform.
This means that files can be attached to windows executables on both windows and linux and vice-versa.

Usage

Ember consists of two parts.

  1. The ember package is imported by the application that receives attachments.
  2. The ember/embedding package is used by a separate application that attaches files to the already-compiled target executable.
    This package can be used as a library. Alternatively there exists a CLI tool at ember/cmd/embedder.

Example

The following example can also be found at examples/list.

Access embedded files from within the target application
package main

import (
	"fmt"
	"io"
	"log"
	"os"

	"github.com/maja42/ember"
)

func main() {
	attachments, err := ember.Open()
	if err != nil {
		log.Fatal(err)
	}
	defer attachments.Close()

	fmt.Printf("Executable contains %d attachments\n", attachments.Count())
	contents := attachments.List()

	for _, name := range contents {
		s := attachments.Size(name)
		fmt.Printf("\nAttachment %q has %d bytes:\n", name, s)
		
		r := attachments.Reader(name)
		io.Copy(os.Stdout, r)
		fmt.Println()
	}
}
Embed files into a target executable

To embed files into a compiled go executable you can use the CLI tool at cmd/embedder. Alternatively, you can also integrate embedding-logic into your own application by importing ember/embedding (see the GoDoc for more information).

To use cmd/embedder, first create an attachments.json file describing the files to embed:

{
  "file A": "path/to/file A.txt",
  "file B": "path/to/file B.txt"
}

Afterwards, attach the files to an already-built executable:

cd cmd/embedder
go build
./embedder -attachments ./attachments.json -exe ./myApp -out ./myFinishedApp

How does it work?

ember uses a very primitive approach for embedding data to support any platform and to be independent of the go version, compiler, linker and so on.

When embedding, the executable file is modified by simply appending additional data at the end. To detect the boundary between the original executable and the attachments, a special marker-string (magic string) is inserted in-between.

   +---------------+
   |               |
   |    original   |
   |   executable  |
   |               |
   +---------------+
   | marker-string |
   +---------------+
   |      TOC      |
   +---------------+
   | marker-string |
   +---------------+
   |     file1     | 
   +---------------+
   |     file2     |
   +---------------+
   |     file3     |
   +---------------+

When starting the application and opening the attachments, the executable file is opened and searched for that specific marker string.

The first blob appended to the executable is a TOC (table of contents) that lists all files, their size and byte-offset. This allows iterating and reading the individual attachments without seeking through the whole executable. It also compares sizes and offsets to ensure that the executable is consistent and complete.

All content afterwards is the attached data.

ember also performs a variety of security-checks to ensure that the produced executable will work correctly:

  • Check if the application imported maja42/ember in a compatible version
  • Ensure that the executable does not already contain attachments

This approach also allows the use of exe-packers (compressors) and code signing.

Contributions

Feel free to submit feature requests, bug reports and pull requests.

Expand ▾ Collapse ▴

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AttErr

type AttErr string

    AttErr reports problems with embedded attachments.

    func (*AttErr) Error

    func (o *AttErr) Error() string

    type Attachments

    type Attachments struct {
    	// contains filtered or unexported fields
    }

      Attachments represent embedded data in an executable.

      func Open

      func Open() (*Attachments, error)

        Open returns the attachments of the running executable.

        func OpenExe

        func OpenExe(exePath string) (*Attachments, error)

          OpenExe returns the attachments of an arbitrary executable.

          func (*Attachments) Close

          func (a *Attachments) Close() error

            Close the executable containing the attachments. Close will return an error if it has already been called.

            func (*Attachments) Count

            func (a *Attachments) Count() int

              Count returns the number of attachments.

              func (*Attachments) List

              func (a *Attachments) List() []string

                List returns a list containing the names of all attachments.

                func (*Attachments) Reader

                func (a *Attachments) Reader(name string) Reader

                  Reader returns a reader for a given attachment. Returns nil if no attachment with that name exists.

                  func (*Attachments) Size

                  func (a *Attachments) Size(name string) int64

                    Size returns the size of a specific attachment in bytes. Returns zero if no attachment with that name exists.

                    type Reader

                    type Reader interface {
                    	io.ReadSeeker
                    	io.ReaderAt
                    	Size() int64
                    }

                      Reader groups basic methods available on attachments.

                      Directories

                      Path Synopsis
                      cmd
                      examples