forge

README

forge

Forge is a library and CLI for running reusable steps from various proprietary CI systems using a pluggable container runtime. This, for example, makes the functionality provided to GitHub Actions easily consumable (or testable) by users of other CI systems.

Forge currently exposes running GitHub Actions (e.g. actions/setup-go), Concourse Resources (e.g. concourse/git-resource) and local Azure DevOps Tasks (e.g. Npm@1).

install

From a release.

Using brew:

brew install frantjc/tap/forge

From source:

git clone https://github.com/frantjc/forge
cd forge
make

Using go:

go install github.com/frantjc/forge/cmd/forge

In GitHub Actions:

  - uses: frantjc/forge@v0

usage

GitHub Actions

For GitHub Actions, Forge will try to source the GitHub Actions variables from the working directory's Git configuration as well as GitHub's default environment variables.

forge use actions/setup-go@v3 -w go-version=1.22

Forge mounts the current working directory to the Action's GITHUB_WORKSPACE as well as cache directories respecting the XDG Base Directory Specification to the Action's RUNNER_TOOLCACHE and RUNNER_TEMP.

That is to say, after running the above command, go should be installed to XDG_CACHE_HOME/forge/runner/toolcache.

You can also use local GitHub Actions by starting the reference with "/" or "." to signify that it is an absolute or relative local filepath, respectively.

forge use ./testdata/actions/docker

For additional debugging, you can attach to the container running the Action:

forge use -a ./testdata/actions/dockerfile

If the Action runs using a custom image, that image must have bash or sh on its PATH for the attach to work.

Local Actions cannot refer to files outside of the action metadata file's directory.

Concourse Resources

For Concourse Resources, Forge will source resource_types and resources from the working directory's .forge.yml (overridable with -c). This schema is conveniently compatible with Concourse's pipeline schema.

Just like Concourse itself, Forge ships with some Resource Types builtin that can be overridden.

forge get mock -v version=v0.0.0

You can also attach to the container executing the Resource to snoop around:

forge get -a mock -v version=v0.0.0

The Resource's image must have bash or sh on its PATH for the attach to work.

Setup Forge

Skip install and get a Concourse Resource using a pre-installed forge:

  - uses: frantjc/forge@v0
    with:
      install: false
      get: my-resource

Install and put a Concourse Resource with the given params and config:

  - uses: frantjc/forge@v0
    with:
      put: my-resource
      params: |
        my-param=my-value
      config: forge.yaml

Azure DevOps Tasks

For Azure DevOps, you can execute local Tasks by starting the reference with "/" or "." to signify that it is an absolute or relative local filepath, respectively. Remote Tasks are not supported at this time as there is no standard protocol for referencing them.

forge task ./testdata/tasks/node

For additional debugging, you can attach to the container running the Task:

forge task -a ./testdata/tasks/node

If the Task runs using a custom image, that image must have bash or sh on its PATH for the attach to work.

why?

Automation begins with a shell script that executes a bunch of CLI commands often to test, build and publish some code. The next step is to set up some continuous integration (CI) system that executes that script in response to some event such as a commit to a Git repository's main branch. Such CI systems tend to identify that all of the scripts that they are executing do a lot of the same things--checkout a Git repository, setup a tool and so on.

In an effort to make their platform easier to use and to refactor the shared functionality out of all of the aforementioned scripts, CI systems in the past have introduced reusable "plugins"/"Actions"/"Resources"/"Tasks"/"Orbs" which take minimal configuration to do a complex task. GitHub Actions' actions/checkout, for example, takes one short line of code to invoke and accepts a bunch of optional configuration to fulfill many related use cases.

Unfortunately, using such powerful plugins outside of the the system they were built for can be wildly difficult. This makes debugging the use of these plugins require long feedback loops. It also makes migrating from one CI system to another treacherous, having to replace uses of one system's plugins with another's.

Forge aims to remedy this.

developing

• git is required
• make is required
• go 1.20 is required for multi-error handling
• docker is required to test as it is its only runtime
• upx is required for compressing shim
• node 20 is required for developing the action

Documentation

Index

• Variables
• func SemVer() string
• func WithLogger(ctx context.Context, logger Logger) context.Context
• type Container
• type ContainerConfig
• type ContainerRuntime
• type Drains
  • func StdDrains() *Drains
  • func (d *Drains) GoString() string
  • func (d *Drains) ToStreams(in io.Reader) *Streams
• type Foundry
  • func NewFoundry(containerRuntime ContainerRuntime) *Foundry
  • func (f *Foundry) GoString() string
  • func (f *Foundry) Process(ctx context.Context, ore Ore, drains *Drains) error
• type Image
• type Logger
  • func LoggerFrom(ctx context.Context) Logger
  • func NewLogger() Logger
• type Mount
• type Ore
• type Streams
  • func StdStreams() *Streams
  • func StdTerminalStreams() (*Streams, func() error)
  • func TerminalStreams(stdin io.Reader, stdout, stderr io.Writer) (*Streams, func() error, error)
• type Volume

Variables

var DefaultDetachKeys = "ctrl-d"

DefaultDetachKeys are the default key combinations to use when detaching from a Container that has been attached to.

var VersionCore = "0.14.0"

VersionCore is the SemVer version core of forge. Meant to be be overridden at build time, but kept up-to-date sometimes to best support `go install`.

Functions

func SemVer

func SemVer() string

SemVer returns the semantic version of forge as built from VersionCore and debug build info.

func WithLogger

func WithLogger(ctx context.Context, logger Logger) context.Context

WithLogger returns a Context from the parent Context with the given Logger inside of it.

Types

type Container

type Container interface {
	GetID() string

	CopyTo(context.Context, string, io.Reader) error
	CopyFrom(context.Context, string) (io.ReadCloser, error)

	Run(context.Context, *Streams) (int, error)
	Start(context.Context) error
	Restart(context.Context) error
	Exec(context.Context, *ContainerConfig, *Streams) (int, error)

	Stop(context.Context) error
	Remove(context.Context) error
	Kill(context.Context) error
}

Container represents a container created by a ContainerRuntime.

type ContainerConfig

type ContainerConfig struct {
	Entrypoint []string `json:"entrypoint,omitempty"`
	Cmd        []string `json:"cmd,omitempty"`
	WorkingDir string   `json:"working_dir,omitempty"`
	Env        []string `json:"env,omitempty"`
	User       string   `json:"user,omitempty"`
	Privileged bool     `json:"privileged,omitempty"`
	Mounts     []Mount  `json:"mounts,omitempty"`
}

ContainerConfig is the configuration that is used to create a container or an exec in a running container.

type ContainerRuntime

type ContainerRuntime interface {
	GetContainer(context.Context, string) (Container, error)
	CreateContainer(context.Context, Image, *ContainerConfig) (Container, error)
	PullImage(context.Context, string) (Image, error)
	CreateVolume(context.Context, string) (Volume, error)
	Close() error
}

ContainerRuntime represents the functionality needed by Ores to pull OCI images and run containers when being processed.

type Drains

type Drains struct {
	Out, Err io.Writer
	Tty      bool
}

Drains represents only outward streams from an Ore, namely stdout and stderr.

func StdDrains

func StdDrains() *Drains

StdDrains returns a Drains draining to os.Stdout and os.Stderr.

func (*Drains) GoString

func (d *Drains) GoString() string

GoString implements fmt.GoStringer.

func (*Drains) ToStreams

func (d *Drains) ToStreams(in io.Reader) *Streams

ToStreams turns a Drains to a Streams for use by Ores to pass to a Container.

type Foundry

type Foundry struct {
	ContainerRuntime
}

Foundry is a wrapper around a ContainerRuntime for processing Ores.

func NewFoundry

func NewFoundry(containerRuntime ContainerRuntime) *Foundry

NewFoundry returns a Foundry.

func (*Foundry) GoString

func (f *Foundry) GoString() string

GoString implements fmt.GoStringer.

func (*Foundry) Process

func (f *Foundry) Process(ctx context.Context, ore Ore, drains *Drains) error

Process Liquifies the Ore and returns the resulting Metal.

type Image

type Image interface {
	Manifest() (*imagespecsv1.Manifest, error)
	Config() (*imagespecsv1.ImageConfig, error)
	Digest() (digest.Digest, error)
	Blob() io.Reader
	Name() string
}

Image represents a image pulled by a ContainerRuntime. Used to create Containers from.

type Logger

type Logger = logr.Logger

Logger is an alias to logr.Logger in case the logging library is desired to be swapped out.

func LoggerFrom

func LoggerFrom(ctx context.Context) Logger

LoggerFrom returns a Logger embedded within the given Context or a no-op Logger if no such Logger exists.

func NewLogger

func NewLogger() Logger

NewLogger creates a new Logger.

type Mount

type Mount struct {
	Source      string `json:"source,omitempty"`
	Destination string `json:"destination,omitempty"`
}

type Ore

type Ore interface {
	Liquify(context.Context, ContainerRuntime, *Drains) error
}

Ore represents one or more sequential containerized commands. Ores are meant to represent the entire input to said commands, so that if two Ore's match, their results should be the same. Because of this, Ores can be cached.

type Streams

type Streams struct {
	*Drains
	In         io.Reader
	DetachKeys string
}

Streams represents streams to and from a process inside of a Container.

func StdStreams

func StdStreams() *Streams

StdStreams returns a Streams consisting of os.Stdin, os.Stdout and os.Stderr.

func StdTerminalStreams

func StdTerminalStreams() (*Streams, func() error)

StdTerminalStreams creates a Streams with os.Stdin, os.Stdout and os.Stderr made raw and a restore function to return them to their previous state. For use with attaching to a shell inside of a Container.

func TerminalStreams

func TerminalStreams(stdin io.Reader, stdout, stderr io.Writer) (*Streams, func() error, error)

TerminalStreams creates a Streams with each of the given streams that is a terminal made raw and a restore function to return them to their previous states. For use with attaching to a shell inside of a Container.

type Volume

type Volume interface {
	GetID() string
	Remove(context.Context) error
}

Volume represents a volume created by a ContainerRuntime which can be attached to a Container via its ContainerConfig.Mounts.