tiap

README

tiap isn't app publisher

tiap is a small Go module and CLI tool to easily create Industrial Edge .app files (packages) for continuous delivery. It does nothing more than pulling the required container images based on your app's composer project and finally bundling all up in an .app package. The .app file then can be imported by users into their IEM systems.

• simple to automatically download and use within your pipeline:

  # >>> consider pinning tiap to a specific release version <<<
  go run github.com/thediveo/tiap/cmd/tiap@latest \
    -o hellorld.app hellorldapp/
  

• defaults to using git describe to set the app version, or set explicitly using --app-version $SEMVER. Even accepts v prefixed semvers and then drops the prefix.

• talks to the Docker API socket, so there's no need to either reconfigure the Docker daemon in your dev system or in pipelines, or to fiddle around with socat to reroute a localhost TCP port to the Docker socker.

  However, in view of supporting IE apps for different (CPU) architectures we recommend to never package image files from the local daemon, but instead to only pull from a (remote) registry. For this, we recommend using --pull-always.

  go run github.com/thediveo/tiap/cmd/tiap@latest \
    -o hellorld.app --pull-always hellorldapp/
  

• no need to deal with stateful IE app publisher workspaces.

• small footprint.

Please note that tiap doesn't lint the Docker composer project, except for:

• rejecting :latest image references (yes, we're more strict than IE App Publisher here for reasons that still hurt),
• enforcing mem_limit service configuration (as this seems to be the most common stumbling block in a survey of one sample).

CLI

The command

tiap -h

outputs

tiap isn't app publisher, but packages Industrial Edge .app files anyway

Usage:
  tiap -o FILE [flags] APP-TEMPLATE-DIR

Flags:
      --app-version string     app semantic version, defaults to git describe
  -h, --help                   help for tiap
  -H, --host string            Docker daemon socket to connect to (only if non-default and using local images)
  -o, --out string             mandatory: name of app package file to write
  -p, --platform string        platform to build app for (default "linux/amd64")
      --pull-always            always pull image from remote registry, never use local images
      --release-notes string   release notes (interpreted as double-quoted Go string literal; use \n, \", …)
  -v, --version                version for tiap

Hellorld Demo

This packages a hellorld.app: when deployed, it runs an HTTP server in a busybox container, serving the (in)famous "Hellorld!" greeting, but in text only.

# while in the toplevel directory of this repository...
go run github.com/thediveo/tiap/cmd/tiap@latest \
    -o hellorld.app --pull-always testdata/app/

outputs

INFO[0000] 🗩  tiap ... isn't app publisher              
INFO[0000]    commit a56f7926 (modified)                
INFO[0000] ⚖  Apache 2.0 License                        
INFO[0000] 🏗  creating temporary project copy in "/tmp/tiap-project-1164041835" 
INFO[0000] 🫙  app repository detected as "hellorld"     
INFO[0000] 📛  semver: "v0.9.2-1-ga56f792" -> app ID: "t0CVuAlaAjzZLuoSqkI8WrblwEwUoqn1" 
INFO[0000] 🚚  pulling images and writing composer project... 
INFO[0000]    🛎  service "hellorld" wants 🖼  image "busybox:stable" 
INFO[0000]    🖭  written 5101568 bytes of 🖼  image with ID 8135583d97fe 
INFO[0000] 🌯  wrapping up...                            
INFO[0000]    🧮  determining package files SHA256 digests... 
INFO[0000]       🧮  digest(ed) detail.json: 0e684f06b98e4d68df942f410cfee52e7e03929b9ce1fca5e38e381d300e9442 
INFO[0000]       🧮  digest(ed) hellorld/appicon.png: 77911f21764738f4c4b717f7bd0371cf752128754c7f0c30d181ee5ffd6adb27 
INFO[0000]       🧮  digest(ed) hellorld/docker-compose.yml: 150cf94801f1132e34d3410358ff93860bf77eab2080c41d33ef8091901bc803 
INFO[0000]       🧮  digest(ed) hellorld/images/8135583d97feb82398909c9c97607159e6db2c4ca2c885c0b8f590ee0f9fe90d.tar: 12f0bd2e0d70f176bbcd64dc44abbb7f44b77e3abf29a327baf8c9c00a40bd55 
INFO[0000]       🧮  digest(ed) hellorld/nginx/nginx.json: 0af30ab022bed4328a143fd74eb754c8829adf62b567f1b2d9d825084d10c554 
INFO[0000]    📦  packaging detail.json                  
INFO[0000]    📦  packaging digests.json                 
INFO[0000]    📦  packaging hellorld/appicon.png         
INFO[0000]    📦  packaging hellorld/docker-compose.yml  
INFO[0000]    📦  packaging hellorld/images/8135583d97feb82398909c9c97607159e6db2c4ca2c885c0b8f590ee0f9fe90d.tar 
INFO[0000]    📦  packaging hellorld/nginx/nginx.json    
INFO[0000] ✅  ...IE app package "hellorld.app" successfully created 
INFO[0000] 🧹  removed temporary folder "/tmp/tiap-project-1164041835" 

App Template

The recommended way to set up your app "template" structure to be used by tiap for packaging is to simply design your app project in IE App Publisher once and then immediately export it. Then unpack the .app file (it's a plain tar after all). Delete the images directory and digests.json. The rest should be checked into your git repository.

See also testdata/app for our canonical "Hellorld!" example.

The sweet size for app icons seem to be 150×150 pixels and they must be in PNG format.

App Architecture/Platform

In order to package an .app file for an architecture other than amd64 (cough x86-64 cough) use the --platform (or -p) flag. Its value can be a proper OCI platform specification, such as linux/amd64, or just an architecture specification like arm64. Aliases like x86-64 are understood and automatically normalized.

When packaging IE app files for multiple architectures we recommend – following Docker and OCI best practises – to only build multi-arch images and push them into a (sometimes private) registry. tiap will automatically pull the correct layers based on the platform setting.

Please note that tiap will default to the architecture tiap itself runs on, unless explicitly told otherwise using --platform!

Also, please note that the Industrial Edge platform requires the same app for multiple architectures to be fully separate apps: the "appId" in detail.json as well as the "app repository" (please see also the Creating a new Edge App documentation) thus must differ (the latter, for instance, hellorld and hellorld-arm64).

App Release Notes

The --release-note option interprets its value as a double-quoted Go string. To add newlines, use the \n escape or alternatively pass a literal newline (consult your shell on how to pass literal newlines). Double quotes must be always escaped as \".

However, be careful that your shell isn't messing around with your escaping on its own.

Copyright and License

Copyright 2023 Harald Albrecht, licensed under the Apache License, Version 2.0.

Documentation

Overview

Package tiap implements a simplistic Industrial Edge .app file packager.

“tiap isn't app publisher.” In can be used instead of the Siemens Industrial Edge App Publisher and iectl tools that are either interactive or an un-wget-able CLI tool. In contrast, tiap is easily “go install”-able, including version pinning. Moreover, tiap doesn't need setting up clean workspaces, et cetera.

All that tiap needs: a “template” folder with the usual app project folder structure. This can be easily gotten by exporting a (new) project once and purging out the image files and the digest.json file. The structure thus is as follows:

• detail.json (but leave the versionNumber and versionId fields empty)
• $REPO/
• $REPO/appicon.png (150⨉150 pixels)
• $REPO/docker-compose.yml (or .yaml)
• $REPO/nginx (where necessary)
• $REPO/nginx/nginx.json

Here, $REPO is an almost arbitrary directory name (except for “images”) that is considered to be the app's “repository” name.

Please note that tiap doesn't lint the Docker composer project, except for:

• rejecting “:latest” image references (yes, we're more strict than IE App Publisher here for a reason),
• enforcing “mem_limit” service configuration.

Index

• Constants
• Variables
• func FileDigests(root string) (map[string]string, error)
• func SaveImageToFile(ctx context.Context, imageref string, platform string, savedir string, ...) (filename string, err error)
• func WriteDigests(w io.Writer, root string) error
• type App
  • func NewApp(source string) (a *App, err error)
  • func (a *App) Done()
  • func (a *App) Package(out string) error
  • func (a *App) PullAndWriteCompose(ctx context.Context, platform string, optclient daemon.Client) error
  • func (a *App) SetDetails(semver string, releasenotes string, iearch string) error
• type ComposerProject
  • func LoadComposerProject(dir string) (*ComposerProject, error)
  • func NewComposerProject(path string) (*ComposerProject, error)
  • func (p *ComposerProject) Images() (ServiceImages, error)
  • func (p *ComposerProject) PullImages(ctx context.Context, serviceimgs ServiceImages, platform string, root string, ...) error
  • func (p *ComposerProject) Save(w io.Writer) error
• type ServiceImages

Constants

const DefaultIEAppArch = "x86-64"

DefaultIEAppArch is the denormalized platform architecture name of the default "unnamed" architecture.

Variables

var DefaultRegistry = name.DefaultRegistry

DefaultRegistry points to the Docker registry.

Functions

func FileDigests

func FileDigests(root string) (map[string]string, error)

FileDigests calculates the SHA256 digests of files inside the “root” directory and its subdirectories, and returns them as a map of filenames to SHA256 hex strings. The SHA256 hex strings do not contain a “sha256:” digist scheme prefix.

Please note that symbolic links are ignored.

func SaveImageToFile

func SaveImageToFile(ctx context.Context,
	imageref string,
	platform string,
	savedir string,
	optclient daemon.Client,
) (filename string, err error)

SaveImageToFile checks if the referenced image (“imageref”) is either available locally for the specific platform or otherwise attempts to pull it, and then immediately saves it to local storage in the specified directory “savedir”. The name of the image file will be the image reference's SHA256. SaveImageToFile either reports success or a more specific error.

Please note that an attempt to find the referenced image with the local daemon is only made when a non-nil client has been passed in. Otherwise, always a pull is attempted only.

func WriteDigests

func WriteDigests(w io.Writer, root string) error

WriteDigests determines the file digests inside the “root” directory and its sub directories and then writes the results to the specified io.Writer in “digests.json” format.

Types

type App

type App struct {
	// contains filtered or unexported fields
}

App represents an IE App (project) to be packaged.

func NewApp

func NewApp(source string) (a *App, err error)

NewApp returns an IE App object initialized from the specified “template” path.

func (*App) Done

func (a *App) Done()

Done removes all temporary work files.

func (*App) Package

func (a *App) Package(out string) error

Package (finally) packages the IE app project in a IE app package tar file indicated by “out”.

func (*App) PullAndWriteCompose

func (a *App) PullAndWriteCompose(
	ctx context.Context,
	platform string,
	optclient daemon.Client,
) error

PullAndWriteCompose analyzes the project's compose deployment in order to pull the required container images, then saves the images into the temporary stage, and writes composer project.

func (*App) SetDetails

func (a *App) SetDetails(semver string, releasenotes string, iearch string) error

SetDetails sets the semver (“versionNumber”, oh well) of this release, notes (if any) and optional architecture, and then writes a new “detail.json” into the build directory. This automatically sets the versionId to some suitable value behind the scenes. At least we think that it might be a suitable versionId value.

type ComposerProject

type ComposerProject struct {
	// contains filtered or unexported fields
}

ComposerProject represents a loaded Docker composer project.

func LoadComposerProject

func LoadComposerProject(dir string) (*ComposerProject, error)

LoadComposerProject looks in the specified “dir” for a Docker composer project file and loads it. This takes the several official variations of composer project file names into account. However, contrary to Docker's composer, it doesn't look into parent directories for project files and it doesn't take overrides into account.

func NewComposerProject

func NewComposerProject(path string) (*ComposerProject, error)

NewComposerProject reads the specified YAML file containing a (Docker) composer project and returns a ComposerProject object for it.

func (*ComposerProject) Images

func (p *ComposerProject) Images() (ServiceImages, error)

Images returns the mapping between services defined in this composer project and the container images they reference.

func (*ComposerProject) PullImages

func (p *ComposerProject) PullImages(
	ctx context.Context,
	serviceimgs ServiceImages,
	platform string,
	root string,
	optclient daemon.Client,
) error

PullImages takes a service-to-image reference mapping and pulls and saves the required container images. The caller is responsible to supply the correct "root" directory path inside which to place the images in a “image/” subdirectory. That is, the root path needs to reference the arbitrarily named “repository” folder.

func (*ComposerProject) Save

func (p *ComposerProject) Save(w io.Writer) error

Save writes the loaded composer project to the specified io.Writer, returning an error in case of failure.

type ServiceImages

type ServiceImages map[string]string

ServiceImages maps service names in Docker composer projects to their image references.