gate

package module
Version: v0.0.0-...-9ccd474 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2022 License: BSD-3-Clause Imports: 0 Imported by: 0

README

Gate

Run untrusted code from anonymous sources. Instead of sending messages composed of passive data, send programs which can react to their environment. Migrate or duplicate running applications across hosts and computer architectures. See Introduction to Gate.

Foundations

WebAssembly is the interchange format of the user programs. However, the APIs are different from the browsers' usual WebAssembly environments. See low-level C API or the higher-level Rust crate for details.

The sandboxing and containerization features of the Linux kernel provide layers of security in addition to WebAssembly. See Security for details.

Gate services are akin to syscalls, but they work differently. New services can be added easily, and available services are discovered at run time. See Service implementation for details.

Building blocks

Gate appears as Go packages and programs. The execution mechanism is implemented in C++ and assembly. It is highly Linux-dependent. x86-64 and ARM64 are supported.

Important Go packages:

  • wag: The WebAssembly compiler (implemented in a separate repository).

  • gate/runtime: Core functionality. Interface to the execution mechanism.

  • gate/image: Low-level executable building and instance management.

  • gate/build: High-level executable building and snapshot restoration.

  • gate/server/web: HTTP server component which executes your code on purpose. It has a RESTful API, but some actions can be invoked also via websocket.

  • gate/service: Service implementation support and built-in services.

See the complete list of Go packages.

Programs:

  • gate: Command-line client for local daemon and remote servers. Uses SSH keys (Ed25519) for authentication.

  • gate-daemon: D-Bus daemon for running and managing instances and wasm modules locally.

  • gate-server: Standalone web server which can serve the public or require authentication.

  • gate-runtime: For optionally preconfiguring the execution environment for daemon/server, e.g. as a system service.

The available services are determined by what is built into the gate-daemon and gate-server programs. The versions provided by this Go module include only the services implemented in this repository. See extension about bundling additional services.

Objectives

While code is data, most of the time data cannot be treated as code for safety reasons. Change that at the Internet level. Data encapsulated in code can describe and transform itself.

Application portability. Migrate processes between mobile devices and servers when circumstances change: user presence, resource availability or demand, continuity etc.

Overhead needs to be low enough so that the system can be practical. Low startup latency for request processing. Low memory overhead for high density of continually running programs.

Work in progress

  • Linux x86-64 host support
  • Android host support (#33)
  • Support for WebAssembly version 1
  • Planned security measures have been implemented
  • HTTP server for running programs
  • Client can communicate with the program it runs on the server
  • Speculative execution security issue mitigations
  • Pluggable authentication
  • Load programs from IPFS
  • Reconnect to program instance
  • Snapshot
  • Restore
  • Mechanism for implementing external services in language agnostic way (gRPC)
  • Full ARM64 host support
  • Programs can discover and communicate with their peers on a server (#23)
  • Milestone 1
  • Clone programs locally or remotely, with or without snapshotting (#9)
  • Milestone 2
  • Useful resource control policies need more thought (cgroup configuration etc.)
  • More services
  • Stable APIs
  • Additional security measures (#24, #25)
  • Non-Linux host support

User program support:

  • Low-level C API
  • Rust support
  • Go support
  • Approach for splitting WebAssembly app between browser (UI) and server (state)

Requirements and build instructions

Run-time dependencies:

  • Programs other than gate require Linux 5.3. gate's remote access features should work on any operating system, but are routinely tested only on Linux.

  • D-Bus is used for communication between gate and gate-daemon, requiring D-Bus user service (dbus-user-session). gate doesn't require D-Bus when accessing a remote server.

  • Programs other than gate may need external tools depending on their configuration and capabilities.

There are two approaches to building Gate: the normal Go way, or via the make.go build system.

Normal build using Go

Build requirements:

  • The gate client program requires Go 1.13
  • Other programs require Go 1.16

The Gate programs and programs importing Gate modules can be built normally using go install, go get etc. See the gate-cmd repository for a list of programs.

Gate runtime needs to execute some separately built binaries. To make the built Go programs self-contained, pre-built binaries are bundled into them by default. The pre-built binary files are under version control, and can be rebuilt using go generate. To disable bundling of pre-built binaries, specify -tags=gateexecdir for the Go build command, and use make.go to build and install them separately.

Build everything using make.go

Build targets:

  • go run make.go lib builds components implemented with C++ and assembly.
  • go run make.go bin builds Go programs without bundling non-Go components.
  • go run make.go builds all of the above and more.
  • go run make.go check runs tests.
  • go run make.go -h shows all targets and options.

Build requirements:

  • Linux
  • C++ compiler
  • Go compiler

Test requirements:

  • Python 3
  • uidmap (shadow-utils)
Installation

The build system builds a standalone installer which can be invoked as root:

  1. go run make.go or go run make.go installer ...
  2. sudo bin/install

See also

Documentation

Overview

Package gate contains general documentation for its subpackages. See https://gate.computer/gate for information about the Gate project.

Errors

Error strings may contain sensitive details. Some errors returned by Gate implement this interface:

interface {
	PublicError() string
}

The public error string is intended to be exposed to the client (if the API was called via a server endpoint). If there is no PublicError method or it returns the empty string, it's an internal error with no public explanation.

Source Files

Directories

Path Synopsis
cmd
gate-resource
Program gate-resource dumps C header file contents.
Program gate-resource dumps C header file contents.
Package facile exposes a simplified API to core functionality.
Package facile exposes a simplified API to core functionality.
Package include embeds C header files.
Package include embeds C header files.
packetio
Package packetio provides streaming utilities.
Package packetio provides streaming utilities.
abi
abi/rt
Package rt defines a public subset of the runtime library ABI.
Package rt defines a public subset of the runtime library ABI.
api
database/sql
Package sql implements Inventory and NonceChecker backed by SQL database.
Package sql implements Inventory and NonceChecker backed by SQL database.
web
web/api
Package api contains definitions useful for accessing the HTTP and websocket APIs.
Package api contains definitions useful for accessing the HTTP and websocket APIs.
web/api/debug
Package debug contains functionality useful for accessing the instance debugging API via HTTP.
Package debug contains functionality useful for accessing the instance debugging API via HTTP.
source
http
Package http can download objects from HTTP server.
Package http can download objects from HTTP server.
ipfs
Package ipfs can download objects via IPFS API server.
Package ipfs can download objects via IPFS API server.
Package trap enumerates trap identifiers.
Package trap enumerates trap identifiers.
bus
sys
Package sys contains syscall wrappers.
Package sys contains syscall wrappers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL