package module
Version: v0.28.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jun 23, 2017 License: Apache-2.0 Imports: 10 Imported by: 0



Build Status Coverage Status Go Report


Eremetic is a Mesos Framework to run one-shot tasks. The vision is to provide a bridge between Applications that need to run tasks and Mesos. That way a developer creating an application that needs to schedule tasks (such as cron) wouldn't need to connect to Mesos directly.


Send a cURL to the eremetic framework with how much cpu and memory you need, what docker image to run and which command to run with that image.

curl -H "Content-Type: application/json" \
     -X POST \
     -d '{"mem":22.0, "cpu":1.0, "image": "busybox", "command": "echo $(date)"}' \

These basic fields are required but you can also specify volumes, ports, environment variables, and URIs for the mesos fetcher to download. See for more examples on how to use eremetic.

JSON format:

  // Float64, fractions of a CPU to request
  "cpu":      1.0,
  // Float64, memory to use (MiB)
  "mem":       22.0,
  // String, full tag or hash of container to run
  "image":   "busybox",
  // Boolean, if set to true, docker image will be pulled before each task launch.
  "force_pull_image": false,
  // String, command to run in the docker container
  "command": "echo $(date)",
  // Array of Strings, arguements to pass to the docker container entrypoint
  "args": ["+%s"],
  // Array of Objects, volumes to mount in the container
  "volumes": [
      "container_path": "/var/run/docker.sock",
      "host_path": "/var/run/docker.sock"
  //String, network mode to pass to the container.
  "network" : "BRIDGE",
  //String, DNS address to be used by the container.
   "dns" : "",
  // Array of Objects, ports to forward to the container.
  // Assigned host ports are available as environment variables (e.g. PORT0, PORT1 and so on with PORT being an alias for PORT0).
  "ports": [
      "container_port": 80,
      "protocol": "tcp"
  // Object, Environment variables to pass to the container
  "env": {
    "KEY": "value"
  // Object, Will be merged to `env` when passed to Mesos, but masked when doing a GET.
  // See Clarification of the Masked Env field below for more information
  "masked_env": {
    "KEY": "value"
  // URIs and attributes of resource to download. You need to explicitly define 
  // `"extract"` to unarchive files.
  "fetch": [
      "uri" : "http://server.local/another_resource",
      "extract": false,
      "executable": false,
      "cache": false
  // Constraints for which agent the task can run on (beyond cpu/memory).
  // Matching is strict and only attributes are currently supported. If
  // multiple constraints exist, they are evaluated using AND (ie: all or none).
  "agent_constraints": [
          "attribute_name": "aws-region",
          "attribute_value": "us-west-2"
  // String, URL to post a callback to. Callback message has format:
  // {"time":1451398320,"status":"TASK_FAILED","task_id":"eremetic-task.79feb50d-3d36-47cf-98ff-a52ef2bc0eb5"}
  "callback_uri": "http://callback.local"

Most of this meta-data will not remain after a full restart of Eremetic.

Clarification of the Masked Env field

The purpose of the field is to provide a way to pass along environment variables that you don't want to have exposed in a subsequent GET call. It is not intended to provide full security, as someone with access to either the machine running Eremetic or the Mesos Agent that the task is being run on will still be able to view these values. These values are not encrypted, but simply masked when retrieved back via the API.

For security purposes, ensure TLS (https) is being used for the Eremetic communication and that access to any machines is properly restricted.


create /etc/eremetic/eremetic.yml with:

port: 8080
master: zk://<zookeeper_node1:port>,<zookeeper_node2:port>,(...)/mesos
messenger_address: <callback address for mesos>
messenger_port: <port for mesos to communicate on>
loglevel: DEBUG
logformat: json
queue_size: 100
url_prefix: <prefix to shim relative URLs behind a reverse proxy>


Eremetic uses a database to store task information. The driver can be configured by setting the database_driver value.

Allowed values are: zk, boltdb

The location of the database can be configured by setting the database value.


The default database that will be used unless anything is configured.

The default value of the database field is db/eremetic.db


If you use zk as a database driver, the database field must be provided as a complete zk-uri (zk://zk1:1234,zk2:1234/my/database).


To enable mesos framework authentication add the location of credential file to your configuration:

credential_file: /var/mesos_secret

The file should contain the Principal to authenticate and the secret separated by white space like so:

principal    secret_key



Clone the repository into $GOPATH/src/ This is needed because of internal package dependencies

Install dependencies

First you need to install dependencies. Parts of the eremetic code is auto-generated (assets and templates for the HTML view are compiled). In order for go generate to work, go-bindata and go-bindata-assetfs needs to be manually installed.

go get
go get

All other dependencies are vendored, so it is recommended to run eremetic with Go >= 1.6 or with GO15VENDOREXPERIMENT=1

Creating the docker image

To build a docker image with eremetic, simply run

make docker

Run make eremetic

Running on mesos

Eremetic can itself by run on mesos using e.g marathon. An example configuration for marathon is provided that is ready to be submitted through the api.

curl -X POST -H 'Content-Type: application/json' $MARATHON/v2/apps -d@misc/eremetic.json

Running tests

The default target of make builds and runs tests. Tests can also be run by running goconvey in the project root.

Running with minimesos

Using minimesos is a very simple way to test and play with eremetic.

docker run -e MASTER=$MINIMESOS_ZOOKEEPER -e HOST= -e DATABASE_DRIVER=zk -e DATABASE=$MINIMESOS_ZOOKEEPER/eremetic -e PORT=8000 -p 8000:8000 alde/eremetic:latest

hermit CLI

hermit is a command-line application to perform operations on a Eremetic server from the terminal.


These are the fine folks who helped build eremetic

  • Rickard Dybeck
  • David Keijser
  • Aidan McGinley
  • William Strucke
  • Charles G.
  • Clément Laforet
  • Marcus Olsson
  • Rares Mirica


Thanks to Sebastian Norde for the awesome logo!






View Source
const Masking = "*******"

Masking is the string used for masking environment variables.


View Source
var ErrQueueFull = errors.New("task queue is full")

ErrQueueFull is returned in the event of a full queue. This allows the caller to handle this as they see fit.


func ApplyMask

func ApplyMask(task *Task)

ApplyMask replaces masked environment variables with a masking string.

func Encode

func Encode(task *Task) ([]byte, error)

Encode encodes a task into a JSON byte array.

func IsArchive added in v0.28.0

func IsArchive(url string) bool

IsArchive is used to determine whether a url is an archive or not

func IsTerminal

func IsTerminal(state TaskState) bool

IsTerminal takes a string representation of a state and returns whether it is terminal or not.

func NotifyCallback

func NotifyCallback(task *Task)

NotifyCallback handles posting a JSON back to the URI given with the task.


type AgentConstraint added in v0.28.0

type AgentConstraint struct {
	AttributeName  string `json:"attribute_name"`
	AttributeValue string `json:"attribute_value"`

AgentConstraint is a constraint that is validated for each agent when determining where to schedule a task.

type CallbackData

type CallbackData struct {
	Time   int64  `json:"time"`
	Status string `json:"status"`
	TaskID string `json:"task_id"`

CallbackData holds information about the status update.

type DefaultTaskDB

type DefaultTaskDB struct {
	// contains filtered or unexported fields

DefaultTaskDB is a in-memory implementation of TaskDB.

func NewDefaultTaskDB

func NewDefaultTaskDB() *DefaultTaskDB

NewDefaultTaskDB returns a new instance of TaskDB.

func (*DefaultTaskDB) Clean

func (db *DefaultTaskDB) Clean() error

Clean removes all tasks from the database.

func (*DefaultTaskDB) Close

func (db *DefaultTaskDB) Close()

Close closes the connection to the database.

func (*DefaultTaskDB) DeleteTask added in v0.27.0

func (db *DefaultTaskDB) DeleteTask(id string) error

DeleteTask removes the task with a given id, or an error if not found.

func (*DefaultTaskDB) ListNonTerminalTasks

func (db *DefaultTaskDB) ListNonTerminalTasks() ([]*Task, error)

ListNonTerminalTasks returns all non-terminal tasks.

func (*DefaultTaskDB) PutTask

func (db *DefaultTaskDB) PutTask(task *Task) error

PutTask adds a new task to the database.

func (*DefaultTaskDB) ReadTask

func (db *DefaultTaskDB) ReadTask(id string) (Task, error)

ReadTask returns a task with a given id, or an error if not found.

func (*DefaultTaskDB) ReadUnmaskedTask

func (db *DefaultTaskDB) ReadUnmaskedTask(id string) (Task, error)

ReadUnmaskedTask returns a task with all its environment variables unmasked.

type Port

type Port struct {
	ContainerPort uint32 `json:"container_port"`
	HostPort      uint32 `json:"host_port"`
	Protocol      string `json:"protocol"`

Port defines a port mapping.

type Request

type Request struct {
	TaskCPUs          float64
	TaskMem           float64
	DockerImage       string
	Command           string
	Args              []string
	Volumes           []Volume
	Ports             []Port
	Network           string
	DNS               string
	Environment       map[string]string
	MaskedEnvironment map[string]string
	AgentConstraints  []AgentConstraint
	CallbackURI       string
	URIs              []string
	Fetch             []URI
	ForcePullImage    bool

Request is the internal structure of a Request

type Scheduler

type Scheduler interface {
	ScheduleTask(request Request) (string, error)
	Kill(taskID string) error

Scheduler defines an interface for scheduling tasks.

type Status

type Status struct {
	Time   int64     `json:"time"`
	Status TaskState `json:"status"`

Status represents the task status at a given time.

type Task

type Task struct {
	TaskCPUs          float64
	TaskMem           float64
	Command           string
	Args              []string
	User              string
	Environment       map[string]string
	MaskedEnvironment map[string]string
	Image             string
	Volumes           []Volume
	Ports             []Port
	Status            []Status
	ID                string
	Name              string
	Network           string
	DNS               string
	FrameworkID       string
	AgentID           string
	AgentConstraints  []AgentConstraint
	Hostname          string
	Retry             int
	CallbackURI       string
	SandboxPath       string
	AgentIP           string
	AgentPort         int32
	ForcePullImage    bool
	FetchURIs         []URI

Task represents the internal structure of a Task objert

func NewTask

func NewTask(request Request, name string) (Task, error)

NewTask returns a new instance of a Task.

func (*Task) CurrentStatus added in v0.27.0

func (task *Task) CurrentStatus() TaskState

CurrentStatus returns the current TaskState

func (*Task) IsRunning

func (task *Task) IsRunning() bool

IsRunning returns whether the task is currently running.

func (*Task) IsTerminated

func (task *Task) IsTerminated() bool

IsTerminated returns whether the task has been terminated.

func (*Task) IsTerminating added in v0.27.0

func (task *Task) IsTerminating() bool

IsTerminating returns whether a task is in the process of terminating

func (*Task) IsWaiting added in v0.27.0

func (task *Task) IsWaiting() bool

IsWaiting returns whether a task is waiting

func (*Task) LastUpdated

func (task *Task) LastUpdated() time.Time

LastUpdated returns the time of the latest status update.

func (*Task) UpdateStatus

func (task *Task) UpdateStatus(status Status)

UpdateStatus updates the current task status.

func (*Task) WasRunning

func (task *Task) WasRunning() bool

WasRunning returns whether the task was running at some point.

type TaskDB

type TaskDB interface {
	Clean() error
	PutTask(task *Task) error
	ReadTask(id string) (Task, error)
	DeleteTask(id string) error
	ReadUnmaskedTask(id string) (Task, error)
	ListNonTerminalTasks() ([]*Task, error)

TaskDB defines the functions needed by the database abstraction layer

type TaskState

type TaskState string

TaskState defines the valid task states.

const (
	// Standard mesos states
	TaskStaging  TaskState = "TASK_STAGING"
	TaskStarting TaskState = "TASK_STARTING"
	TaskRunning  TaskState = "TASK_RUNNING"
	TaskFinished TaskState = "TASK_FINISHED"
	TaskFailed   TaskState = "TASK_FAILED"
	TaskKilled   TaskState = "TASK_KILLED"
	TaskLost     TaskState = "TASK_LOST"
	TaskError    TaskState = "TASK_ERROR"

	// Custom eremetic states
	TaskQueued      TaskState = "TASK_QUEUED"
	TaskTerminating TaskState = "TASK_TERMINATING"

Valid task states

func (TaskState) String

func (s TaskState) String() string

type URI

type URI struct {
	URI        string `json:"uri"`
	Executable bool   `json:"executable"`
	Extract    bool   `json:"extract"`
	Cache      bool   `json:"cache"`

URI holds meta-data for a sandbox resource.

type Volume

type Volume struct {
	ContainerPath string `json:"container_path"`
	HostPath      string `json:"host_path"`

Volume is a mapping between ContainerPath and HostPath, to allow Docker to mount volumes.


Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL