mesos

package
v1.2.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 18, 2017 License: Apache-2.0, CC-BY-SA-4.0 Imports: 25 Imported by: 0

README

Using Docker Swarm and Mesos

Swarm comes with a built-in scheduler that works with the Swarm manager to schedule container resources. You can completely replace the built-in scheduler with a 3rd party scheduler. For example, you can replace it with the Mesos scheduler as described here.

When using Docker Swarm and Mesos, you use the Docker client to ask the Swarm manager to schedule containers. The Swarm manager then schedules those containers on a Mesos cluster.

The Docker Swarm on Mesos integration is experimental and not production ready, please use with caution.

Prerequisites

Each node in your Swarm must run a Mesos agent. The agent must be capable of starting tasks in a Docker Container using the --containerizers=docker option.

You need to configure two TCP ports on the agent. One port to listen for the Swarm manager, for example 2375. And a second TCP port to listen for the Mesos master, for example 3375.

Start the Docker Swarm manager

If you use a single Mesos master:

$ docker run -d -p <swarm_port>:2375 -p 3375:3375 \
    swarm manage \
        -c mesos-experimental \
        --cluster-opt mesos.address=<public_machine_ip> \
        --cluster-opt mesos.port=3375 \
        <mesos_master_ip>:<mesos_master_port>

The command above creates a Swarm manager listening at <swarm_port>. The <mesos_master_ip> value points to where Mesos master lives in the cluster. Typically, this is localhost, a hostname, or an IP address. The <public_machine_ip> value is the IP address for Mesos master to talk to Swarm manager. If Mesos master and Swarm manager are co-located on the same machine, you can use the 0.0.0.0 or localhost value.

If you use multiple Mesos masters:

$ docker run -d -p <swarm_port>:2375 -p 3375:3375 \
    swarm manage \
        -c mesos-experimental \
        --cluster-opt mesos.address=<public_machine_ip> \
        --cluster-opt mesos.port=3375 \
        zk://<mesos_masters_url>

Once the manager is running, check your configuration by running docker info as follows:

$ docker -H tcp://<swarm_ip>:<swarm_port> info
Containers: 0
Offers: 2
  Offer: 20150609-222929-1327399946-5050-14390-O6286
     └ cpus: 2
     └ mem: 1006 MiB
     └ disk: 34.37 GiB
     └ ports: 31000-32000
  Offer: 20150609-222929-1327399946-5050-14390-O6287
     └ cpus: 2
     └ mem: 1006 MiB
     └ disk: 34.37 GiB
     └ ports: 31000-32000

If you run into Abnormal executor termination error, you might want to run the Swarm container with an additional environment variable: SWARM_MESOS_USER=root.

Exporting the environment variable DOCKER_HOST=<swarm_ip>:<swarm_port> will facilitate interaction with the Docker client as it will not require -H anymore.

Limitations

Docker Swarm on Mesos provides a basic Mesos offers negotiation It is also possible to use Docker-compose to communicate to the Docker Swarm manager how to schedule Docker containers on the Mesos cluster. Docker Swarm on Mesos supports Mesos master detection with Zookeeper

Functionality

Docker Swarm on Mesos supports the following subset of Docker CLI commands:

  • attach
  • build
  • commit
  • cp
  • diff
  • events
  • export
  • history
  • images
  • info
  • inspect
  • logs
  • network
    • create
    • remove
    • ls
    • inspect
    • connect
    • disconnect
  • port
  • ps
  • run
  • save
  • search
  • stats
  • top

Unsupported docker commands

The current Swarm scheduler for Mesos implementation is not feature complete. Hence the following Docker CLI commands are unsupported and may cause unpredictable results:

  • create
  • import
  • kill
  • load
  • login
  • logout
  • pause
  • pull
  • push
  • rename
  • restart
  • rm
  • rmi
  • start
  • stop
  • tag
  • unpause
  • update
  • volume
  • wait

Known issues

  • Docker Swarm on Mesos only uses unreserved resources; and if Mesos offers reserved resources, the role info is also ignored which is rejected by Mesos master. See here for a proposal on letting Docker Swarm on Mesos use both un-reserved/reserved resources.
  • See here for a proposal on letting Docker Swarm on Mesos use revocable resources.
  • Restarted Mesos agents have flaky recovery in conjunction with Docker Swarm. When Mesos agents restart, Mesos master don’t always send offers for those agents to Docker Swarm. This issue seems to be solved with the Swarm 1.1 release.

To have a global view of the tracked issues please refer to Mesos issues in Github.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCluster 

func NewCluster(scheduler *scheduler.Scheduler, TLSConfig *tls.Config, master string, options cluster.DriverOpts, engineOptions *cluster.EngineOpts) (cluster.Cluster, error)

NewCluster for mesos Cluster creation

Types

type Cluster 

type Cluster struct {
	sync.RWMutex

	TLSConfig *tls.Config
	// contains filtered or unexported fields
}

Cluster struct for mesos

func (*Cluster) BuildImage added in v0.4.0 

func (c *Cluster) BuildImage(buildContext io.Reader, buildImage *types.ImageBuildOptions, out io.Writer) error

BuildImage builds an image

func (*Cluster) Container 

func (c *Cluster) Container(IDOrName string) *cluster.Container

Container returns the container with IdOrName in the cluster

func (*Cluster) Containers 

func (c *Cluster) Containers() cluster.Containers

Containers returns all the containers in the cluster.

func (*Cluster) CreateContainer 

func (c *Cluster) CreateContainer(config *cluster.ContainerConfig, name string, authConfig *types.AuthConfig) (*cluster.Container, error)

CreateContainer for container creation in Mesos task

func (*Cluster) CreateNetwork added in v1.0.0 

func (c *Cluster) CreateNetwork(name string, request *types.NetworkCreate) (*types.NetworkCreateResponse, error)

CreateNetwork creates a network in the cluster

func (*Cluster) CreateVolume added in v1.0.0 

func (c *Cluster) CreateVolume(request *volumetypes.VolumesCreateBody) (*types.Volume, error)

CreateVolume creates a volume in the cluster

func (*Cluster) Handle added in v0.4.0 

func (c *Cluster) Handle(e *cluster.Event) error

Handle callbacks for the events

func (*Cluster) Image 

func (c *Cluster) Image(IDOrName string) *cluster.Image

Image returns an image with IdOrName in the cluster

func (*Cluster) Images 

func (c *Cluster) Images() cluster.Images

Images returns all the images in the cluster.

func (*Cluster) Import 

func (c *Cluster) Import(source string, ref string, tag string, imageReader io.Reader, callback func(what, status string, err error))

Import image

func (*Cluster) Info 

func (c *Cluster) Info() [][2]string

Info gives minimal information about containers and resources on the mesos cluster

func (*Cluster) LaunchTask added in v1.1.0 

func (c *Cluster) LaunchTask(t *task.Task) bool

LaunchTask selects node and calls driver to launch a task

func (*Cluster) Load 

func (c *Cluster) Load(imageReader io.Reader, callback func(where, status string, err error))

Load images

func (*Cluster) Networks added in v1.0.0 

func (c *Cluster) Networks() cluster.Networks

Networks returns all the networks in the cluster.

func (*Cluster) Pull 

func (c *Cluster) Pull(name string, authConfig *types.AuthConfig, callback func(where, status string, err error))

Pull pulls images on the cluster nodes

func (*Cluster) RANDOMENGINE 

func (c *Cluster) RANDOMENGINE() (*cluster.Engine, error)

RANDOMENGINE returns a random engine.

func (*Cluster) RefreshEngine added in v1.2.6 

func (c *Cluster) RefreshEngine(hostname string) error

func (*Cluster) RefreshEngines added in v1.2.6 

func (c *Cluster) RefreshEngines() error

func (*Cluster) RegisterEventHandler 

func (c *Cluster) RegisterEventHandler(h cluster.EventHandler) error

RegisterEventHandler registers an event handler.

func (*Cluster) RemoveContainer 

func (c *Cluster) RemoveContainer(container *cluster.Container, force, volumes bool) error

RemoveContainer removes containers on mesos cluster

func (*Cluster) RemoveImage 

func (c *Cluster) RemoveImage(image *cluster.Image) ([]types.ImageDelete, error)

RemoveImage removes an image from the cluster

func (*Cluster) RemoveImages 

func (c *Cluster) RemoveImages(name string, force bool) ([]types.ImageDelete, error)

RemoveImages removes images from the cluster

func (*Cluster) RemoveNetwork added in v1.0.0 

func (c *Cluster) RemoveNetwork(network *cluster.Network) error

RemoveNetwork removes network from the cluster

func (*Cluster) RemoveVolumes added in v1.0.0 

func (c *Cluster) RemoveVolumes(name string) (bool, error)

RemoveVolumes removes volumes from the cluster

func (*Cluster) RenameContainer 

func (c *Cluster) RenameContainer(container *cluster.Container, newName string) error

RenameContainer renames a container

func (*Cluster) StartContainer added in v1.1.0 

func (c *Cluster) StartContainer(container *cluster.Container, hostConfig *dockerclient.HostConfig) error

StartContainer starts a container

func (*Cluster) TagImage added in v1.0.0 

func (c *Cluster) TagImage(IDOrName string, ref string, force bool) error

TagImage tags an image

func (*Cluster) TotalCpus 

func (c *Cluster) TotalCpus() int64

TotalCpus returns the total memory of the cluster

func (*Cluster) TotalMemory 

func (c *Cluster) TotalMemory() int64

TotalMemory returns the total memory of the cluster

func (*Cluster) UnregisterEventHandler added in v1.1.0 

func (c *Cluster) UnregisterEventHandler(h cluster.EventHandler)

UnregisterEventHandler unregisters a previously registered event handler.

func (*Cluster) Volumes added in v1.0.0 

func (c *Cluster) Volumes() cluster.Volumes

Volumes returns all the volumes in the cluster.

type Scheduler added in v1.1.0 

type Scheduler struct {
	scheduler.Scheduler
	// contains filtered or unexported fields
}

Scheduler structure for mesos driver

func NewScheduler added in v1.1.0 

func NewScheduler(config mesosscheduler.DriverConfig, cluster *Cluster, sched *scheduler.Scheduler) (*Scheduler, error)

NewScheduler for Scheduler mesos driver creation

func (*Scheduler) Disconnected added in v1.1.0 

func (s *Scheduler) Disconnected(mesosscheduler.SchedulerDriver)

Disconnected method

func (*Scheduler) Error added in v1.1.0 

func (s *Scheduler) Error(d mesosscheduler.SchedulerDriver, msg string)

Error method

func (*Scheduler) ExecutorLost added in v1.1.0 

func (s *Scheduler) ExecutorLost(mesosscheduler.SchedulerDriver, *mesosproto.ExecutorID, *mesosproto.SlaveID, int)

ExecutorLost method

func (*Scheduler) FrameworkMessage added in v1.1.0 

func (s *Scheduler) FrameworkMessage(mesosscheduler.SchedulerDriver, *mesosproto.ExecutorID, *mesosproto.SlaveID, string)

FrameworkMessage method

func (*Scheduler) OfferRescinded added in v1.1.0 

func (s *Scheduler) OfferRescinded(_ mesosscheduler.SchedulerDriver, offerID *mesosproto.OfferID)

OfferRescinded method

func (*Scheduler) Registered added in v1.1.0 

func (s *Scheduler) Registered(driver mesosscheduler.SchedulerDriver, fwID *mesosproto.FrameworkID, masterInfo *mesosproto.MasterInfo)

Registered method for registered mesos framework

func (*Scheduler) Reregistered added in v1.1.0 

func (s *Scheduler) Reregistered(mesosscheduler.SchedulerDriver, *mesosproto.MasterInfo)

Reregistered method for registered mesos framework

func (*Scheduler) ResourceOffers added in v1.1.0 

func (s *Scheduler) ResourceOffers(_ mesosscheduler.SchedulerDriver, offers []*mesosproto.Offer)

ResourceOffers method

func (*Scheduler) SlaveLost added in v1.1.0 

func (s *Scheduler) SlaveLost(mesosscheduler.SchedulerDriver, *mesosproto.SlaveID)

SlaveLost method

func (*Scheduler) StatusUpdate added in v1.1.0 

func (s *Scheduler) StatusUpdate(_ mesosscheduler.SchedulerDriver, taskStatus *mesosproto.TaskStatus)

StatusUpdate method

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.