Documentation ¶
Overview ¶
Package ec2system implements a bigmachine System that launches machines on dedicated EC2 spot instances. Ec2machine bootstraps instances through the use of cloud config and a bootstrap binary that runs the bigmachine supervisor service. The new binaries are then uploaded via bigmachine's RPC mechanism and execed remotely.
Ec2machine instances may be configured, but uses good defaults for GRAIL. It uses instance descriptions generated from Reflow's ec2instances tool to construct appropriate spot bid prices, and to configure instances according to their underlying characteristics. Ec2machine does not currently set up local storage beyond the boot gp2 EBS volume. (Its size may be configured.)
Secure communications is set up through an ephemeral CA stored at /tmp/bigmachine.pem.
TODO(marius): generalize this somewhere: grailmachine?
Index ¶
- Variables
- func SetMortality(v bool)
- type CloudFile
- type CloudUnit
- type Flavor
- type System
- func (s *System) Event(typ string, fieldPairs ...interface{})
- func (s *System) Exit(code int)
- func (s *System) HTTPClient() *http.Client
- func (s *System) Init(b *bigmachine.B) error
- func (*System) KeepaliveConfig() (period, timeout, rpcTimeout time.Duration)
- func (s *System) ListenAndServe(addr string, handler http.Handler) error
- func (s *System) Main() error
- func (s *System) Maxprocs() int
- func (s *System) Name() string
- func (s *System) Read(ctx context.Context, m *bigmachine.Machine, filename string) (io.Reader, error)
- func (s *System) Shutdown()
- func (s *System) Start(ctx context.Context, count int) ([]*bigmachine.Machine, error)
- func (s *System) Tail(ctx context.Context, m *bigmachine.Machine) (io.Reader, error)
Constants ¶
This section is empty.
Variables ¶
var Instance = new(System)
Instance is a default ec2machine System.
Functions ¶
func SetMortality ¶ added in v0.5.5
func SetMortality(v bool)
SetMortality conrols the mortality of EC2 instances for help with debugging low level boot time issues. It is equivalent to the 'ec2machineimmportal' flag for configurations where flags cannot be used.
Types ¶
type CloudFile ¶
type CloudFile struct { Path string `yaml:"path,omitempty"` Permissions string `yaml:"permissions,omitempty"` Owner string `yaml:"owner,omitempty"` Content string `yaml:"content,omitempty"` }
CloudFile is a component of the cloudConfig configuration as accepted by cloud-init. It represents a file that will be written to the filesystem.
type CloudUnit ¶
type CloudUnit struct { Name string `yaml:"name,omitempty"` Command string `yaml:"command,omitempty"` Enable bool `yaml:"enable,omitempty"` Content string `yaml:"content,omitempty"` // Sync determines whether the command should be run synchronously. Sync bool `yaml:"-"` }
CloudUnit is a component of the cloudConfig configuration as accepted by cloud-init. It represents a systemd unit.
type Flavor ¶
type Flavor int
A Flavor is the flavor of operating system used in the system. The system flavor adjusts for local variations in setup.
type System ¶
type System struct { // OnDemand determines whether to use on-demand or spot instances. OnDemand bool // InstanceType is the EC2 instance type to launch in this system. // It defaults to m3.medium. InstanceType string // AMI is the AMI used to boot instances with. The AMI must support // cloud config and use systemd. The default AMI is a recent stable CoreOS // build. AMI string // Flavor is the operating system flavor of the AMI. Flavor Flavor // AWSConfig is used to launch the system's instances. AWSConfig *aws.Config // DefaultRegion is the preferred default region for this system // instance to use if one is not specified in AWSConfig. It this is // not set the default region will continue to be us-west-2. DefaultRegion string // InstanceProfile is the instance profile with which to launch the instance. // This should be set if the instances need AWS credentials. InstanceProfile string // SecurityGroup is the security group into which instances are launched. SecurityGroup string // SecurityGroups are the security group into which instances are launched. // If set, it used in preference to SecurityGroup above. SecurityGroups []string // Subnet is the subnet into which instances are launched. Subnet string // Diskspace is the amount of disk space in GiB allocated // to the instance's root EBS volume. Its default is 200. Diskspace uint // Dataspace is the amount of data disk space allocated // in /mnt/data. It defaults to 0. Data are striped across // multiple gp2 EBS slices in order to improve throughput. Dataspace uint // Binary is the URL to a bootstrap binary to be used when launching // system instances. It should be a minimal bigmachine build that // contains the ec2machine implementation and runs bigmachine's // supervisor service. If the value of Binary is empty, then the // default ec2boot binary is used. // // The binary is fetched by a vanilla curl(1) invocation, and thus needs // to be publicly available. Binary string // SshKeys is the list of sshkeys that installed as authorized keys // in the instance. On system initialization, SshKeys is amended // with the contents of $HOME/.ssh/id_rsa.pub, if it exists, and the // keys available in the SSH agent reachable by $SSH_AUTH_SOCK, if // one exists. SshKeys []string // The EC2 key pair name to associate with the created instances when // the instance this launched. This key name will appear in the EC2 // instance's metadata. EC2KeyName string // The user running the application. For tagging. Username string // AdditionalFiles are added to the worker cloud-init configuration. AdditionalFiles []CloudFile // AdditionalUnits are added to the worker cloud-init configuration. AdditionalUnits []CloudUnit // AdditionalEC2Tags will be applied to this system's instances. AdditionalEC2Tags []*ec2.Tag // Eventer is used to log semi-structured events in service of analytics. Eventer eventlog.Eventer // contains filtered or unexported fields }
System implements a bigmachine system for EC2 instances. See package docs for more details.
func (*System) HTTPClient ¶
HTTPClient returns an HTTP client configured to securely call instances launched by ec2machine over http/2.
func (*System) Init ¶
func (s *System) Init(b *bigmachine.B) error
Init initializes the system. Before validating the system configuration and providing defaults, Init checks that the architecture and OS reported by Go's runtime is amd64 and linux respectively. Currently these are the only supported architectures from which to launch ec2machine systems.
Init also establishes the AWS API session with which it communicates to the EC2 API. It uses the default session constructor furnished by the AWS SDK.
func (*System) KeepaliveConfig ¶
func (*System) ListenAndServe ¶
ListenAndServe serves the provided handler on a HTTP server configured for secure communications between ec2system instances.
func (*System) Main ¶
Main runs a bigmachine worker node. It sets up an HTTP server that performs mutual authentication with bigmachine clients launched from the same system instance. Main also starts a local HTTP server on port 3333 for debugging and local inspection.
func (*System) Shutdown ¶
func (s *System) Shutdown()
Shutdown is a no-op.
TODO(marius): consider setting longer keepalives to maintain instances for future invocations.