ec2

Documentation

Overview

Package ec2 is a stand-in for the Amazon EC2 API, for testing code that calls it through the official SDKs.

Server is an http.Handler that keeps EC2 resources in memory and answers the EC2 Query API: form-encoded calls with an Action and Version 2016-11-15, answered in EC2's XML. Serve it with net/http/httptest and point a client at it with the SDK's base endpoint setting, or the AWS_ENDPOINT_URL_EC2 or AWS_ENDPOINT_URL environment variable, and any static credentials. A fault.Injector makes chosen calls fail, including after they took effect, so tests can check that code which retries converges.

It covers Elastic IP addresses (AllocateAddress, DescribeAddresses, ReleaseAddress, AssociateAddress, DisassociateAddress), network interfaces (CreateNetworkInterface, DescribeNetworkInterfaces, DeleteNetworkInterface), instances (RunInstances, DescribeInstances, StartInstances, StopInstances, TerminateInstances, ModifyInstanceAttribute for the instance type), images (DescribeImages, RegisterImage, DeregisterImage), instance types (DescribeInstanceTypes), EBS volumes (CreateVolume, DescribeVolumes, DeleteVolume, AttachVolume, DetachVolume), EBS snapshots (CreateSnapshot, DescribeSnapshots, DeleteSnapshot), placement groups (CreatePlacementGroup, DescribePlacementGroups, DeletePlacementGroup) and tags (CreateTags, DeleteTags), with the real API's error codes. Other actions answer 404 with the code InvalidAction. Where it differs from EC2:

• Signatures are not checked. The region comes from the credential scope of the SigV4 Authorization header (or the X-Amz-Credential query parameter), and each region keeps its own resources. There is one account: the server keeps no owner IDs.
• Subnet and security group IDs and key pair names are not validated, and there are no VPCs: interfaces report no VPC ID, and their availability zone is the region with "a" appended unless an instance launched in another. ReleaseAddress fails with InvalidIPAddress.InUse for an associated address, as in a nondefault VPC, rather than disassociating it. Private IP addresses are unique within a subnet and come from 10.0.0.0/8; public ones are unique across every region and come from 100.64.0.0/10, never from routable ranges.
• There is no quota on addresses unless WithAddressLimit sets one, and no capacity or instance limit: a fault rule with a code such as InsufficientInstanceCapacity stands in for one.
• Each interface has one private IP address; an address associates with it. Associating an address with an interface that already has another disassociates the other, and associating it again with the interface it is on returns its current association. AssociateAddress with an InstanceId associates with the instance's primary interface, and fails with InvalidInstanceID unless the instance is running or stopped; with a NetworkInterfaceId it fails with IncorrectInstanceState while the interface's instance is pending.
• A CreateNetworkInterface, RunInstances or CreateVolume that repeats a ClientToken in the same region with the same parameters gets what the first call created without taking effect again: the interface or volume as first created, or the reservation's instances as they are now. With other parameters it fails with IdempotentParameterMismatch. The server keeps the last 10,000 tokens.
• Filter values take the * and ? wildcards. A filter name the server does not support, like any parameter it does not support, is rejected with 400 rather than ignored.

Instances:

• There is no default VPC: RunInstances needs a SubnetId or one network interface at device index 0, either new or an existing available one, and fails with VPCIdNotSpecified without. Instances are EBS-backed HVM instances with one network interface; launch templates, spot options, instance profiles and the like are rejected as unknown parameters.
• RunInstances records MetadataOptions, PrivateDnsNameOptions and CpuOptions, with EC2's defaults for what it leaves out, and DescribeInstances returns them. A resource-name hostname type makes the private DNS name the instance ID's. The primary interface's InterfaceType may be "interface" or "efa", or with an existing interface only that interface's type, and its NetworkCardIndex only 0, which counts as the same launch as leaving it out. Metadata options read pending until the instance first settles. EFA and AMD SEV-SNP are not checked against the instance type. CpuOptions needs both CoreCount and ThreadsPerCore or neither, and takes up to the type's default cores and threads per core, which DescribeInstanceTypes reports. Any core count from 1 up to the default is accepted, where EC2 accepts only the counts it lists for each type. A type change resets the core count and threads to the new type's defaults.
• An instance's image must be registered with Server.PutImage, and its type must be one Server.PutInstanceType added or one of a small built-in set, which DescribeInstanceTypes lists, that runs the image's architecture. The default type is m1.small, as in EC2.
• Instances move through EC2's states, pending to running, stopping to stopped, and shutting-down to terminated, when describes see them: by default the first DescribeInstances after a change shows the state it settles in, so the SDK's waiters return at once. WithSettleAfter holds them in transition for more describes, or until Server.Settle. Describes are consistent at once; EC2's are eventually consistent.
• A terminated instance stays visible, as in EC2, until Server.ForgetInstance removes it. On termination it releases its interface and volumes and deletes those marked to delete on termination: a new interface and the image's volumes by default, an existing interface not.
• An instance launched with AssociatePublicIpAddress gets a public IP address, released while it is stopped or has an Elastic IP address and assigned again when it starts or the Elastic IP is disassociated.
• DescribeImages lists only registered images. ExecutableBy is not supported; the owner "self" matches images with no OwnerID, and other owners match the OwnerID or OwnerAlias. Unknown image IDs fail with InvalidAMIID.NotFound.
• RegisterImage registers an EBS-backed HVM image of the caller's own from snapshots the server holds: its root device needs a mapping with a snapshot ID. The snapshot's state is not checked, and an image is available at once. A mapping's volume type is gp2 unless set. DeregisterImage removes any registered image, whatever its owner, and leaves its snapshots.
• Volumes go from creating to available, and attachments from attaching to attached and from detaching to detached, the same way instances move. Snapshots go from pending to completed the same way, as DescribeSnapshots sees them, and record the volume's size and encryption, not its data. A volume from a snapshot the server holds is the snapshot's size unless set; any other snapshot ID is recorded, not read, and a volume from it without a size is 8 GiB. Volumes, snapshots and placement groups are deleted at once rather than passing through a deleting state. DeleteSnapshot fails with InvalidSnapshot.InUse while a registered image maps the snapshot. Every snapshot is the caller's own: DescribeSnapshots with an owner other than "self" lists none, and snapshots report no owner ID. CreatePlacementGroup needs a GroupName and a Strategy, which EC2 makes optional.
• CreateTags and DeleteTags work on instances, volumes, snapshots, images, placement groups (by group ID), network interfaces and Elastic IP addresses, and change nothing unless every resource exists.

WithPolicy grants the caller a set of IAM actions and refuses a call that needs one it lacks with 403 UnauthorizedOperation, before any fault. A call needs ec2: and its action, and ec2:CreateTags when it has a TagSpecification. Actions are not checked against the resources they act on or against condition keys: a grant covers every resource.

Fault method names are "ec2." and the action, such as "ec2.AllocateAddress". A call's resource is the region and the ID it acts on, such as "us-east-1/eipalloc-00000000000000001", or the first of them for a call that takes several; a create call names the resource it creates (the first instance, for RunInstances), placement group calls name the group by name, and a describe call names the collection, such as "us-east-1/addresses". EC2 has no long-running operations, so fault.ModeFailOperation fails a call as fault.ModeReject does. A fault's Reason is the error code, except that the fault package's generic reasons become InternalError, Unavailable (for 503) and RequestLimitExceeded. A call a fault rejects has no effect, so a rejected describe does not move an instance along.

A lost response aborts the handler with http.ErrAbortHandler after sending the response header, so the http.Server serving it closes the connection. Calling ServeHTTP directly propagates that panic.

Example

The first allocation takes effect but its response is lost, so the caller sees an error even though the address now exists. Code under test must find and adopt it rather than leak it. Real tests point the official SDK at srv.URL with its base endpoint setting.

package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"
	"strings"

	"github.com/parallelworks/squall/aws/ec2"
	"github.com/parallelworks/squall/fault"
)

func main() {
	faults := fault.New().Add(fault.Rule{Method: "ec2.AllocateAddress", Nth: 1, Fault: fault.LoseResponse()})
	s := ec2.New(ec2.WithFaults(faults))
	srv := httptest.NewServer(s)
	defer srv.Close()

	req, _ := http.NewRequest(http.MethodPost, srv.URL,
		strings.NewReader("Action=AllocateAddress&Version=2016-11-15&Domain=vpc"))
	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential=AKID/20260101/us-east-1/ec2/aws4_request, SignedHeaders=host, Signature=0")
	resp, err := http.DefaultClient.Do(req)
	if err == nil {
		_, err = resp.Body.Read(make([]byte, 1<<10))
		resp.Body.Close()
	}
	fmt.Println("allocate failed:", err != nil)

	a, _ := s.LookupAddress("us-east-1", "eipalloc-00000000000000001")
	fmt.Println("address exists:", a.PublicIP)
	fmt.Println(faults.Injected()[0])
}

Output:
allocate failed: true
address exists: 100.64.0.1
#1 ec2.AllocateAddress us-east-1/eipalloc-00000000000000001: lose response

Index

• Variables
• type Address
• type BlockDeviceMapping
• type CPUOptions
• type Image
• type Instance
• type InstanceType
• type MetadataOptions
• type NetworkInterface
• type Option
  • func WithAddressLimit(n int) Option
  • func WithFaults(in *fault.Injector) Option
  • func WithPolicy(p *access.Policy) Option
  • func WithSettleAfter(n int) Option
• type PlacementGroup
• type PrivateDNSNameOptions
• type Server
  • func New(opts ...Option) *Server
  • func (s *Server) Addresses() []Address
  • func (s *Server) AttachNetworkInterface(region, id, instanceID string) error
  • func (s *Server) DetachNetworkInterface(region, id string) error
  • func (s *Server) ForgetInstance(region, id string) error
  • func (s *Server) Images() []Image
  • func (s *Server) Instances() []Instance
  • func (s *Server) LookupAddress(region, allocationID string) (Address, bool)
  • func (s *Server) LookupInstance(region, id string) (Instance, bool)
  • func (s *Server) LookupNetworkInterface(region, id string) (NetworkInterface, bool)
  • func (s *Server) NetworkInterfaces() []NetworkInterface
  • func (s *Server) PlacementGroups() []PlacementGroup
  • func (s *Server) PutAddress(region string, a Address) (Address, error)
  • func (s *Server) PutImage(region string, img Image) (Image, error)
  • func (s *Server) PutInstanceType(t InstanceType) error
  • func (s *Server) PutNetworkInterface(region string, ni NetworkInterface) (NetworkInterface, error)
  • func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (s *Server) Settle()
  • func (s *Server) Snapshots() []Snapshot
  • func (s *Server) Volumes() []Volume
• type Snapshot
• type Volume

Examples

• Package

Variables

var ErrNotFound = errors.New("ec2: resource not found")

ErrNotFound is returned by the server's methods for a resource it does not hold.

Types

type Address

type Address struct {
	// Region is where the address is allocated. The server sets it.
	Region       string
	AllocationID string
	PublicIP     string
	// Domain is always "vpc".
	Domain string
	// AssociationID, NetworkInterfaceID and PrivateIPAddress are set while the
	// address is associated with a network interface.
	AssociationID      string
	NetworkInterfaceID string
	PrivateIPAddress   string
	Tags               map[string]string
}

Address is an Elastic IP address.

type BlockDeviceMapping

type BlockDeviceMapping struct {
	DeviceName string
	SnapshotID string
	// VolumeSize is in GiB.
	VolumeSize          int
	VolumeType          string
	Iops                int
	Throughput          int
	DeleteOnTermination bool
	Encrypted           bool
}

BlockDeviceMapping is an EBS volume an image or instance has at a device.

type CPUOptions

type CPUOptions struct {
	CoreCount      int
	ThreadsPerCore int
	// AmdSevSnp is "enabled", "disabled", or empty when the launch left it out.
	AmdSevSnp string
}

CPUOptions are an instance's CPU settings.

type Image

type Image struct {
	// Region is where the image is. The server sets it.
	Region      string
	ImageID     string
	Name        string
	Description string
	// Architecture is "x86_64" unless set.
	Architecture string
	// RootDeviceName is "/dev/xvda" unless set.
	RootDeviceName string
	// BlockDeviceMappings are the volumes an instance launches with. Unless
	// set, the image has an 8 GiB gp3 root volume deleted on termination.
	BlockDeviceMappings []BlockDeviceMapping
	// OwnerID is empty for an image of the caller's own, which the owner
	// "self" matches.
	OwnerID      string
	OwnerAlias   string
	Public       bool
	CreationDate time.Time
	Tags         map[string]string
}

Image is an Amazon Machine Image (AMI).

type Instance

type Instance struct {
	// Region is where the instance is. The server sets it.
	Region        string
	InstanceID    string
	ReservationID string
	ImageID       string
	InstanceType  string
	Architecture  string
	// State is pending, running, stopping, stopped, shutting-down or terminated.
	State            string
	AvailabilityZone string
	PlacementGroup   string
	PlacementGroupID string
	// NetworkInterfaceID, SubnetID and PrivateIPAddress are the primary
	// interface's, until the instance is terminated.
	NetworkInterfaceID string
	SubnetID           string
	PrivateIPAddress   string
	// PublicIPAddress is the public IP address EC2 assigned the instance at
	// launch. It is released while the instance is stopped or has an Elastic
	// IP address.
	PublicIPAddress string
	RootDeviceName  string
	KeyName         string
	ClientToken     string
	// UserData is the user data, decoded.
	UserData     string
	EBSOptimized bool
	LaunchIndex  int
	LaunchTime   time.Time
	Tags         map[string]string
	// MetadataOptions, PrivateDNSNameOptions and CPUOptions are as launched,
	// with EC2's defaults for what the launch left out.
	MetadataOptions       MetadataOptions
	PrivateDNSNameOptions PrivateDNSNameOptions
	CPUOptions            CPUOptions
	// contains filtered or unexported fields
}

Instance is an EC2 instance.

type InstanceType

type InstanceType struct {
	InstanceType string
	// Architectures are the architectures the type runs, such as "x86_64".
	Architectures []string
	VCPUs         int
	MemoryMiB     int
	// ThreadsPerCore is how many threads each core runs by default, 1 or 2.
	ThreadsPerCore int
}

InstanceType is an instance type DescribeInstanceTypes reports and RunInstances accepts.

type MetadataOptions

type MetadataOptions struct {
	// HTTPTokens is "optional" or "required".
	HTTPTokens              string
	HTTPPutResponseHopLimit int
	// HTTPEndpoint, HTTPProtocolIPv6 and InstanceMetadataTags are "enabled"
	// or "disabled".
	HTTPEndpoint         string
	HTTPProtocolIPv6     string
	InstanceMetadataTags string
}

MetadataOptions are an instance's instance metadata service settings.

type NetworkInterface

type NetworkInterface struct {
	// Region is where the interface is. The server sets it.
	Region             string
	NetworkInterfaceID string
	SubnetID           string
	AvailabilityZone   string
	Description        string
	// Status is "available", or "in-use" while the interface is attached.
	Status           string
	MACAddress       string
	PrivateIPAddress string
	SecurityGroupIDs []string
	Tags             map[string]string
	// AttachmentID and InstanceID are set while the interface is attached.
	AttachmentID string
	InstanceID   string
	// DeleteOnTermination is whether terminating the instance deletes the interface.
	DeleteOnTermination bool
	// InterfaceType is "interface", the default, or "efa".
	InterfaceType string
}

NetworkInterface is an elastic network interface.

type Option

type Option func(*Server)

Option configures a Server.

func WithAddressLimit

func WithAddressLimit(n int) Option

WithAddressLimit caps the Elastic IP addresses a region holds, as the account quota does. AllocateAddress past it fails with AddressLimitExceeded. The default, 0, is no limit.

func WithFaults

func WithFaults(in *fault.Injector) Option

WithFaults makes the server ask in which calls to fail.

func WithPolicy

func WithPolicy(p *access.Policy) Option

WithPolicy makes the server refuse calls that need an IAM action p lacks, as EC2 refuses them, and records what each call needed. Without it every call is allowed.

func WithSettleAfter

func WithSettleAfter(n int) Option

WithSettleAfter sets how many describes see an instance, volume or snapshot in a transitional state, such as pending or stopping, before one shows the state it settles in: DescribeInstances calls for an instance, DescribeVolumes calls for a volume and DescribeSnapshots calls for a snapshot, counting each call that names it or names no IDs. The default, 0, settles it on the first describe, so the SDK's waiters return at once. A negative n settles it only on Server.Settle.

type PlacementGroup

type PlacementGroup struct {
	// Region is where the group is. The server sets it.
	Region    string
	GroupID   string
	GroupName string
	// Strategy is "cluster", "spread" or "partition".
	Strategy       string
	PartitionCount int
	SpreadLevel    string
	Tags           map[string]string
}

PlacementGroup is a placement group.

type PrivateDNSNameOptions

type PrivateDNSNameOptions struct {
	// HostnameType is "ip-name" or "resource-name".
	HostnameType                    string
	EnableResourceNameDNSARecord    bool
	EnableResourceNameDNSAAAARecord bool
}

PrivateDNSNameOptions are how an instance's private DNS name is formed.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is a stand-in for the Amazon EC2 API. Create one with New. It is safe for concurrent use.

func New

func New(opts ...Option) *Server

New returns an empty server.

func (*Server) Addresses

func (s *Server) Addresses() []Address

Addresses returns every address the server holds, ordered by region and allocation ID.

func (*Server) AttachNetworkInterface

func (s *Server) AttachNetworkInterface(region, id, instanceID string) error

AttachNetworkInterface marks the interface with id in region attached to instanceID, as attaching it to an instance would: its status becomes "in-use", so it cannot be deleted until Server.DetachNetworkInterface.

func (*Server) DetachNetworkInterface

func (s *Server) DetachNetworkInterface(region, id string) error

DetachNetworkInterface marks the interface with id in region detached: its status becomes "available".

func (*Server) ForgetInstance

func (s *Server) ForgetInstance(region, id string) error

ForgetInstance removes the terminated instance with id in region, as EC2 does some time after it terminates.

func (*Server) Images

func (s *Server) Images() []Image

Images returns every image the server holds, ordered by region and ID.

func (*Server) Instances

func (s *Server) Instances() []Instance

Instances returns every instance the server holds, terminated ones included, ordered by region and ID.

func (*Server) LookupAddress

func (s *Server) LookupAddress(region, allocationID string) (Address, bool)

LookupAddress returns the address with allocationID in region.

func (*Server) LookupInstance

func (s *Server) LookupInstance(region, id string) (Instance, bool)

LookupInstance returns the instance with id in region.

func (*Server) LookupNetworkInterface

func (s *Server) LookupNetworkInterface(region, id string) (NetworkInterface, bool)

LookupNetworkInterface returns the network interface with id in region.

func (*Server) NetworkInterfaces

func (s *Server) NetworkInterfaces() []NetworkInterface

NetworkInterfaces returns every network interface the server holds, ordered by region and ID.

func (*Server) PlacementGroups

func (s *Server) PlacementGroups() []PlacementGroup

PlacementGroups returns every placement group the server holds, ordered by region and name.

func (*Server) PutAddress

func (s *Server) PutAddress(region string, a Address) (Address, error)

PutAddress stores a in region as if it had been allocated, replacing an address with the same allocation ID, and returns it as stored. It fills in what a leaves empty, such as the allocation ID and public IP address. When a names a network interface, the address is associated with it. Use it to set up state, such as an address another owner already holds.

func (*Server) PutImage

func (s *Server) PutImage(region string, img Image) (Image, error)

PutImage registers img in region, replacing an image with the same ID, and returns it as stored. It fills in what img leaves empty, such as the ID, architecture and root volume.

func (*Server) PutInstanceType

func (s *Server) PutInstanceType(t InstanceType) error

PutInstanceType adds t to the instance types the server knows, replacing one with the same name. Its architecture is "x86_64" unless set. Unless set, its threads per core are 2 for an x86 type with an even number of vCPUs and 1 otherwise; set 1 for x86 types that EC2 runs without simultaneous multithreading, such as m7a.

func (*Server) PutNetworkInterface

func (s *Server) PutNetworkInterface(region string, ni NetworkInterface) (NetworkInterface, error)

PutNetworkInterface stores ni in region as if it had been created, replacing an interface with the same ID, and returns it as stored. It requires a subnet ID and fills in what ni leaves empty, such as the ID, private IP address and MAC address. An interface with an InstanceID is in use. Use it to set up state, such as an interface another owner holds.

func (*Server) ServeHTTP

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP answers an EC2 Query API call.

func (*Server) Settle

func (s *Server) Settle()

Settle completes every transition in progress, as enough describes would: pending instances run, stopping ones stop, shutting-down ones terminate, volumes finish creating, attaching and detaching, and snapshots complete.

func (*Server) Snapshots

func (s *Server) Snapshots() []Snapshot

Snapshots returns every snapshot the server holds, ordered by region and ID.

func (*Server) Volumes

func (s *Server) Volumes() []Volume

Volumes returns every volume the server holds, ordered by region and ID.

type Snapshot

type Snapshot struct {
	// Region is where the snapshot is. The server sets it.
	Region     string
	SnapshotID string
	VolumeID   string
	// VolumeSize is in GiB.
	VolumeSize  int
	Description string
	Encrypted   bool
	// State is "pending" or "completed".
	State     string
	StartTime time.Time
	Tags      map[string]string
	// contains filtered or unexported fields
}

Snapshot is an EBS snapshot.

type Volume

type Volume struct {
	// Region is where the volume is. The server sets it.
	Region           string
	VolumeID         string
	AvailabilityZone string
	// Size is in GiB.
	Size       int
	VolumeType string
	Iops       int
	Throughput int
	SnapshotID string
	Encrypted  bool
	// State is "creating", "available" or "in-use".
	State      string
	CreateTime time.Time
	Tags       map[string]string
	// InstanceID, Device, AttachmentState, AttachTime and DeleteOnTermination
	// are set while the volume is attached. AttachmentState is "attaching",
	// "attached" or "detaching".
	InstanceID          string
	Device              string
	AttachmentState     string
	AttachTime          time.Time
	DeleteOnTermination bool
	// contains filtered or unexported fields
}

Volume is an EBS volume.