cloud

Documentation

Overview

Package cloud contains utilities for distributed InMAP simulations.

Index

• func JobSpec(root *cobra.Command, config *viper.Viper, name string, ...) (*cloudrpc.JobSpec, error)
• func OpenBucket(ctx context.Context, bucketName string) (*blob.Bucket, error)
• type Client
  • func NewClient(k kubernetes.Interface, root *cobra.Command, config *viper.Viper, ...) (*Client, error)
  • func NewFakeClient(checkConfig func([]string), checkRun func([]byte, error), bucket string, ...) (*Client, error)
  • func (c *Client) Delete(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobName, error)
  • func (c *Client) Output(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobOutput, error)
  • func (c *Client) RunJob(ctx context.Context, job *cloudrpc.JobSpec) (*cloudrpc.JobStatus, error)
  • func (c *Client) Status(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobStatus, error)
• type FakeRPCClient
  • func (c FakeRPCClient) Delete(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobName, error)
  • func (c FakeRPCClient) Output(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobOutput, error)
  • func (c FakeRPCClient) RunJob(ctx context.Context, job *cloudrpc.JobSpec, op ...grpc.CallOption) (*cloudrpc.JobStatus, error)
  • func (c FakeRPCClient) Status(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobStatus, error)

Functions

func JobSpec

func JobSpec(root *cobra.Command, config *viper.Viper, name string, cmdArgs, inputFiles []string, memoryGB int32) (*cloudrpc.JobSpec, error)

JobSpec initializes a cloudrpc.JobSpec object from the given configuration information. memoryGB and storageGB are the required amounts of RAM and hard-disk storage, respectively, in gigabytes. name is the user-specified job name, cmdArgs is a list of InMAP sub-commands (e.g., "run steady"), and inputFiles is a list of the configuration arguments that represent input files.

func OpenBucket

func OpenBucket(ctx context.Context, bucketName string) (*blob.Bucket, error)

OpenBucket returns the blob storage bucket specified by bucketName, where bucketName must be in the format 'provider://name' where provider is the name of the storage provider and name is the name of the bucket. Even if name contains subdirectories, only the base directory name will be used when opening the bucket. The currently accepted storage providers are "file" for the local filesystem (e.g., for testing), "gs" for Google Cloud Storage, and "s3" for AWS S3.

Types

type Client

type Client struct {
	*grpcweb.WrappedGrpcServer

	kubernetes.Interface

	// Image holds the container image to be used.
	// The default is "inmap/inmap:latest".
	Image string

	// Volumes specifies any Kubernetes volumes that are to be
	// mounted in the containers that are created.
	// Each volume will be mounted at /data/volumeName
	// with read-only access.
	Volumes []core.Volume
	// contains filtered or unexported fields
}

Client is a Kubernetes client for InMAP.

func NewClient

func NewClient(k kubernetes.Interface, root *cobra.Command, config *viper.Viper, bucketName string, inputFileArgs, outputFileArgs []string) (*Client, error)

NewClient creates a new distributed InMAP Kubernetes client. root is the root command to be run, config holds simulation configuration information, and bucketName is the name of a blob storage bucket for storing output files in the format gs://bucketname. inputFileArgs and outputFileArgs list the names of the configuration arguments that represent input and output files.

func NewFakeClient

func NewFakeClient(checkConfig func([]string), checkRun func([]byte, error), bucket string, root *cobra.Command, config *viper.Viper, inputFileArgs, outputFileArgs []string) (*Client, error)

NewFakeClient creates a client for testing. Jobs that are created using this client are run locally. The InMAP command must be compiled for it to work, e.g., `go install github.com/spatialmodel/inmap/cmd/inmap`. The checkConfig and checkRun functions, if not nil, will be run before and after executing the inmap command, respectively.

func (*Client) Delete

func (c *Client) Delete(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobName, error)

Delete deletes the given job.

func (*Client) Output

func (c *Client) Output(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobOutput, error)

Output returns the output of the specified job.

func (*Client) RunJob

func (c *Client) RunJob(ctx context.Context, job *cloudrpc.JobSpec) (*cloudrpc.JobStatus, error)

RunJob creates (and queues) a Kubernetes job with the given name that executes the given command with the given command-line arguments on the given container image. resources specifies the minimum required resources for execution.

func (*Client) Status

func (c *Client) Status(ctx context.Context, job *cloudrpc.JobName) (*cloudrpc.JobStatus, error)

Status returns the status of the given job.

type FakeRPCClient

type FakeRPCClient struct {
	Client *Client
}

FakeRPCClient is a local RPC client for testing.

func (FakeRPCClient) Delete

func (c FakeRPCClient) Delete(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobName, error)

func (FakeRPCClient) Output

func (c FakeRPCClient) Output(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobOutput, error)

func (FakeRPCClient) RunJob

func (c FakeRPCClient) RunJob(ctx context.Context, job *cloudrpc.JobSpec, op ...grpc.CallOption) (*cloudrpc.JobStatus, error)

func (FakeRPCClient) Status

func (c FakeRPCClient) Status(ctx context.Context, job *cloudrpc.JobName, op ...grpc.CallOption) (*cloudrpc.JobStatus, error)