realis

package module
Version: v2.22.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 17, 2020 License: Apache-2.0 Imports: 25 Imported by: 1

README

gorealis GoDoc codecov

Go library for interacting with Aurora Scheduler.

Aurora version compatibility

Please see .auroraversion to see the latest Aurora version against which this library has been tested.

Usage

Projects using gorealis

Contributions

Contributions are always welcome. Please raise an issue to discuss a contribution before it is made.

Documentation

Overview

Collection of monitors to create synchronicity

Package realis provides the ability to use Thrift API to communicate with Apache Aurora.

Index

Constants

View Source
const VERSION = "2.22.1"

Variables

View Source
var ActiveJobUpdateStates = make(map[aurora.JobUpdateStatus]bool)
View Source
var ActiveStates = make(map[aurora.ScheduleStatus]bool)
View Source
var AwaitingPulseJobUpdateStates = make(map[aurora.JobUpdateStatus]bool)
View Source
var LiveStates = make(map[aurora.ScheduleStatus]bool)
View Source
var SlaveAssignedStates = make(map[aurora.ScheduleStatus]bool)
View Source
var TerminalStates = make(map[aurora.ScheduleStatus]bool)

Functions

func ExponentialBackoff

func ExponentialBackoff(backoff Backoff, logger Logger, condition ConditionFunc) error

func GetCerts

func GetCerts(certPath string) (*x509.CertPool, error)

func IsTemporary

func IsTemporary(err error) bool

func IsTimeout

func IsTimeout(err error) bool

func Jitter

func Jitter(duration time.Duration, maxFactor float64) time.Duration

Jitter returns a time.Duration between duration and duration + maxFactor * duration.

This allows clients to avoid converging on periodic behavior. If maxFactor is 0.0, a suggested default value will be chosen.

func LeaderFromZK

func LeaderFromZK(cluster Cluster) (string, error)

Retrieves current Aurora leader from ZK.

func LeaderFromZKOpts

func LeaderFromZKOpts(options ...ZKOpt) (string, error)

Retrieves current Aurora leader from ZK with a custom configuration.

func LoadClusters

func LoadClusters(config string) (map[string]Cluster, error)

Loads clusters.json file traditionally located at /etc/aurora/clusters.json

func MesosFromZKOpts added in v2.22.2

func MesosFromZKOpts(options ...ZKOpt) (string, error)

Retrieves current mesos leader from ZK with a custom configuration.

func TerminalUpdateStates added in v2.22.0

func TerminalUpdateStates() []aurora.JobUpdateStatus

TerminalJobUpdateStates returns a slice containing all the terminal states an update may end up in. This is a function in order to avoid having a slice that can be accidentally mutated.

func ToRetryCount

func ToRetryCount(err error) *retryErr

Helper function for testing verification to avoid whitebox testing as well as keeping retryErr as a private. Should NOT be used under any other context.

Types

type AuroraJob

type AuroraJob struct {
	// contains filtered or unexported fields
}

Structure to collect all information pertaining to an Aurora job.

func NewJob

func NewJob() *AuroraJob

Create a AuroraJob object with everything initialized.

func (*AuroraJob) AddDedicatedConstraint

func (j *AuroraJob) AddDedicatedConstraint(role, name string) *AuroraJob

func (*AuroraJob) AddLabel

func (j *AuroraJob) AddLabel(key string, value string) *AuroraJob

func (*AuroraJob) AddLimitConstraint

func (j *AuroraJob) AddLimitConstraint(name string, limit int32) *AuroraJob

func (*AuroraJob) AddNamedPorts

func (j *AuroraJob) AddNamedPorts(names ...string) *AuroraJob

func (*AuroraJob) AddPorts

func (j *AuroraJob) AddPorts(num int) *AuroraJob

func (*AuroraJob) AddURIs

func (j *AuroraJob) AddURIs(extract bool, cache bool, values ...string) *AuroraJob

func (*AuroraJob) AddValueConstraint

func (j *AuroraJob) AddValueConstraint(name string, negated bool, values ...string) *AuroraJob

func (*AuroraJob) AuroraTask

func (j *AuroraJob) AuroraTask() *AuroraTask

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) BuildThermosPayload

func (j *AuroraJob) BuildThermosPayload() error

func (*AuroraJob) CPU

func (j *AuroraJob) CPU(cpus float64) *AuroraJob

func (*AuroraJob) Container

func (j *AuroraJob) Container(container Container) *AuroraJob

func (*AuroraJob) CronCollisionPolicy

func (j *AuroraJob) CronCollisionPolicy(policy aurora.CronCollisionPolicy) *AuroraJob

func (*AuroraJob) CronSchedule

func (j *AuroraJob) CronSchedule(cron string) *AuroraJob

func (*AuroraJob) Disk

func (j *AuroraJob) Disk(disk int64) *AuroraJob

func (*AuroraJob) Environment

func (j *AuroraJob) Environment(env string) *AuroraJob

Set AuroraJob Key environment. Explicit changes to AuroraTask's job key are not needed because they share a pointer to the same JobKey.

func (*AuroraJob) ExecutorData

func (j *AuroraJob) ExecutorData(data string) *AuroraJob

func (*AuroraJob) ExecutorName

func (j *AuroraJob) ExecutorName(name string) *AuroraJob

func (*AuroraJob) GPU

func (j *AuroraJob) GPU(gpu int64) *AuroraJob

func (*AuroraJob) GetInstanceCount

func (j *AuroraJob) GetInstanceCount() int32

How many instances of the job to run

func (*AuroraJob) InstanceCount

func (j *AuroraJob) InstanceCount(instCount int32) *AuroraJob

How many instances of the job to run

func (*AuroraJob) IsService

func (j *AuroraJob) IsService(isService bool) *AuroraJob

func (*AuroraJob) JobConfig

func (j *AuroraJob) JobConfig() *aurora.JobConfiguration

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) JobKey

func (j *AuroraJob) JobKey() aurora.JobKey

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) MaxFailure

func (j *AuroraJob) MaxFailure(maxFail int32) *AuroraJob

func (*AuroraJob) Name

func (j *AuroraJob) Name(name string) *AuroraJob

Set AuroraJob Key Name.

func (*AuroraJob) PartitionPolicy

func (j *AuroraJob) PartitionPolicy(reschedule bool, delay int64) *AuroraJob

func (*AuroraJob) RAM

func (j *AuroraJob) RAM(ram int64) *AuroraJob

func (*AuroraJob) Role

func (j *AuroraJob) Role(role string) *AuroraJob

Set AuroraJob Key Role.

func (*AuroraJob) TaskConfig

func (j *AuroraJob) TaskConfig() *aurora.TaskConfig

func (*AuroraJob) ThermosExecutor

func (j *AuroraJob) ThermosExecutor(thermos ThermosExecutor) *AuroraJob

func (*AuroraJob) Tier

func (j *AuroraJob) Tier(tier string) *AuroraJob

type AuroraTask

type AuroraTask struct {
	// contains filtered or unexported fields
}

func NewTask

func NewTask() *AuroraTask

func TaskFromThrift

func TaskFromThrift(config *aurora.TaskConfig) *AuroraTask

Helper method to convert aurora.TaskConfig to gorealis AuroraTask type

func (*AuroraTask) AddDedicatedConstraint

func (t *AuroraTask) AddDedicatedConstraint(role, name string) *AuroraTask

From Aurora Docs: dedicated attribute. Aurora treats this specially, and only allows matching jobs to run on these machines, and will only schedule matching jobs on these machines. When a job is created, the scheduler requires that the $role component matches the role field in the job configuration, and will reject the job creation otherwise. A wildcard (*) may be used for the role portion of the dedicated attribute, which will allow any owner to elect for a job to run on the host(s)

func (*AuroraTask) AddLabel

func (t *AuroraTask) AddLabel(key string, value string) *AuroraTask

Adds a Mesos label to the job. Note that Aurora will add the prefix "org.apache.aurora.metadata." to the beginning of each key.

func (*AuroraTask) AddLimitConstraint

func (t *AuroraTask) AddLimitConstraint(name string, limit int32) *AuroraTask

From Aurora Docs: A constraint that specifies the maximum number of active tasks on a host with a matching attribute that may be scheduled simultaneously.

func (*AuroraTask) AddNamedPorts

func (t *AuroraTask) AddNamedPorts(names ...string) *AuroraTask

Add a named port to the job configuration These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraTask) AddPorts

func (t *AuroraTask) AddPorts(num int) *AuroraTask

Adds a request for a number of ports to the job configuration. The names chosen for these ports will be org.apache.aurora.port.X, where X is the current port count for the job configuration starting at 0. These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraTask) AddURIs

func (t *AuroraTask) AddURIs(extract bool, cache bool, values ...string) *AuroraTask

Add a list of URIs with the same extract and cache configuration. Scheduler must have --enable_mesos_fetcher flag enabled. Currently there is no duplicate detection.

func (*AuroraTask) AddValueConstraint

func (t *AuroraTask) AddValueConstraint(name string, negated bool, values ...string) *AuroraTask

From Aurora Docs: Add a Value constraint name - Mesos slave attribute that the constraint is matched against. If negated = true , treat this as a 'not' - to avoid specific values. Values - list of values we look for in attribute name

func (*AuroraTask) BuildThermosPayload

func (t *AuroraTask) BuildThermosPayload() error

func (*AuroraTask) CPU

func (t *AuroraTask) CPU(cpus float64) *AuroraTask

func (*AuroraTask) Clone

func (t *AuroraTask) Clone() *AuroraTask

func (*AuroraTask) Container

func (t *AuroraTask) Container(container Container) *AuroraTask

Set a container to run for the job configuration to run.

func (*AuroraTask) Disk

func (t *AuroraTask) Disk(disk int64) *AuroraTask

func (*AuroraTask) Environment

func (t *AuroraTask) Environment(env string) *AuroraTask

Set AuroraTask Key environment.

func (*AuroraTask) ExecutorData

func (t *AuroraTask) ExecutorData(data string) *AuroraTask

Will be included as part of entire task inside the scheduler that will be serialized.

func (*AuroraTask) ExecutorName

func (t *AuroraTask) ExecutorName(name string) *AuroraTask

Set name of the executor that will the task will be configured to.

func (*AuroraTask) GPU

func (t *AuroraTask) GPU(gpu int64) *AuroraTask

func (*AuroraTask) IsService

func (t *AuroraTask) IsService(isService bool) *AuroraTask

Restart the job's tasks if they fail

func (*AuroraTask) JobKey

func (t *AuroraTask) JobKey() aurora.JobKey

func (*AuroraTask) MaxFailure

func (t *AuroraTask) MaxFailure(maxFail int32) *AuroraTask

How many failures to tolerate before giving up.

func (*AuroraTask) Name

func (t *AuroraTask) Name(name string) *AuroraTask

Set AuroraTask Key Name.

func (*AuroraTask) PartitionPolicy

func (t *AuroraTask) PartitionPolicy(policy aurora.PartitionPolicy) *AuroraTask

Set a partition policy for the job configuration to implement.

func (*AuroraTask) RAM

func (t *AuroraTask) RAM(ram int64) *AuroraTask

func (*AuroraTask) Role

func (t *AuroraTask) Role(role string) *AuroraTask

Set AuroraTask Key Role.

func (*AuroraTask) TaskConfig

func (t *AuroraTask) TaskConfig() *aurora.TaskConfig

func (*AuroraTask) ThermosExecutor

func (t *AuroraTask) ThermosExecutor(thermos ThermosExecutor) *AuroraTask

func (*AuroraTask) Tier

func (t *AuroraTask) Tier(tier string) *AuroraTask

type Backoff

type Backoff struct {
	Duration time.Duration // the base duration
	Factor   float64       // Duration is multipled by factor each iteration
	Jitter   float64       // The amount of jitter applied each iteration
	Steps    int           // Exit with error after this many steps
}

type Client

type Client struct {
	// contains filtered or unexported fields
}

func NewClient

func NewClient(options ...ClientOption) (*Client, error)

This client implementation uses a retry mechanism for all Thrift Calls. It will retry all calls which result in a temporary failure as well as calls that fail due to an EOF being returned by the http client. Most permanent failures are now being caught by the thriftCallWithRetries function and not being retried but there may be corner cases not yet handled.

func (*Client) AbortJobUpdate

func (c *Client) AbortJobUpdate(updateKey aurora.JobUpdateKey, message string) error

Abort AuroraJob Update on Aurora. Requires the updateId which can be obtained on the Aurora web UI.

func (*Client) AddInstances

func (c *Client) AddInstances(instKey aurora.InstanceKey, count int32) error

Scale up the number of instances under a job configuration using the configuration for specific instance to scale up.

func (*Client) Close

func (c *Client) Close()

Releases resources associated with the realis client.

func (*Client) CreateJob

func (c *Client) CreateJob(auroraJob *AuroraJob) error

Sends a create job message to the scheduler with a specific job configuration. Although this API is able to create service jobs, it is better to use CreateService instead as that API uses the update thrift call which has a few extra features available. Use this API to create ad-hoc jobs.

func (*Client) CreateService

func (c *Client) CreateService(update *JobUpdate) (*aurora.StartJobUpdateResult_, error)

This API uses an update thrift call to create the services giving a few more robust features.

func (*Client) DescheduleCronJob

func (c *Client) DescheduleCronJob(key aurora.JobKey) error

func (*Client) DrainHosts

func (c *Client) DrainHosts(hosts ...string) ([]*aurora.HostStatus, error)

Set a list of nodes to DRAINING. This means nothing will be able to be scheduled on them and any existing tasks will be killed and re-scheduled elsewhere in the cluster. Tasks from DRAINING nodes are not guaranteed to return to running unless there is enough capacity in the cluster to run them.

func (*Client) EndMaintenance

func (c *Client) EndMaintenance(hosts ...string) ([]*aurora.HostStatus, error)

func (*Client) FetchTaskConfig

func (c *Client) FetchTaskConfig(instKey aurora.InstanceKey) (*aurora.TaskConfig, error)

Get the task configuration from the aurora scheduler for a job

func (*Client) ForceExplicitTaskReconciliation

func (c *Client) ForceExplicitTaskReconciliation(batchSize *int32) error

Force an Explicit reconciliation between Mesos and Aurora

func (*Client) ForceImplicitTaskReconciliation

func (c *Client) ForceImplicitTaskReconciliation() error

Force an Implicit reconciliation between Mesos and Aurora

func (*Client) GetInstanceIds

func (c *Client) GetInstanceIds(key aurora.JobKey, states []aurora.ScheduleStatus) ([]int32, error)

Uses predefined set of states to retrieve a set of active jobs in Apache Aurora.

func (*Client) GetJobUpdateSummaries

func (c *Client) GetJobUpdateSummaries(jobUpdateQuery *aurora.JobUpdateQuery) (*aurora.GetJobUpdateSummariesResult_, error)

func (*Client) GetJobs

func (c *Client) GetJobs(role string) (*aurora.GetJobsResult_, error)

func (*Client) GetPendingReason

func (c *Client) GetPendingReason(query *aurora.TaskQuery) ([]*aurora.PendingReason, error)

Get pending reason

func (*Client) GetQuota

func (c *Client) GetQuota(role string) (*aurora.GetQuotaResult_, error)

GetQuota returns the resource aggregate for the given role

func (*Client) GetTaskStatus

func (c *Client) GetTaskStatus(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)

Get information about task including a fully hydrated task configuration object

func (*Client) GetTasksWithoutConfigs

func (c *Client) GetTasksWithoutConfigs(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)

Get information about task including without a task configuration object

func (*Client) JobUpdateDetails

func (c *Client) JobUpdateDetails(updateQuery aurora.JobUpdateQuery) ([]*aurora.JobUpdateDetails, error)

func (*Client) KillInstances

func (c *Client) KillInstances(key aurora.JobKey, instances ...int32) (bool, error)

Kill specific instances of a job. Returns true, nil if a task was actually killed as a result of this API call. Returns false, nil if no tasks were killed as a result of this call but there was no error making the call.

func (*Client) KillJob

func (c *Client) KillJob(key aurora.JobKey) error

Sends a kill message to the scheduler for all active tasks under a job.

func (*Client) MaintenanceStatus

func (c *Client) MaintenanceStatus(hosts ...string) (*aurora.MaintenanceStatusResult_, error)

func (*Client) MonitorAutoPausedUpdate added in v2.22.0

func (c *Client) MonitorAutoPausedUpdate(key aurora.JobUpdateKey, interval, timeout time.Duration) (int, error)

AutoPaused monitor is a special monitor for auto pause enabled batch updates. This monitor ensures that the update being monitored is capable of auto pausing and has auto pausing enabled. After verifying this information, the monitor watches for the job to enter the ROLL_FORWARD_PAUSED state and calculates the current batch the update is in using information from the update configuration.

func (*Client) MonitorHostMaintenance

func (c *Client) MonitorHostMaintenance(hosts []string,
	modes []aurora.MaintenanceMode,
	interval, timeout time.Duration) (map[string]bool, error)

Monitor host status until all hosts match the status provided. Returns a map where the value is true if the host is in one of the desired mode(s) or false if it is not as of the time when the monitor exited.

func (*Client) MonitorInstances

func (c *Client) MonitorInstances(key aurora.JobKey, instances int32, interval, timeout time.Duration) (bool, error)

Monitor a AuroraJob until all instances enter one of the LiveStates

func (*Client) MonitorJobUpdate

func (c *Client) MonitorJobUpdate(updateKey aurora.JobUpdateKey, interval, timeout time.Duration) (bool, error)

MonitorJobUpdate polls the scheduler every certain amount of time to see if the update has succeeded. If the update entered a terminal update state but it is not ROLLED_FORWARD, this function will return an error.

func (*Client) MonitorJobUpdateQuery

func (c *Client) MonitorJobUpdateQuery(
	updateQuery aurora.JobUpdateQuery,
	interval time.Duration,
	timeout time.Duration) ([]*aurora.JobUpdateSummary, error)

func (*Client) MonitorJobUpdateStatus

func (c *Client) MonitorJobUpdateStatus(updateKey aurora.JobUpdateKey,
	desiredStatuses []aurora.JobUpdateStatus,
	interval, timeout time.Duration) (aurora.JobUpdateStatus, error)

MonitorJobUpdateStatus polls the scheduler for information about an update until the update enters one of the desired states or until the function times out.

func (*Client) MonitorScheduleStatus

func (c *Client) MonitorScheduleStatus(key aurora.JobKey,
	instanceCount int32,
	desiredStatuses []aurora.ScheduleStatus,
	interval, timeout time.Duration) (bool, error)

Monitor a AuroraJob until all instances enter a desired status. Defaults sets of desired statuses provided by the thrift API include: ActiveStates, SlaveAssignedStates, LiveStates, and TerminalStates

func (*Client) PauseJobUpdate

func (c *Client) PauseJobUpdate(updateKey *aurora.JobUpdateKey, message string) error

Pause AuroraJob Update. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) PerformBackup

func (c *Client) PerformBackup() error

Force Aurora Scheduler to write backup file to a file in the backup directory

func (*Client) PulseJobUpdate

func (c *Client) PulseJobUpdate(updateKey aurora.JobUpdateKey) (aurora.JobUpdatePulseStatus, error)

Pulse AuroraJob Update on Aurora. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) RealisConfig

func (c *Client) RealisConfig() *clientConfig

func (*Client) ReestablishConn

func (c *Client) ReestablishConn() error

func (*Client) RemoveInstances

func (c *Client) RemoveInstances(key aurora.JobKey, count int) error

Scale down the number of instances under a job configuration using the configuration of a specific instance Instances with a higher instance ID will be removed first. For example, if our instance ID list is [0,1,2,3] and we want to remove 2 instances, 2 and 3 will always be picked.

func (*Client) RestartInstances

func (c *Client) RestartInstances(key aurora.JobKey, instances ...int32) error

Restarts specific instances specified

func (*Client) RestartJob

func (c *Client) RestartJob(key aurora.JobKey) error

Restarts all active tasks under a job configuration.

func (*Client) ResumeJobUpdate

func (c *Client) ResumeJobUpdate(updateKey aurora.JobUpdateKey, message string) error

Resume Paused AuroraJob Update. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) RollbackJobUpdate

func (c *Client) RollbackJobUpdate(key aurora.JobUpdateKey, message string) error

func (*Client) SLADrainHosts

func (c *Client) SLADrainHosts(policy *aurora.SlaPolicy, timeout int64, hosts ...string) ([]*aurora.HostStatus, error)

Start SLA Aware Drain. defaultSlaPolicy is the fallback SlaPolicy to use if a task does not have an SlaPolicy. After timeoutSecs, tasks will be forcefully drained without checking SLA.

func (*Client) ScheduleCronJob

func (c *Client) ScheduleCronJob(auroraJob *AuroraJob) error

func (*Client) SetQuota

func (c *Client) SetQuota(role string, cpu *float64, ramMb *int64, diskMb *int64) error

SetQuota sets a quota aggregate for the given role TODO(zircote) Currently investigating an error that is returned from thrift calls that include resources for `NamedPort` and `NumGpu`

func (*Client) Snapshot

func (c *Client) Snapshot() error

Force Aurora Scheduler to perform a snapshot and write to Mesos log

func (*Client) StartCronJob

func (c *Client) StartCronJob(key aurora.JobKey) error

func (*Client) StartJobUpdate

func (c *Client) StartJobUpdate(updateJob *JobUpdate, message string) (*aurora.StartJobUpdateResult_, error)

Update all tasks under a job configuration. Currently gorealis doesn't support for canary deployments.

func (*Client) StartMaintenance

func (c *Client) StartMaintenance(hosts ...string) ([]*aurora.HostStatus, error)

type ClientOption

type ClientOption func(*clientConfig)

func BackOff

func BackOff(b Backoff) ClientOption

func BasicAuth

func BasicAuth(username, password string) ClientOption

clientConfig sets for options in clientConfig.

func CertsPath

func CertsPath(certspath string) ClientOption

func ClientCerts

func ClientCerts(clientKey, clientCert string) ClientOption

func Debug

func Debug() ClientOption

Enable debug statements.

func FailOnPermanentErrors

func FailOnPermanentErrors() ClientOption

FailOnPermanentErrors - If the client encounters a connection error the standard library considers permanent, stop retrying and return an error to the user.

func InsecureSkipVerify

func InsecureSkipVerify(InsecureSkipVerify bool) ClientOption

func SchedulerUrl

func SchedulerUrl(url string) ClientOption

func SetLogger

func SetLogger(l Logger) ClientOption

Using the word set to avoid name collision with Interface.

func ThriftBinary

func ThriftBinary() ClientOption

func ThriftJSON

func ThriftJSON() ClientOption

func Timeout

func Timeout(timeout time.Duration) ClientOption

func Trace

func Trace() ClientOption

Enable trace statements.

func ZKCluster

func ZKCluster(cluster *Cluster) ClientOption

func ZKUrl

func ZKUrl(url string) ClientOption

func ZookeeperOptions

func ZookeeperOptions(opts ...ZKOpt) ClientOption

Use this option if you'd like to override default settings for connecting to Zookeeper. See zk.go for what is possible to set as an option.

type Cluster

type Cluster struct {
	Name          string `json:"name"`
	AgentRoot     string `json:"slave_root"`
	AgentRunDir   string `json:"slave_run_directory"`
	ZK            string `json:"zk"`
	ZKPort        int    `json:"zk_port"`
	SchedZKPath   string `json:"scheduler_zk_path"`
	SchedURI      string `json:"scheduler_uri"`
	ProxyURL      string `json:"proxy_url"`
	AuthMechanism string `json:"auth_mechanism"`
}

func GetDefaultClusterFromZKUrl

func GetDefaultClusterFromZKUrl(zkURL string) *Cluster

type ConditionFunc

type ConditionFunc func() (done bool, err error)

ConditionFunc returns true if the condition is satisfied, or an error if the loop should be aborted.

type Container

type Container interface {
	Build() *aurora.Container
}

type DockerContainer

type DockerContainer struct {
	// contains filtered or unexported fields
}

func NewDockerContainer

func NewDockerContainer() *DockerContainer

func (*DockerContainer) AddParameter

func (c *DockerContainer) AddParameter(name, value string) *DockerContainer

func (*DockerContainer) Build

func (c *DockerContainer) Build() *aurora.Container

func (*DockerContainer) Image

func (c *DockerContainer) Image(image string) *DockerContainer

type Endpoint

type Endpoint struct {
	Host string `json:"host"`
	Port int    `json:"port"`
}

type JobUpdate

type JobUpdate struct {
	// contains filtered or unexported fields
}

Structure to collect all information required to create job update

func JobUpdateFromAuroraTask

func JobUpdateFromAuroraTask(task *AuroraTask) *JobUpdate

Creates an update with default values using an AuroraTask as the underlying task configuration. This function has a high level understanding of Aurora Tasks and thus will support copying a task that is configured to use Thermos.

func JobUpdateFromConfig

func JobUpdateFromConfig(task *aurora.TaskConfig) *JobUpdate

JobUpdateFromConfig creates an update with default values using an aurora.TaskConfig primitive as the underlying task configuration. This function should not be used unless the implications of using a primitive value are understood. For example, the primitive has no concept of Thermos.

func NewJobUpdate

func NewJobUpdate() *JobUpdate

Create a default JobUpdate object with an empty task and no fields filled in.

func (*JobUpdate) AddDedicatedConstraint

func (j *JobUpdate) AddDedicatedConstraint(role, name string) *JobUpdate

func (*JobUpdate) AddInstanceRange added in v2.22.1

func (j *JobUpdate) AddInstanceRange(first, last int32) *JobUpdate

AddInstanceRange allows updates to only touch a certain specific range of instances

func (*JobUpdate) AddLabel

func (j *JobUpdate) AddLabel(key string, value string) *JobUpdate

func (*JobUpdate) AddLimitConstraint

func (j *JobUpdate) AddLimitConstraint(name string, limit int32) *JobUpdate

func (*JobUpdate) AddNamedPorts

func (j *JobUpdate) AddNamedPorts(names ...string) *JobUpdate

func (*JobUpdate) AddPorts

func (j *JobUpdate) AddPorts(num int) *JobUpdate

func (*JobUpdate) AddURIs

func (j *JobUpdate) AddURIs(extract bool, cache bool, values ...string) *JobUpdate

func (*JobUpdate) AddValueConstraint

func (j *JobUpdate) AddValueConstraint(name string, negated bool, values ...string) *JobUpdate

func (*JobUpdate) BatchSize

func (j *JobUpdate) BatchSize(size int32) *JobUpdate

Max number of instances being updated at any given moment.

func (*JobUpdate) BatchUpdateStrategy added in v2.22.0

func (j *JobUpdate) BatchUpdateStrategy(autoPause bool, batchSize int32) *JobUpdate

func (*JobUpdate) BuildThermosPayload

func (j *JobUpdate) BuildThermosPayload() error

func (*JobUpdate) CPU

func (j *JobUpdate) CPU(cpus float64) *JobUpdate

func (*JobUpdate) Container

func (j *JobUpdate) Container(container Container) *JobUpdate

func (*JobUpdate) Disk

func (j *JobUpdate) Disk(disk int64) *JobUpdate

func (*JobUpdate) Environment

func (j *JobUpdate) Environment(env string) *JobUpdate

func (*JobUpdate) ExecutorData

func (j *JobUpdate) ExecutorData(data string) *JobUpdate

func (*JobUpdate) ExecutorName

func (j *JobUpdate) ExecutorName(name string) *JobUpdate

func (*JobUpdate) InstanceCount

func (j *JobUpdate) InstanceCount(inst int32) *JobUpdate

Set instance count the job will have after the update.

func (*JobUpdate) IsService

func (j *JobUpdate) IsService(isService bool) *JobUpdate

func (*JobUpdate) JobKey

func (j *JobUpdate) JobKey() aurora.JobKey

func (*JobUpdate) MaxFailedInstances

func (j *JobUpdate) MaxFailedInstances(inst int32) *JobUpdate

Max number of FAILED instances to tolerate before terminating the update.

func (*JobUpdate) MaxPerInstanceFailures

func (j *JobUpdate) MaxPerInstanceFailures(inst int32) *JobUpdate

Max number of instance failures to tolerate before marking instance as FAILED.

func (*JobUpdate) Name

func (j *JobUpdate) Name(name string) *JobUpdate

func (*JobUpdate) PartitionPolicy

func (j *JobUpdate) PartitionPolicy(reschedule bool, delay int64) *JobUpdate

func (*JobUpdate) PulseIntervalTimeout

func (j *JobUpdate) PulseIntervalTimeout(timeout time.Duration) *JobUpdate

Sets the interval at which pulses should be received by the job update before timing out.

func (*JobUpdate) QueueUpdateStrategy added in v2.22.0

func (j *JobUpdate) QueueUpdateStrategy(groupSize int32) *JobUpdate

func (*JobUpdate) RAM

func (j *JobUpdate) RAM(ram int64) *JobUpdate

func (*JobUpdate) Role

func (j *JobUpdate) Role(role string) *JobUpdate

func (*JobUpdate) RollbackOnFail

func (j *JobUpdate) RollbackOnFail(rollback bool) *JobUpdate

When False, prevents auto rollback of a failed update.

func (*JobUpdate) SlaAware added in v2.22.1

func (j *JobUpdate) SlaAware(slaAware bool) *JobUpdate

SlaAware makes the scheduler enforce the SLA Aware policy if the job meets the SLA awareness criteria. By default, the scheduler will only apply SLA Awareness to jobs in the production tier with 20 or more instances.

func (*JobUpdate) TaskConfig

func (j *JobUpdate) TaskConfig() *aurora.TaskConfig

func (*JobUpdate) TaskMaxFailure added in v2.22.1

func (j *JobUpdate) TaskMaxFailure(maxFail int32) *JobUpdate

func (*JobUpdate) ThermosExecutor

func (j *JobUpdate) ThermosExecutor(thermos ThermosExecutor) *JobUpdate

func (*JobUpdate) Tier

func (j *JobUpdate) Tier(tier string) *JobUpdate

func (*JobUpdate) VariableBatchStrategy added in v2.22.0

func (j *JobUpdate) VariableBatchStrategy(autoPause bool, batchSizes ...int32) *JobUpdate

func (*JobUpdate) WaitForBatchCompletion

func (j *JobUpdate) WaitForBatchCompletion(batchWait bool) *JobUpdate

Wait for all instances in a group to be done before moving on.

func (*JobUpdate) WatchTime

func (j *JobUpdate) WatchTime(timeout time.Duration) *JobUpdate

Minimum number of seconds a shard must remain in RUNNING state before considered a success.

type LevelLogger

type LevelLogger struct {
	Logger
	// contains filtered or unexported fields
}

func (LevelLogger) DebugPrint

func (l LevelLogger) DebugPrint(a ...interface{})

func (LevelLogger) DebugPrintf

func (l LevelLogger) DebugPrintf(format string, a ...interface{})

func (LevelLogger) DebugPrintln

func (l LevelLogger) DebugPrintln(a ...interface{})

func (*LevelLogger) EnableDebug

func (l *LevelLogger) EnableDebug(enable bool)

func (*LevelLogger) EnableTrace

func (l *LevelLogger) EnableTrace(enable bool)

func (LevelLogger) TracePrint

func (l LevelLogger) TracePrint(a ...interface{})

func (LevelLogger) TracePrintf

func (l LevelLogger) TracePrintf(format string, a ...interface{})

func (LevelLogger) TracePrintln

func (l LevelLogger) TracePrintln(a ...interface{})

type Logger

type Logger interface {
	Println(v ...interface{})
	Printf(format string, v ...interface{})
	Print(v ...interface{})
}

type MesosAddress added in v2.22.2

type MesosAddress struct {
	Hostname string `json:"hostname"`
	IP       string `json:"ip"`
	Port     uint16 `json:"port"`
}

type MesosContainer

type MesosContainer struct {
	// contains filtered or unexported fields
}

func NewMesosContainer

func NewMesosContainer() *MesosContainer

func (*MesosContainer) AddVolume

func (c *MesosContainer) AddVolume(hostPath, containerPath string, mode aurora.Mode) *MesosContainer

func (*MesosContainer) AppcImage

func (c *MesosContainer) AppcImage(name, imageId string) *MesosContainer

func (*MesosContainer) Build

func (c *MesosContainer) Build() *aurora.Container

func (*MesosContainer) DockerImage

func (c *MesosContainer) DockerImage(name, tag string) *MesosContainer

type MesosInstance added in v2.22.2

type MesosInstance struct {
	Address MesosAddress `json:"address"`
	Version string       `json:"version"`
}

MesosInstance is defined for mesos json stored in ZK.

type NoopLogger

type NoopLogger struct{}

func (NoopLogger) Print

func (NoopLogger) Print(a ...interface{})

func (NoopLogger) Printf

func (NoopLogger) Printf(format string, a ...interface{})

func (NoopLogger) Println

func (NoopLogger) Println(a ...interface{})

type ResourceType

type ResourceType int
const (
	CPU ResourceType = iota
	RAM
	DISK
	GPU
)

type ServiceInstance

type ServiceInstance struct {
	Service             Endpoint            `json:"serviceEndpoint"`
	AdditionalEndpoints map[string]Endpoint `json:"additionalEndpoints"`
	Status              string              `json:"status"`
}

type TemporaryErr

type TemporaryErr struct {
	// contains filtered or unexported fields
}

func NewTemporaryError

func NewTemporaryError(err error) *TemporaryErr

Retrying after receiving this error is advised

func (*TemporaryErr) Temporary

func (t *TemporaryErr) Temporary() bool

type ThermosConstraint

type ThermosConstraint struct {
	Order []string `json:"order,omitempty"`
}

type ThermosExecutor

type ThermosExecutor struct {
	Task ThermosTask `json:"task""`
	// contains filtered or unexported fields
}

func (*ThermosExecutor) AddProcess

func (t *ThermosExecutor) AddProcess(process ThermosProcess) *ThermosExecutor

Processes must have unique names. Adding a process whose name already exists will result in overwriting the previous version of the process.

func (*ThermosExecutor) Clone

func (t *ThermosExecutor) Clone() *ThermosExecutor

Deep copy of Thermos executor

func (*ThermosExecutor) ProcessOrder

func (t *ThermosExecutor) ProcessOrder(order ...string) *ThermosExecutor

Order in which the Processes should be executed. Index 0 will be executed first, index N will be executed last.

type ThermosProcess

type ThermosProcess struct {
	Name        string `json:"name"`
	Cmdline     string `json:"cmdline"`
	Daemon      bool   `json:"daemon"`
	Ephemeral   bool   `json:"ephemeral"`
	MaxFailures int    `json:"max_failures"`
	MinDuration int    `json:"min_duration"`
	Final       bool   `json:"final"`
}

func NewThermosProcess

func NewThermosProcess(name, command string) ThermosProcess

type ThermosTask

type ThermosTask struct {
	Processes   map[string]*ThermosProcess `json:"processes"`
	Constraints []*ThermosConstraint       `json:"constraints"`
	Resources   thermosResources           `json:"resources"`
}

func (*ThermosTask) MarshalJSON

func (t *ThermosTask) MarshalJSON() ([]byte, error)

Custom Marshaling for Thermos Task to match what Thermos expects

func (*ThermosTask) UnmarshalJSON

func (t *ThermosTask) UnmarshalJSON(data []byte) error

Custom Unmarshaling to match what Thermos would contain

type TransportProtocol

type TransportProtocol int

type ZKOpt

type ZKOpt func(z *zkConfig)

func ZKBackoff

func ZKBackoff(b Backoff) ZKOpt

func ZKEndpoints

func ZKEndpoints(endpoints ...string) ZKOpt

func ZKLogger

func ZKLogger(l Logger) ZKOpt

func ZKPath

func ZKPath(path string) ZKOpt

func ZKTimeout

func ZKTimeout(d time.Duration) ZKOpt

Directories

Path Synopsis
gen-go
Helper functions to process aurora.Response
Helper functions to process aurora.Response

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL