realis

package module
v2.30.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 26, 2023 License: Apache-2.0 Imports: 26 Imported by: 1

README

gorealis GoDoc codecov

Go library for interacting with Aurora Scheduler.

Aurora version compatibility

Please see .auroraversion to see the latest Aurora version against which this library has been tested.

Usage

Projects using gorealis

Contributions

Contributions are always welcome. Please raise an issue to discuss a contribution before it is made.

Documentation

Overview

Collection of monitors to create synchronicity

Package realis provides the ability to use Thrift API to communicate with Apache Aurora.

Index

Constants

View Source 
const VERSION = "2.28.0"

Variables

View Source 
var ActiveJobUpdateStates = make(map[aurora.JobUpdateStatus]bool)
View Source 
var ActiveStates = make(map[aurora.ScheduleStatus]bool)
View Source 
var AwaitingPulseJobUpdateStates = make(map[aurora.JobUpdateStatus]bool)
View Source 
var LiveStates = make(map[aurora.ScheduleStatus]bool)
View Source 
var SlaveAssignedStates = make(map[aurora.ScheduleStatus]bool)
View Source 
var TerminalStates = make(map[aurora.ScheduleStatus]bool)

Functions

func ExponentialBackoff 

func ExponentialBackoff(backoff Backoff, logger Logger, condition ConditionFunc) error

ExponentialBackoff is a modified version of the Kubernetes exponential-backoff code. It repeats a condition check with exponential backoff and checks the condition up to Steps times, increasing the wait by multiplying the previous duration by Factor.

If Jitter is greater than zero, a random amount of each duration is added (between duration and duration*(1+jitter)).

If the condition never returns true, ErrWaitTimeout is returned. Errors do not cause the function to return.

func GetCerts 

func GetCerts(certPath string) (*x509.CertPool, error)

func IsTemporary 

func IsTemporary(err error) bool

func IsTimeout 

func IsTimeout(err error) bool

func Jitter 

func Jitter(duration time.Duration, maxFactor float64) time.Duration

Jitter returns a time.Duration between duration and duration + maxFactor * duration.

This allows clients to avoid converging on periodic behavior. If maxFactor is 0.0, a suggested default value will be chosen.

func LeaderFromZK 

func LeaderFromZK(cluster Cluster) (string, error)

Retrieves current Aurora leader from ZK.

func LeaderFromZKOpts 

func LeaderFromZKOpts(options ...ZKOpt) (string, error)

Retrieves current Aurora leader from ZK with a custom configuration.

func LoadClusters 

func LoadClusters(config string) (map[string]Cluster, error)

Loads clusters.json file traditionally located at /etc/aurora/clusters.json

func MasterNodesFromZK added in v2.29.0 

func MasterNodesFromZK(cluster Cluster) (map[string][]string, error)

Retrieves current Aurora master nodes from ZK.

func MasterNodesFromZKOpts added in v2.29.0 

func MasterNodesFromZKOpts(options ...ZKOpt) (map[string][]string, error)

Retrieves current Mesos master nodes/leader from ZK with a custom configuration.

func MesosFromZKOpts added in v2.22.2 

func MesosFromZKOpts(options ...ZKOpt) (string, error)

Retrieves current mesos leader from ZK with a custom configuration.

func MesosMasterNodesFromZK added in v2.29.0 

func MesosMasterNodesFromZK(cluster Cluster) (map[string][]string, error)

Retrieves current Mesos Aurora master nodes from ZK.

func MesosMasterNodesFromZKOpts added in v2.29.0 

func MesosMasterNodesFromZKOpts(options ...ZKOpt) (map[string][]string, error)

Retrieves current mesos master nodes/leader from ZK with a custom configuration.

func ResourcesToMap added in v2.28.0 

func ResourcesToMap(resources []*aurora.Resource) map[string]float64

func TerminalUpdateStates added in v2.22.0 

func TerminalUpdateStates() []aurora.JobUpdateStatus

TerminalUpdateStates returns a slice containing all the terminal states an update may be in. This is a function in order to avoid having a slice that can be accidentally mutated.

func ToRetryCount 

func ToRetryCount(err error) *retryErr

Helper function for testing verification to avoid whitebox testing as well as keeping retryErr as a private. Should NOT be used under any other context.

Types

type AuroraJob 

type AuroraJob struct {
	// contains filtered or unexported fields
}

Structure to collect all information pertaining to an Aurora job.

func NewJob 

func NewJob() *AuroraJob

Create a AuroraJob object with everything initialized.

func (*AuroraJob) AddDedicatedConstraint 

func (j *AuroraJob) AddDedicatedConstraint(role, name string) *AuroraJob

func (*AuroraJob) AddLabel 

func (j *AuroraJob) AddLabel(key string, value string) *AuroraJob

func (*AuroraJob) AddLimitConstraint 

func (j *AuroraJob) AddLimitConstraint(name string, limit int32) *AuroraJob

func (*AuroraJob) AddNamedPorts 

func (j *AuroraJob) AddNamedPorts(names ...string) *AuroraJob

func (*AuroraJob) AddPorts 

func (j *AuroraJob) AddPorts(num int) *AuroraJob

func (*AuroraJob) AddURIs 

func (j *AuroraJob) AddURIs(extract bool, cache bool, values ...string) *AuroraJob

func (*AuroraJob) AddValueConstraint 

func (j *AuroraJob) AddValueConstraint(name string, negated bool, values ...string) *AuroraJob

func (*AuroraJob) AuroraTask 

func (j *AuroraJob) AuroraTask() *AuroraTask

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) BuildThermosPayload 

func (j *AuroraJob) BuildThermosPayload() error

func (*AuroraJob) CPU 

func (j *AuroraJob) CPU(cpus float64) *AuroraJob

func (*AuroraJob) Container 

func (j *AuroraJob) Container(container Container) *AuroraJob

func (*AuroraJob) CronCollisionPolicy 

func (j *AuroraJob) CronCollisionPolicy(policy aurora.CronCollisionPolicy) *AuroraJob

func (*AuroraJob) CronSchedule 

func (j *AuroraJob) CronSchedule(cron string) *AuroraJob

func (*AuroraJob) Disk 

func (j *AuroraJob) Disk(disk int64) *AuroraJob

func (*AuroraJob) Environment 

func (j *AuroraJob) Environment(env string) *AuroraJob

Set AuroraJob Key environment. Explicit changes to AuroraTask's job key are not needed because they share a pointer to the same JobKey.

func (*AuroraJob) ExecutorData 

func (j *AuroraJob) ExecutorData(data string) *AuroraJob

func (*AuroraJob) ExecutorName 

func (j *AuroraJob) ExecutorName(name string) *AuroraJob

func (*AuroraJob) GPU 

func (j *AuroraJob) GPU(gpu int64) *AuroraJob

func (*AuroraJob) GetInstanceCount 

func (j *AuroraJob) GetInstanceCount() int32

How many instances of the job to run

func (*AuroraJob) InstanceCount 

func (j *AuroraJob) InstanceCount(instCount int32) *AuroraJob

How many instances of the job to run

func (*AuroraJob) IsService 

func (j *AuroraJob) IsService(isService bool) *AuroraJob

func (*AuroraJob) JobConfig 

func (j *AuroraJob) JobConfig() *aurora.JobConfiguration

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) JobKey 

func (j *AuroraJob) JobKey() aurora.JobKey

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) MaxFailure 

func (j *AuroraJob) MaxFailure(maxFail int32) *AuroraJob

func (*AuroraJob) Name 

func (j *AuroraJob) Name(name string) *AuroraJob

Set AuroraJob Key Name.

func (*AuroraJob) PartitionPolicy 

func (j *AuroraJob) PartitionPolicy(reschedule bool, delay int64) *AuroraJob

func (*AuroraJob) Priority added in v2.25.0 

func (j *AuroraJob) Priority(priority int32) *AuroraJob

func (*AuroraJob) Production added in v2.26.0 

func (j *AuroraJob) Production(production bool) *AuroraJob

func (*AuroraJob) RAM 

func (j *AuroraJob) RAM(ram int64) *AuroraJob

func (*AuroraJob) Role 

func (j *AuroraJob) Role(role string) *AuroraJob

Set AuroraJob Key Role.

func (*AuroraJob) TaskConfig 

func (j *AuroraJob) TaskConfig() *aurora.TaskConfig

func (*AuroraJob) ThermosExecutor 

func (j *AuroraJob) ThermosExecutor(thermos ThermosExecutor) *AuroraJob

func (*AuroraJob) Tier 

func (j *AuroraJob) Tier(tier string) *AuroraJob

type AuroraTask 

type AuroraTask struct {
	// contains filtered or unexported fields
}

func NewTask 

func NewTask() *AuroraTask

func TaskFromThrift 

func TaskFromThrift(config *aurora.TaskConfig) *AuroraTask

Helper method to convert aurora.TaskConfig to gorealis AuroraTask type

func (*AuroraTask) AddDedicatedConstraint 

func (t *AuroraTask) AddDedicatedConstraint(role, name string) *AuroraTask

From Aurora Docs: dedicated attribute. Aurora treats this specially, and only allows matching jobs to run on these machines, and will only schedule matching jobs on these machines. When a job is created, the scheduler requires that the $role component matches the role field in the job configuration, and will reject the job creation otherwise. A wildcard (*) may be used for the role portion of the dedicated attribute, which will allow any owner to elect for a job to run on the host(s)

func (*AuroraTask) AddLabel 

func (t *AuroraTask) AddLabel(key string, value string) *AuroraTask

Adds a Mesos label to the job. Note that Aurora will add the prefix "org.apache.aurora.metadata." to the beginning of each key.

func (*AuroraTask) AddLimitConstraint 

func (t *AuroraTask) AddLimitConstraint(name string, limit int32) *AuroraTask

From Aurora Docs: A constraint that specifies the maximum number of active tasks on a host with a matching attribute that may be scheduled simultaneously.

func (*AuroraTask) AddNamedPorts 

func (t *AuroraTask) AddNamedPorts(names ...string) *AuroraTask

Add a named port to the job configuration These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraTask) AddPorts 

func (t *AuroraTask) AddPorts(num int) *AuroraTask

Adds a request for a number of ports to the job configuration. The names chosen for these ports will be org.apache.aurora.port.X, where X is the current port count for the job configuration starting at 0. These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraTask) AddURIs 

func (t *AuroraTask) AddURIs(extract bool, cache bool, values ...string) *AuroraTask

Add a list of URIs with the same extract and cache configuration. Scheduler must have --enable_mesos_fetcher flag enabled. Currently there is no duplicate detection.

func (*AuroraTask) AddValueConstraint 

func (t *AuroraTask) AddValueConstraint(name string, negated bool, values ...string) *AuroraTask

From Aurora Docs: Add a Value constraint name - Mesos slave attribute that the constraint is matched against. If negated = true , treat this as a 'not' - to avoid specific values. Values - list of values we look for in attribute name

func (*AuroraTask) BuildThermosPayload 

func (t *AuroraTask) BuildThermosPayload() error

func (*AuroraTask) CPU 

func (t *AuroraTask) CPU(cpus float64) *AuroraTask

func (*AuroraTask) Clone 

func (t *AuroraTask) Clone() *AuroraTask

func (*AuroraTask) Container 

func (t *AuroraTask) Container(container Container) *AuroraTask

Set a container to run for the job configuration to run.

func (*AuroraTask) Disk 

func (t *AuroraTask) Disk(disk int64) *AuroraTask

func (*AuroraTask) Environment 

func (t *AuroraTask) Environment(env string) *AuroraTask

Set AuroraTask Key environment.

func (*AuroraTask) ExecutorData 

func (t *AuroraTask) ExecutorData(data string) *AuroraTask

Will be included as part of entire task inside the scheduler that will be serialized.

func (*AuroraTask) ExecutorName 

func (t *AuroraTask) ExecutorName(name string) *AuroraTask

Set name of the executor that will the task will be configured to.

func (*AuroraTask) GPU 

func (t *AuroraTask) GPU(gpu int64) *AuroraTask

func (*AuroraTask) IsService 

func (t *AuroraTask) IsService(isService bool) *AuroraTask

Restart the job's tasks if they fail

func (*AuroraTask) JobKey 

func (t *AuroraTask) JobKey() aurora.JobKey

func (*AuroraTask) MaxFailure 

func (t *AuroraTask) MaxFailure(maxFail int32) *AuroraTask

How many failures to tolerate before giving up.

func (*AuroraTask) Name 

func (t *AuroraTask) Name(name string) *AuroraTask

Set AuroraTask Key Name.

func (*AuroraTask) PartitionPolicy 

func (t *AuroraTask) PartitionPolicy(policy aurora.PartitionPolicy) *AuroraTask

Set a partition policy for the job configuration to implement.

func (*AuroraTask) Priority added in v2.25.0 

func (t *AuroraTask) Priority(priority int32) *AuroraTask

set priority for preemption or priority-queueing

func (*AuroraTask) Production added in v2.26.0 

func (t *AuroraTask) Production(production bool) *AuroraTask

func (*AuroraTask) RAM 

func (t *AuroraTask) RAM(ram int64) *AuroraTask

func (*AuroraTask) Role 

func (t *AuroraTask) Role(role string) *AuroraTask

Set AuroraTask Key Role.

func (*AuroraTask) TaskConfig 

func (t *AuroraTask) TaskConfig() *aurora.TaskConfig

func (*AuroraTask) ThermosExecutor 

func (t *AuroraTask) ThermosExecutor(thermos ThermosExecutor) *AuroraTask

func (*AuroraTask) Tier 

func (t *AuroraTask) Tier(tier string) *AuroraTask

type Backoff 

type Backoff struct {
	Duration time.Duration // the base duration
	Factor   float64       // Duration is multiplied by a factor each iteration
	Jitter   float64       // The amount of jitter applied each iteration
	Steps    int           // Exit with error after this many steps
}

Backoff determines how the retry mechanism should react after each failure and how many failures it should tolerate.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(options ...ClientOption) (*Client, error)

This client implementation uses a retry mechanism for all Thrift Calls. It will retry all calls which result in a temporary failure as well as calls that fail due to an EOF being returned by the http client. Most permanent failures are now being caught by the thriftCallWithRetries function and not being retried but there may be corner cases not yet handled.

func (*Client) AbortJobUpdate 

func (c *Client) AbortJobUpdate(updateKey aurora.JobUpdateKey, message string) error

AbortJobUpdate terminates a job update in the scheduler. It requires the updateId which can be obtained on the Aurora web UI. This API is meant to be synchronous. It will attempt to wait until the update transitions to the aborted state. However, if the job update does not transition to the ABORT state an error will be returned.

func (*Client) AddInstances 

func (c *Client) AddInstances(instKey aurora.InstanceKey, count int32) error

Scale up the number of instances under a job configuration using the configuration for specific instance to scale up.

func (*Client) AvailOfferReport added in v2.28.0 

func (c *Client) AvailOfferReport() (OfferReport, error)

AvailOfferReport returns a detailed summary of offers available for use. For example, 2 nodes offer 32 cpus and 10 nodes offer 1 cpus.

func (*Client) Close 

func (c *Client) Close()

Releases resources associated with the realis client.

func (*Client) CreateJob 

func (c *Client) CreateJob(auroraJob *AuroraJob) error

Sends a create job message to the scheduler with a specific job configuration. Although this API is able to create service jobs, it is better to use CreateService instead as that API uses the update thrift call which has a few extra features available. Use this API to create ad-hoc jobs.

func (*Client) CreateService 

func (c *Client) CreateService(update *JobUpdate) (*aurora.StartJobUpdateResult_, error)

This API uses an update thrift call to create the services giving a few more robust features.

func (*Client) DescheduleCronJob 

func (c *Client) DescheduleCronJob(key aurora.JobKey) error

func (*Client) DrainHosts 

func (c *Client) DrainHosts(hosts ...string) ([]*aurora.HostStatus, error)

Set a list of nodes to DRAINING. This means nothing will be able to be scheduled on them and any existing tasks will be killed and re-scheduled elsewhere in the cluster. Tasks from DRAINING nodes are not guaranteed to return to running unless there is enough capacity in the cluster to run them.

func (*Client) EndMaintenance 

func (c *Client) EndMaintenance(hosts ...string) ([]*aurora.HostStatus, error)

func (*Client) FetchTaskConfig 

func (c *Client) FetchTaskConfig(instKey aurora.InstanceKey) (*aurora.TaskConfig, error)

Get the task configuration from the aurora scheduler for a job

func (*Client) FitTasks added in v2.28.0 

func (c *Client) FitTasks(taskConfig *aurora.TaskConfig, offers []Offer) (int64, error)

FitTasks computes the number tasks can be fit in a list of offer

func (*Client) ForceExplicitTaskReconciliation 

func (c *Client) ForceExplicitTaskReconciliation(batchSize *int32) error

Force an Explicit reconciliation between Mesos and Aurora

func (*Client) ForceImplicitTaskReconciliation 

func (c *Client) ForceImplicitTaskReconciliation() error

Force an Implicit reconciliation between Mesos and Aurora

func (*Client) GetInstanceIds 

func (c *Client) GetInstanceIds(key aurora.JobKey, states []aurora.ScheduleStatus) ([]int32, error)

Uses predefined set of states to retrieve a set of active jobs in Apache Aurora.

func (*Client) GetJobSummary added in v2.25.0 

func (c *Client) GetJobSummary(role string) (*aurora.JobSummaryResult_, error)

func (*Client) GetJobUpdateSummaries 

func (c *Client) GetJobUpdateSummaries(jobUpdateQuery *aurora.JobUpdateQuery) (*aurora.GetJobUpdateSummariesResult_, error)

func (*Client) GetJobs 

func (c *Client) GetJobs(role string) (*aurora.GetJobsResult_, error)

func (*Client) GetPendingReason 

func (c *Client) GetPendingReason(query *aurora.TaskQuery) ([]*aurora.PendingReason, error)

Get pending reason

func (*Client) GetQuota 

func (c *Client) GetQuota(role string) (*aurora.GetQuotaResult_, error)

GetQuota returns the resource aggregate for the given role

func (*Client) GetSchedulerURL added in v2.28.0 

func (c *Client) GetSchedulerURL() string

func (*Client) GetTaskStatus 

func (c *Client) GetTaskStatus(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)

Get information about task including a fully hydrated task configuration object

func (*Client) GetTasksWithoutConfigs 

func (c *Client) GetTasksWithoutConfigs(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)

GetTasksWithoutConfigs gets information about task including without a task configuration object. This is a more lightweight version of GetTaskStatus but contains less information as a result.

func (*Client) JobExists added in v2.30.0 

func (r *Client) JobExists(key aurora.JobKey) (bool, error)

func (*Client) JobUpdateDetails 

func (c *Client) JobUpdateDetails(updateQuery aurora.JobUpdateQuery) ([]*aurora.JobUpdateDetails, error)

func (*Client) KillInstances 

func (c *Client) KillInstances(key aurora.JobKey, instances ...int32) (bool, error)

Kill specific instances of a job. Returns true, nil if a task was actually killed as a result of this API call. Returns false, nil if no tasks were killed as a result of this call but there was no error making the call.

func (*Client) KillJob 

func (c *Client) KillJob(key aurora.JobKey) error

Sends a kill message to the scheduler for all active tasks under a job.

func (*Client) MaintenanceHosts added in v2.28.0 

func (c *Client) MaintenanceHosts() ([]string, error)

MaintenanceHosts list all the hosts under maintenance

func (*Client) MaintenanceStatus 

func (c *Client) MaintenanceStatus(hosts ...string) (*aurora.MaintenanceStatusResult_, error)

func (*Client) MonitorAutoPausedUpdate added in v2.22.0 

func (c *Client) MonitorAutoPausedUpdate(key aurora.JobUpdateKey, interval, timeout time.Duration) (int, error)

MonitorAutoPausedUpdate is a special monitor for auto pause enabled batch updates. This monitor ensures that the update being monitored is capable of auto pausing and has auto pausing enabled. After verifying this information, the monitor watches for the job to enter the ROLL_FORWARD_PAUSED state and calculates the current batch the update is in using information from the update configuration.

func (*Client) MonitorHostMaintenance 

func (c *Client) MonitorHostMaintenance(hosts []string,
	modes []aurora.MaintenanceMode,
	interval, timeout time.Duration) (map[string]bool, error)

Monitor host status until all hosts match the status provided. Returns a map where the value is true if the host is in one of the desired mode(s) or false if it is not as of the time when the monitor exited.

func (*Client) MonitorInstances 

func (c *Client) MonitorInstances(key aurora.JobKey, instances int32, interval, timeout time.Duration) (bool, error)

Monitor a AuroraJob until all instances enter one of the LiveStates

func (*Client) MonitorJobUpdate 

func (c *Client) MonitorJobUpdate(updateKey aurora.JobUpdateKey, interval, timeout time.Duration) (bool, error)

MonitorJobUpdate polls the scheduler every certain amount of time to see if the update has succeeded. If the update entered a terminal update state but it is not ROLLED_FORWARD, this function will return an error.

func (*Client) MonitorJobUpdateQuery 

func (c *Client) MonitorJobUpdateQuery(
	updateQuery aurora.JobUpdateQuery,
	interval time.Duration,
	timeout time.Duration) ([]*aurora.JobUpdateSummary, error)

func (*Client) MonitorJobUpdateStatus 

func (c *Client) MonitorJobUpdateStatus(updateKey aurora.JobUpdateKey,
	desiredStatuses []aurora.JobUpdateStatus,
	interval, timeout time.Duration) (aurora.JobUpdateStatus, error)

MonitorJobUpdateStatus polls the scheduler for information about an update until the update enters one of the desired states or until the function times out.

func (*Client) MonitorScheduleStatus 

func (c *Client) MonitorScheduleStatus(key aurora.JobKey,
	instanceCount int32,
	desiredStatuses []aurora.ScheduleStatus,
	interval, timeout time.Duration) (bool, error)

Monitor a AuroraJob until all instances enter a desired status. Defaults sets of desired statuses provided by the thrift API include: ActiveStates, SlaveAssignedStates, LiveStates, and TerminalStates

func (*Client) Offers added in v2.28.0 

func (c *Client) Offers() ([]Offer, error)

Offers pulls data from /offers endpoint

func (*Client) PauseJobUpdate 

func (c *Client) PauseJobUpdate(updateKey *aurora.JobUpdateKey, message string) error

Pause AuroraJob Update. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) PerformBackup 

func (c *Client) PerformBackup() error

Force Aurora Scheduler to write backup file to a file in the backup directory

func (*Client) PulseJobUpdate 

func (c *Client) PulseJobUpdate(updateKey aurora.JobUpdateKey) (aurora.JobUpdatePulseStatus, error)

Pulse AuroraJob Update on Aurora. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) RealisConfig 

func (c *Client) RealisConfig() *clientConfig

func (*Client) ReestablishConn 

func (c *Client) ReestablishConn() error

func (*Client) RemoveInstances 

func (c *Client) RemoveInstances(key aurora.JobKey, count int) error

Scale down the number of instances under a job configuration using the configuration of a specific instance Instances with a higher instance ID will be removed first. For example, if our instance ID list is [0,1,2,3] and we want to remove 2 instances, 2 and 3 will always be picked.

func (*Client) RestartInstances 

func (c *Client) RestartInstances(key aurora.JobKey, instances ...int32) error

Restarts specific instances specified

func (*Client) RestartJob 

func (c *Client) RestartJob(key aurora.JobKey) error

Restarts all active tasks under a job configuration.

func (*Client) ResumeJobUpdate 

func (c *Client) ResumeJobUpdate(updateKey aurora.JobUpdateKey, message string) error

Resume Paused AuroraJob Update. UpdateID is returned from StartJobUpdate or the Aurora web UI.

func (*Client) RollbackJobUpdate 

func (c *Client) RollbackJobUpdate(key aurora.JobUpdateKey, message string) error

func (*Client) SLADrainHosts 

func (c *Client) SLADrainHosts(policy *aurora.SlaPolicy, timeout int64, hosts ...string) ([]*aurora.HostStatus, error)

Start SLA Aware Drain. defaultSlaPolicy is the fallback SlaPolicy to use if a task does not have an SlaPolicy. After timeoutSecs, tasks will be forcefully drained without checking SLA.

func (*Client) ScheduleCronJob 

func (c *Client) ScheduleCronJob(auroraJob *AuroraJob) error

func (*Client) SetQuota 

func (c *Client) SetQuota(role string, cpu *float64, ramMb *int64, diskMb *int64) error

SetQuota sets a quota aggregate for the given role TODO(zircote) Currently investigating an error that is returned from thrift calls that include resources for `NamedPort` and `NumGpu`

func (*Client) Snapshot 

func (c *Client) Snapshot() error

Force Aurora Scheduler to perform a snapshot and write to Mesos log

func (*Client) StartCronJob 

func (c *Client) StartCronJob(key aurora.JobKey) error

func (*Client) StartJobUpdate 

func (c *Client) StartJobUpdate(updateJob *JobUpdate, message string) (*aurora.StartJobUpdateResult_, error)

Update all tasks under a job configuration. Currently gorealis doesn't support for canary deployments.

func (*Client) StartMaintenance 

func (c *Client) StartMaintenance(hosts ...string) ([]*aurora.HostStatus, error)

type ClientOption 

type ClientOption func(*clientConfig)

func BackOff 

func BackOff(b Backoff) ClientOption

func BasicAuth 

func BasicAuth(username, password string) ClientOption

clientConfig sets for options in clientConfig.

func CertsPath 

func CertsPath(certspath string) ClientOption

func ClientCerts 

func ClientCerts(clientKey, clientCert string) ClientOption

func Debug 

func Debug() ClientOption

Enable debug statements.

func FailOnPermanentErrors 

func FailOnPermanentErrors() ClientOption

FailOnPermanentErrors - If the client encounters a connection error the standard library considers permanent, stop retrying and return an error to the user.

func InsecureSkipVerify 

func InsecureSkipVerify(InsecureSkipVerify bool) ClientOption

func SchedulerUrl 

func SchedulerUrl(url string) ClientOption

func SetLogger 

func SetLogger(l Logger) ClientOption

Using the word set to avoid name collision with Interface.

func ThriftBinary 

func ThriftBinary() ClientOption

func ThriftJSON 

func ThriftJSON() ClientOption

func Timeout 

func Timeout(timeout time.Duration) ClientOption

func Trace 

func Trace() ClientOption

Enable trace statements.

func ZKCluster 

func ZKCluster(cluster *Cluster) ClientOption

func ZKUrl 

func ZKUrl(url string) ClientOption

func ZookeeperOptions 

func ZookeeperOptions(opts ...ZKOpt) ClientOption

Use this option if you'd like to override default settings for connecting to Zookeeper. See zk.go for what is possible to set as an option.

type Cluster 

type Cluster struct {
	Name          string `json:"name"`
	AgentRoot     string `json:"slave_root"`
	AgentRunDir   string `json:"slave_run_directory"`
	ZK            string `json:"zk"`
	ZKPort        int    `json:"zk_port"`
	SchedZKPath   string `json:"scheduler_zk_path"`
	MesosZKPath   string `json:"mesos_zk_path"`
	SchedURI      string `json:"scheduler_uri"`
	ProxyURL      string `json:"proxy_url"`
	AuthMechanism string `json:"auth_mechanism"`
}

func GetDefaultClusterFromZKUrl 

func GetDefaultClusterFromZKUrl(zkURL string) *Cluster

type ConditionFunc 

type ConditionFunc func() (done bool, err error)

ConditionFunc returns true if the condition is satisfied, or an error if the loop should be aborted.

type Container 

type Container interface {
	Build() *aurora.Container
}

type DockerContainer 

type DockerContainer struct {
	// contains filtered or unexported fields
}

func NewDockerContainer 

func NewDockerContainer() *DockerContainer

func (*DockerContainer) AddParameter 

func (c *DockerContainer) AddParameter(name, value string) *DockerContainer

func (*DockerContainer) Build 

func (c *DockerContainer) Build() *aurora.Container

func (*DockerContainer) Image 

func (c *DockerContainer) Image(image string) *DockerContainer

type Endpoint 

type Endpoint struct {
	Host string `json:"host"`
	Port int    `json:"port"`
}

type JobUpdate 

type JobUpdate struct {
	// contains filtered or unexported fields
}

Structure to collect all information required to create job update

func JobUpdateFromAuroraTask 

func JobUpdateFromAuroraTask(task *AuroraTask) *JobUpdate

Creates an update with default values using an AuroraTask as the underlying task configuration. This function has a high level understanding of Aurora Tasks and thus will support copying a task that is configured to use Thermos.

func JobUpdateFromConfig 

func JobUpdateFromConfig(task *aurora.TaskConfig) *JobUpdate

JobUpdateFromConfig creates an update with default values using an aurora.TaskConfig primitive as the underlying task configuration. This function should not be used unless the implications of using a primitive value are understood. For example, the primitive has no concept of Thermos.

func NewJobUpdate 

func NewJobUpdate() *JobUpdate

Create a default JobUpdate object with an empty task and no fields filled in.

func (*JobUpdate) AddDedicatedConstraint 

func (j *JobUpdate) AddDedicatedConstraint(role, name string) *JobUpdate

func (*JobUpdate) AddInstanceRange added in v2.22.1 

func (j *JobUpdate) AddInstanceRange(first, last int32) *JobUpdate

AddInstanceRange allows updates to only touch a certain specific range of instances

func (*JobUpdate) AddLabel 

func (j *JobUpdate) AddLabel(key string, value string) *JobUpdate

func (*JobUpdate) AddLimitConstraint 

func (j *JobUpdate) AddLimitConstraint(name string, limit int32) *JobUpdate

func (*JobUpdate) AddNamedPorts 

func (j *JobUpdate) AddNamedPorts(names ...string) *JobUpdate

func (*JobUpdate) AddPorts 

func (j *JobUpdate) AddPorts(num int) *JobUpdate

func (*JobUpdate) AddURIs 

func (j *JobUpdate) AddURIs(extract bool, cache bool, values ...string) *JobUpdate

func (*JobUpdate) AddValueConstraint 

func (j *JobUpdate) AddValueConstraint(name string, negated bool, values ...string) *JobUpdate

func (*JobUpdate) BatchSize 

func (j *JobUpdate) BatchSize(size int32) *JobUpdate

Max number of instances being updated at any given moment.

func (*JobUpdate) BatchUpdateStrategy added in v2.22.0 

func (j *JobUpdate) BatchUpdateStrategy(autoPause bool, batchSize int32) *JobUpdate

func (*JobUpdate) BuildThermosPayload 

func (j *JobUpdate) BuildThermosPayload() error

func (*JobUpdate) CPU 

func (j *JobUpdate) CPU(cpus float64) *JobUpdate

func (*JobUpdate) Container 

func (j *JobUpdate) Container(container Container) *JobUpdate

func (*JobUpdate) Disk 

func (j *JobUpdate) Disk(disk int64) *JobUpdate

func (*JobUpdate) Environment 

func (j *JobUpdate) Environment(env string) *JobUpdate

func (*JobUpdate) ExecutorData 

func (j *JobUpdate) ExecutorData(data string) *JobUpdate

func (*JobUpdate) ExecutorName 

func (j *JobUpdate) ExecutorName(name string) *JobUpdate

func (*JobUpdate) InstanceCount 

func (j *JobUpdate) InstanceCount(inst int32) *JobUpdate

Set instance count the job will have after the update.

func (*JobUpdate) IsService 

func (j *JobUpdate) IsService(isService bool) *JobUpdate

func (*JobUpdate) JobKey 

func (j *JobUpdate) JobKey() aurora.JobKey

func (*JobUpdate) MaxFailedInstances 

func (j *JobUpdate) MaxFailedInstances(inst int32) *JobUpdate

Max number of FAILED instances to tolerate before terminating the update.

func (*JobUpdate) MaxPerInstanceFailures 

func (j *JobUpdate) MaxPerInstanceFailures(inst int32) *JobUpdate

Max number of instance failures to tolerate before marking instance as FAILED.

func (*JobUpdate) Name 

func (j *JobUpdate) Name(name string) *JobUpdate

func (*JobUpdate) PartitionPolicy 

func (j *JobUpdate) PartitionPolicy(reschedule bool, delay int64) *JobUpdate

func (*JobUpdate) Priority added in v2.25.0 

func (j *JobUpdate) Priority(priority int32) *JobUpdate

func (*JobUpdate) Production added in v2.26.0 

func (j *JobUpdate) Production(production bool) *JobUpdate

func (*JobUpdate) PulseIntervalTimeout 

func (j *JobUpdate) PulseIntervalTimeout(timeout time.Duration) *JobUpdate

Sets the interval at which pulses should be received by the job update before timing out.

func (*JobUpdate) QueueUpdateStrategy added in v2.22.0 

func (j *JobUpdate) QueueUpdateStrategy(groupSize int32) *JobUpdate

func (*JobUpdate) RAM 

func (j *JobUpdate) RAM(ram int64) *JobUpdate

func (*JobUpdate) Role 

func (j *JobUpdate) Role(role string) *JobUpdate

func (*JobUpdate) RollbackOnFail 

func (j *JobUpdate) RollbackOnFail(rollback bool) *JobUpdate

When False, prevents auto rollback of a failed update.

func (*JobUpdate) SlaAware added in v2.22.1 

func (j *JobUpdate) SlaAware(slaAware bool) *JobUpdate

SlaAware makes the scheduler enforce the SLA Aware policy if the job meets the SLA awareness criteria. By default, the scheduler will only apply SLA Awareness to jobs in the production tier with 20 or more instances.

func (*JobUpdate) TaskConfig 

func (j *JobUpdate) TaskConfig() *aurora.TaskConfig

func (*JobUpdate) TaskMaxFailure added in v2.22.1 

func (j *JobUpdate) TaskMaxFailure(maxFail int32) *JobUpdate

func (*JobUpdate) ThermosExecutor 

func (j *JobUpdate) ThermosExecutor(thermos ThermosExecutor) *JobUpdate

func (*JobUpdate) Tier 

func (j *JobUpdate) Tier(tier string) *JobUpdate

func (*JobUpdate) VariableBatchStrategy added in v2.22.0 

func (j *JobUpdate) VariableBatchStrategy(autoPause bool, batchSizes ...int32) *JobUpdate

func (*JobUpdate) WaitForBatchCompletion 

func (j *JobUpdate) WaitForBatchCompletion(batchWait bool) *JobUpdate

Wait for all instances in a group to be done before moving on.

func (*JobUpdate) WatchTime 

func (j *JobUpdate) WatchTime(timeout time.Duration) *JobUpdate

Minimum number of seconds a shard must remain in RUNNING state before considered a success.

type LevelLogger 

type LevelLogger struct {
	Logger
	// contains filtered or unexported fields
}

func (LevelLogger) DebugPrint 

func (l LevelLogger) DebugPrint(a ...interface{})

func (LevelLogger) DebugPrintf 

func (l LevelLogger) DebugPrintf(format string, a ...interface{})

func (LevelLogger) DebugPrintln 

func (l LevelLogger) DebugPrintln(a ...interface{})

func (*LevelLogger) EnableDebug 

func (l *LevelLogger) EnableDebug(enable bool)

func (*LevelLogger) EnableTrace 

func (l *LevelLogger) EnableTrace(enable bool)

func (LevelLogger) TracePrint 

func (l LevelLogger) TracePrint(a ...interface{})

func (LevelLogger) TracePrintf 

func (l LevelLogger) TracePrintf(format string, a ...interface{})

func (LevelLogger) TracePrintln 

func (l LevelLogger) TracePrintln(a ...interface{})

type Logger 

type Logger interface {
	Println(v ...interface{})
	Printf(format string, v ...interface{})
	Print(v ...interface{})
}

type MaintenanceList added in v2.28.0 

type MaintenanceList struct {
	Drained   []string            `json:"DRAINED"`
	Scheduled []string            `json:"SCHEDULED"`
	Draining  map[string][]string `json:"DRAINING"`
}

hosts on [aurora-scheduler]/maintenance endpoint

type MesosAddress added in v2.22.2 

type MesosAddress struct {
	Hostname string `json:"hostname"`
	IP       string `json:"ip"`
	Port     uint16 `json:"port"`
}

type MesosContainer 

type MesosContainer struct {
	// contains filtered or unexported fields
}

func NewMesosContainer 

func NewMesosContainer() *MesosContainer

func (*MesosContainer) AddVolume 

func (c *MesosContainer) AddVolume(hostPath, containerPath string, mode aurora.Mode) *MesosContainer

func (*MesosContainer) AppcImage 

func (c *MesosContainer) AppcImage(name, imageId string) *MesosContainer

func (*MesosContainer) Build 

func (c *MesosContainer) Build() *aurora.Container

func (*MesosContainer) DockerImage 

func (c *MesosContainer) DockerImage(name, tag string) *MesosContainer

type MesosInstance added in v2.22.2 

type MesosInstance struct {
	Address MesosAddress `json:"address"`
	Version string       `json:"version"`
}

MesosInstance is defined for mesos json stored in ZK.

type NoopLogger 

type NoopLogger struct{}

func (NoopLogger) Print 

func (NoopLogger) Print(a ...interface{})

func (NoopLogger) Printf 

func (NoopLogger) Printf(format string, a ...interface{})

func (NoopLogger) Println 

func (NoopLogger) Println(a ...interface{})

type Offer added in v2.28.0 

type Offer struct {
	ID struct {
		Value string `json:"value"`
	} `json:"id"`
	FrameworkID struct {
		Value string `json:"value"`
	} `json:"framework_id"`
	AgentID struct {
		Value string `json:"value"`
	} `json:"agent_id"`
	Hostname string `json:"hostname"`
	URL      struct {
		Scheme  string `json:"scheme"`
		Address struct {
			Hostname string `json:"hostname"`
			IP       string `json:"ip"`
			Port     int    `json:"port"`
		} `json:"address"`
		Path  string        `json:"path"`
		Query []interface{} `json:"query"`
	} `json:"url"`
	Resources []struct {
		Name   string `json:"name"`
		Type   string `json:"type"`
		Ranges struct {
			Range []struct {
				Begin int `json:"begin"`
				End   int `json:"end"`
			} `json:"range"`
		} `json:"ranges,omitempty"`
		Role         string        `json:"role"`
		Reservations []interface{} `json:"reservations"`
		Scalar       struct {
			Value float64 `json:"value"`
		} `json:"scalar,omitempty"`
	} `json:"resources"`
	Attributes []struct {
		Name string `json:"name"`
		Type string `json:"type"`
		Text struct {
			Value string `json:"value"`
		} `json:"text"`
	} `json:"attributes"`
	ExecutorIds []struct {
		Value string `json:"value"`
	} `json:"executor_ids"`
}

Offers on [aurora-scheduler]/offers endpoint

type OfferCount added in v2.28.0 

type OfferCount map[float64]int64

type OfferGroupReport added in v2.28.0 

type OfferGroupReport map[string]OfferCount

type OfferReport added in v2.28.0 

type OfferReport map[string]OfferGroupReport

type ResourceType 

type ResourceType int
const (
	CPU ResourceType = iota
	RAM
	DISK
	GPU
)

type ServiceInstance 

type ServiceInstance struct {
	Service             Endpoint            `json:"serviceEndpoint"`
	AdditionalEndpoints map[string]Endpoint `json:"additionalEndpoints"`
	Status              string              `json:"status"`
}

type TemporaryErr 

type TemporaryErr struct {
	// contains filtered or unexported fields
}

func NewTemporaryError 

func NewTemporaryError(err error) *TemporaryErr

Retrying after receiving this error is advised

func (*TemporaryErr) Temporary 

func (t *TemporaryErr) Temporary() bool

type ThermosConstraint 

type ThermosConstraint struct {
	Order []string `json:"order,omitempty"`
}

type ThermosExecutor 

type ThermosExecutor struct {
	Task ThermosTask `json:"task""`
	// contains filtered or unexported fields
}

func (*ThermosExecutor) AddProcess 

func (t *ThermosExecutor) AddProcess(process ThermosProcess) *ThermosExecutor

Processes must have unique names. Adding a process whose name already exists will result in overwriting the previous version of the process.

func (*ThermosExecutor) Clone 

func (t *ThermosExecutor) Clone() *ThermosExecutor

Deep copy of Thermos executor

func (*ThermosExecutor) ProcessOrder 

func (t *ThermosExecutor) ProcessOrder(order ...string) *ThermosExecutor

Order in which the Processes should be executed. Index 0 will be executed first, index N will be executed last.

type ThermosProcess 

type ThermosProcess struct {
	Name        string `json:"name"`
	Cmdline     string `json:"cmdline"`
	Daemon      bool   `json:"daemon"`
	Ephemeral   bool   `json:"ephemeral"`
	MaxFailures int    `json:"max_failures"`
	MinDuration int    `json:"min_duration"`
	Final       bool   `json:"final"`
}

func NewThermosProcess 

func NewThermosProcess(name, command string) ThermosProcess

type ThermosTask 

type ThermosTask struct {
	Processes   map[string]*ThermosProcess `json:"processes"`
	Constraints []*ThermosConstraint       `json:"constraints"`
	Resources   thermosResources           `json:"resources"`
}

func (*ThermosTask) MarshalJSON 

func (t *ThermosTask) MarshalJSON() ([]byte, error)

Custom Marshaling for Thermos Task to match what Thermos expects

func (*ThermosTask) UnmarshalJSON 

func (t *ThermosTask) UnmarshalJSON(data []byte) error

Custom Unmarshaling to match what Thermos would contain

type TransportProtocol 

type TransportProtocol int

type ZKOpt 

type ZKOpt func(z *zkConfig)

func ZKBackoff 

func ZKBackoff(b Backoff) ZKOpt

func ZKEndpoints 

func ZKEndpoints(endpoints ...string) ZKOpt

func ZKLogger 

func ZKLogger(l Logger) ZKOpt

func ZKPath 

func ZKPath(path string) ZKOpt

func ZKTimeout 

func ZKTimeout(d time.Duration) ZKOpt

Source Files

View all Source files

Directories

Path Synopsis
gen-go
apache/aurora
apache/aurora/aurora_admin-remote
apache/aurora/aurora_scheduler_manager-remote
apache/aurora/read_only_scheduler-remote
Helper functions to process aurora.Response
 Helper functions to process aurora.Response

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.