realis

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 24, 2018 License: Apache-2.0 Imports: 21 Imported by: 0

README

gorealis GoDoc

Go library for interacting with Apache Aurora.

Aurora version compatibility

Please see .auroraversion to see the latest Aurora version against which this library has been tested.

Usage

To Do

Contributions

Contributions are always welcome. Please raise an issue so that the contribution may be discussed before it's made.

Documentation

Overview

Collection of monitors to create synchronicity

Package realis provides the ability to use Thrift API to communicate with Apache Aurora.

Index

Constants

View Source 
const (
	UpdateFailed = "update failed"
	RolledBack   = "update rolled back"
	Timeout      = "timeout"
)
View Source 
const VERSION = "1.1.0"

Variables

This section is empty.

Functions

func AddBasicAuth 

func AddBasicAuth(config *RealisConfig, username string, password string)

Helper function to add basic authorization needed to communicate with Apache Aurora.

func ExponentialBackoff added in v1.1.0 

func ExponentialBackoff(backoff Backoff, condition ConditionFunc) error

Modified version of the Kubernetes exponential-backoff code. ExponentialBackoff repeats a condition check with exponential backoff.

It checks the condition up to Steps times, increasing the wait by multiplying the previous duration by Factor.

If Jitter is greater than zero, a random amount of each duration is added (between duration and duration*(1+jitter)).

If the condition never returns true, ErrWaitTimeout is returned. All other errors terminate immediately.

func GetCerts added in v1.2.0 

func GetCerts(certpath string) (*x509.CertPool, error)

func IsTemporary added in v1.2.0 

func IsTemporary(err error) bool

func IsTimeout added in v1.2.0 

func IsTimeout(err error) bool

func Jitter added in v1.1.0 

func Jitter(duration time.Duration, maxFactor float64) time.Duration

Jitter returns a time.Duration between duration and duration + maxFactor * duration.

This allows clients to avoid converging on periodic behavior. If maxFactor is 0.0, a suggested default value will be chosen.

func LeaderFromZK 

func LeaderFromZK(cluster Cluster) (string, error)

Retrieves current Aurora leader from ZK.

func LoadClusters 

func LoadClusters(config string) (map[string]Cluster, error)

Loads clusters.json file traditionally located at /etc/aurora/clusters.json

Types

type AuroraJob 

type AuroraJob struct {
	// contains filtered or unexported fields
}

Structure to collect all information pertaining to an Aurora job.

func (*AuroraJob) AddLabel 

func (j *AuroraJob) AddLabel(key string, value string) Job

Adds a Mesos label to the job. Note that Aurora will add the prefix "org.apache.aurora.metadata." to the beginning of each key.

func (*AuroraJob) AddLimitConstraint 

func (j *AuroraJob) AddLimitConstraint(name string, limit int32) Job

From Aurora Docs: A constraint that specifies the maximum number of active tasks on a host with a matching attribute that may be scheduled simultaneously.

func (*AuroraJob) AddNamedPorts 

func (j *AuroraJob) AddNamedPorts(names ...string) Job

Add a named port to the job configuration These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraJob) AddPorts 

func (j *AuroraJob) AddPorts(num int) Job

Adds a request for a number of ports to the job configuration. The names chosen for these ports will be org.apache.aurora.port.X, where X is the current port count for the job configuration starting at 0. These are random ports as it's not currently possible to request specific ports using Aurora.

func (*AuroraJob) AddURIs 

func (j *AuroraJob) AddURIs(extract bool, cache bool, values ...string) Job

Add a list of URIs with the same extract and cache configuration. Scheduler must have --enable_mesos_fetcher flag enabled. Currently there is no duplicate detection.

func (*AuroraJob) AddValueConstraint 

func (j *AuroraJob) AddValueConstraint(name string, negated bool, values ...string) Job

From Aurora Docs: Add a Value constraint name - Mesos slave attribute that the constraint is matched against. If negated = true , treat this as a 'not' - to avoid specific values. Values - list of values we look for in attribute name

func (*AuroraJob) CPU 

func (j *AuroraJob) CPU(cpus float64) Job

func (*AuroraJob) Container added in v1.0.2 

func (j *AuroraJob) Container(container Container) Job

Set a container to run for the job configuration to run.

func (*AuroraJob) CronCollisionPolicy 

func (j *AuroraJob) CronCollisionPolicy(policy aurora.CronCollisionPolicy) Job

func (*AuroraJob) CronSchedule 

func (j *AuroraJob) CronSchedule(cron string) Job

func (*AuroraJob) Disk 

func (j *AuroraJob) Disk(disk int64) Job

func (*AuroraJob) Environment 

func (j *AuroraJob) Environment(env string) Job

Set Job Key environment.

func (*AuroraJob) ExecutorData 

func (j *AuroraJob) ExecutorData(data string) Job

Will be included as part of entire task inside the scheduler that will be serialized.

func (*AuroraJob) ExecutorName 

func (j *AuroraJob) ExecutorName(name string) Job

Set name of the executor that will the task will be configured to.

func (*AuroraJob) GetInstanceCount 

func (j *AuroraJob) GetInstanceCount() int32

How many instances of the job to run

func (*AuroraJob) InstanceCount 

func (j *AuroraJob) InstanceCount(instCount int32) Job

How many instances of the job to run

func (*AuroraJob) IsService 

func (j *AuroraJob) IsService(isService bool) Job

Restart the job's tasks if they fail

func (*AuroraJob) JobConfig 

func (j *AuroraJob) JobConfig() *aurora.JobConfiguration

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) JobKey 

func (j *AuroraJob) JobKey() *aurora.JobKey

Get the current job configurations key to use for some realis calls.

func (*AuroraJob) MaxFailure 

func (j *AuroraJob) MaxFailure(maxFail int32) Job

How many failures to tolerate before giving up.

func (*AuroraJob) Name 

func (j *AuroraJob) Name(name string) Job

Set Job Key Name.

func (*AuroraJob) RAM 

func (j *AuroraJob) RAM(ram int64) Job

func (*AuroraJob) Role 

func (j *AuroraJob) Role(role string) Job

Set Job Key Role.

func (*AuroraJob) TaskConfig 

func (j *AuroraJob) TaskConfig() *aurora.TaskConfig

type Backoff added in v1.1.0 

type Backoff struct {
	Duration time.Duration // the base duration
	Factor   float64       // Duration is multipled by factor each iteration
	Jitter   float64       // The amount of jitter applied each iteration
	Steps    int           // Exit with error after this many steps
}

type ClientOption added in v1.1.0 

type ClientOption func(*RealisConfig)

func BackOff added in v1.1.0 

func BackOff(b *Backoff) ClientOption

func BasicAuth added in v1.1.0 

func BasicAuth(username, password string) ClientOption

Config sets for options in RealisConfig.

func Certspath added in v1.1.0 

func Certspath(certspath string) ClientOption

func ClientCerts added in v1.1.0 

func ClientCerts(clientKey, clientCert string) ClientOption

func InsecureSkipVerify added in v1.1.0 

func InsecureSkipVerify(InsecureSkipVerify bool) ClientOption

func Retries added in v1.1.0 

func Retries(backoff *Backoff) ClientOption

func SchedulerUrl added in v1.1.0 

func SchedulerUrl(url string) ClientOption

func SetLogger added in v1.1.0 

func SetLogger(l Logger) ClientOption

Using the word set to avoid name collision with Interface

func ThriftBinary added in v1.1.0 

func ThriftBinary() ClientOption

func ThriftJSON added in v1.1.0 

func ThriftJSON() ClientOption

func TimeoutMS added in v1.1.0 

func TimeoutMS(timeout int) ClientOption

func ZKCluster added in v1.1.0 

func ZKCluster(cluster *Cluster) ClientOption

func ZKUrl added in v1.1.0 

func ZKUrl(url string) ClientOption

type Cluster 

type Cluster struct {
	Name          string `json:"name"`
	AgentRoot     string `json:"slave_root"`
	AgentRunDir   string `json:"slave_run_directory"`
	ZK            string `json:"zk"`
	ZKPort        int    `json:"zk_port"`
	SchedZKPath   string `json:"scheduler_zk_path"`
	SchedURI      string `json:"scheduler_uri"`
	ProxyURL      string `json:"proxy_url"`
	AuthMechanism string `json:"auth_mechanism"`
}

func GetDefaultClusterFromZKUrl added in v1.1.0 

func GetDefaultClusterFromZKUrl(zkurl string) *Cluster

type ConditionFunc added in v1.1.0 

type ConditionFunc func() (done bool, err error)

ConditionFunc returns true if the condition is satisfied, or an error if the loop should be aborted.

type Container added in v1.0.2 

type Container interface {
	Build() *aurora.Container
}

type DockerContainer added in v1.0.2 

type DockerContainer struct {
	// contains filtered or unexported fields
}

func NewDockerContainer added in v1.0.2 

func NewDockerContainer() DockerContainer

func (DockerContainer) AddParameter added in v1.0.2 

func (c DockerContainer) AddParameter(name, value string) DockerContainer

func (DockerContainer) Build added in v1.0.2 

func (c DockerContainer) Build() *aurora.Container

func (DockerContainer) Image added in v1.0.2 

func (c DockerContainer) Image(image string) DockerContainer

type Endpoint 

type Endpoint struct {
	Host string `json:"host"`
	Port int    `json:"port"`
}

type Job 

type Job interface {
	// Set Job Key environment.
	Environment(env string) Job
	Role(role string) Job
	Name(name string) Job
	CPU(cpus float64) Job
	CronSchedule(cron string) Job
	CronCollisionPolicy(policy aurora.CronCollisionPolicy) Job
	Disk(disk int64) Job
	RAM(ram int64) Job
	ExecutorName(name string) Job
	ExecutorData(data string) Job
	AddPorts(num int) Job
	AddLabel(key string, value string) Job
	AddNamedPorts(names ...string) Job
	AddLimitConstraint(name string, limit int32) Job
	AddValueConstraint(name string, negated bool, values ...string) Job
	AddURIs(extract bool, cache bool, values ...string) Job
	JobKey() *aurora.JobKey
	JobConfig() *aurora.JobConfiguration
	TaskConfig() *aurora.TaskConfig
	IsService(isService bool) Job
	InstanceCount(instCount int32) Job
	GetInstanceCount() int32
	MaxFailure(maxFail int32) Job
	Container(container Container) Job
}

func NewJob 

func NewJob() Job

Create a Job object with everything initialized.

type Logger added in v1.1.0 

type Logger interface {
	Println(v ...interface{})
	Printf(format string, v ...interface{})
	Print(v ...interface{})
}

type MesosContainer added in v1.0.2 

type MesosContainer struct {
	// contains filtered or unexported fields
}

func NewMesosContainer added in v1.1.0 

func NewMesosContainer() MesosContainer

func (MesosContainer) AppcImage added in v1.1.0 

func (c MesosContainer) AppcImage(name, imageId string) MesosContainer

func (MesosContainer) Build added in v1.1.0 

func (c MesosContainer) Build() *aurora.Container

func (MesosContainer) DockerImage added in v1.1.0 

func (c MesosContainer) DockerImage(name, tag string) MesosContainer

type Monitor 

type Monitor struct {
	Client Realis
}

func (*Monitor) HostMaintenance added in v1.1.0 

func (m *Monitor) HostMaintenance(hosts []string, modes []aurora.MaintenanceMode, interval, timeout int) (map[string]bool, error)

Monitor host status until all hosts match the status provided. Returns a map where the value is true if the host is in one of the desired mode(s) or false if it is not as of the time when the monitor exited.

func (*Monitor) Instances 

func (m *Monitor) Instances(key *aurora.JobKey, instances int32, interval, timeout int) (bool, error)

Monitor a Job until all instances enter one of the LIVE_STATES

func (*Monitor) JobUpdate 

func (m *Monitor) JobUpdate(updateKey aurora.JobUpdateKey, interval int, timeout int) (bool, error)

Polls the scheduler every certain amount of time to see if the update has succeeded

func (*Monitor) ScheduleStatus added in v1.2.0 

func (m *Monitor) ScheduleStatus(key *aurora.JobKey, instanceCount int32, desiredStatuses map[aurora.ScheduleStatus]bool, interval, timeout int) (bool, error)

Monitor a Job until all instances enter a desired status. Defaults sets of desired statuses provided by the thrift API include: ACTIVE_STATES, SLAVE_ASSIGNED_STATES, LIVE_STATES, and TERMINAL_STATES

type NoopLogger added in v1.0.4 

type NoopLogger struct{}

func (NoopLogger) Print added in v1.1.0 

func (NoopLogger) Print(a ...interface{})

func (NoopLogger) Printf added in v1.0.4 

func (NoopLogger) Printf(format string, a ...interface{})

func (NoopLogger) Println added in v1.1.0 

func (NoopLogger) Println(a ...interface{})

type Realis 

type Realis interface {
	AbortJobUpdate(updateKey aurora.JobUpdateKey, message string) (*aurora.Response, error)
	AddInstances(instKey aurora.InstanceKey, count int32) (*aurora.Response, error)
	CreateJob(auroraJob Job) (*aurora.Response, error)
	CreateService(auroraJob Job, settings UpdateSettings) (*aurora.Response, *aurora.StartJobUpdateResult_, error)
	DescheduleCronJob(key *aurora.JobKey) (*aurora.Response, error)
	FetchTaskConfig(instKey aurora.InstanceKey) (*aurora.TaskConfig, error)
	GetInstanceIds(key *aurora.JobKey, states map[aurora.ScheduleStatus]bool) (map[int32]bool, error)
	GetJobUpdateSummaries(jobUpdateQuery *aurora.JobUpdateQuery) (*aurora.Response, error)
	GetTaskStatus(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)
	GetTasksWithoutConfigs(query *aurora.TaskQuery) ([]*aurora.ScheduledTask, error)
	JobUpdateDetails(updateQuery aurora.JobUpdateQuery) (*aurora.Response, error)
	KillJob(key *aurora.JobKey) (*aurora.Response, error)
	KillInstances(key *aurora.JobKey, instances ...int32) (*aurora.Response, error)
	RemoveInstances(key *aurora.JobKey, count int32) (*aurora.Response, error)
	RestartInstances(key *aurora.JobKey, instances ...int32) (*aurora.Response, error)
	RestartJob(key *aurora.JobKey) (*aurora.Response, error)
	RollbackJobUpdate(key aurora.JobUpdateKey, message string) (*aurora.Response, error)
	ScheduleCronJob(auroraJob Job) (*aurora.Response, error)
	StartJobUpdate(updateJob *UpdateJob, message string) (*aurora.Response, error)
	StartCronJob(key *aurora.JobKey) (*aurora.Response, error)
	// TODO: Remove this method and make it private to avoid race conditions
	ReestablishConn() error
	RealisConfig() *RealisConfig
	Close()

	// Admin functions
	DrainHosts(hosts ...string) (*aurora.Response, *aurora.DrainHostsResult_, error)
	EndMaintenance(hosts ...string) (*aurora.Response, *aurora.EndMaintenanceResult_, error)
	MaintenanceStatus(hosts ...string) (*aurora.Response, *aurora.MaintenanceStatusResult_, error)
}

func NewRealisClient added in v1.1.0 

func NewRealisClient(options ...ClientOption) (Realis, error)

type RealisConfig 

type RealisConfig struct {
	InsecureSkipVerify bool
	// contains filtered or unexported fields
}

type ServiceInstance 

type ServiceInstance struct {
	Service             Endpoint            `json:"serviceEndpoint"`
	AdditionalEndpoints map[string]Endpoint `json:"additionalEndpoints"`
	Status              string              `json:"status"`
}

type TemporaryErr added in v1.2.0 

type TemporaryErr struct {
	// contains filtered or unexported fields
}

func NewPermamentError added in v1.2.0 

func NewPermamentError(err error) TemporaryErr

Nothing can be done about this error

func NewTemporaryError added in v1.2.0 

func NewTemporaryError(err error) *TemporaryErr

Retrying after receiving this error is advised

func (*TemporaryErr) Temporary added in v1.2.0 

func (t *TemporaryErr) Temporary() bool

type TimeoutErr added in v1.2.0 

type TimeoutErr struct {
	// contains filtered or unexported fields
}

func NewTimeoutError added in v1.2.0 

func NewTimeoutError(err error) *TimeoutErr

func (*TimeoutErr) Timeout added in v1.2.0 

func (t *TimeoutErr) Timeout() bool

type UpdateJob 

type UpdateJob struct {
	Job // SetInstanceCount for job is hidden, access via full qualifier
	// contains filtered or unexported fields
}

Structure to collect all information required to create job update

func NewDefaultUpdateJob added in v1.1.0 

func NewDefaultUpdateJob(config *aurora.TaskConfig) *UpdateJob

Create a default UpdateJob object.

func NewUpdateJob 

func NewUpdateJob(config *aurora.TaskConfig, settings *aurora.JobUpdateSettings) *UpdateJob

func (*UpdateJob) BatchSize 

func (u *UpdateJob) BatchSize(size int32) *UpdateJob

Max number of instances being updated at any given moment.

func (*UpdateJob) InstanceCount 

func (u *UpdateJob) InstanceCount(inst int32) *UpdateJob

Set instance count the job will have after the update.

func (*UpdateJob) MaxFailedInstances 

func (u *UpdateJob) MaxFailedInstances(inst int32) *UpdateJob

Max number of FAILED instances to tolerate before terminating the update.

func (*UpdateJob) MaxPerInstanceFailures 

func (u *UpdateJob) MaxPerInstanceFailures(inst int32) *UpdateJob

Max number of instance failures to tolerate before marking instance as FAILED.

func (*UpdateJob) RollbackOnFail 

func (u *UpdateJob) RollbackOnFail(rollback bool) *UpdateJob

When False, prevents auto rollback of a failed update.

func (*UpdateJob) WaitForBatchCompletion 

func (u *UpdateJob) WaitForBatchCompletion(batchWait bool) *UpdateJob

Wait for all instances in a group to be done before moving on.

func (*UpdateJob) WatchTime 

func (u *UpdateJob) WatchTime(ms int32) *UpdateJob

Minimum number of seconds a shard must remain in RUNNING state before considered a success.

type UpdateSettings added in v1.1.0 

type UpdateSettings struct {
	// contains filtered or unexported fields
}

TODO(rdelvalle): Integrate this struct with the JobUpdate struct so that we don't repeat code

func NewUpdateSettings added in v1.1.0 

func NewUpdateSettings() *UpdateSettings

func (*UpdateSettings) BatchSize added in v1.1.0 

func (u *UpdateSettings) BatchSize(size int32) *UpdateSettings

Max number of instances being updated at any given moment.

func (*UpdateSettings) MaxFailedInstances added in v1.1.0 

func (u *UpdateSettings) MaxFailedInstances(inst int32) *UpdateSettings

Max number of FAILED instances to tolerate before terminating the update.

func (*UpdateSettings) MaxPerInstanceFailures added in v1.1.0 

func (u *UpdateSettings) MaxPerInstanceFailures(inst int32) *UpdateSettings

Max number of instance failures to tolerate before marking instance as FAILED.

func (*UpdateSettings) RollbackOnFail added in v1.1.0 

func (u *UpdateSettings) RollbackOnFail(rollback bool) *UpdateSettings

When False, prevents auto rollback of a failed update.

func (UpdateSettings) Settings added in v1.1.0 

func (u UpdateSettings) Settings() aurora.JobUpdateSettings

Return internal Thrift API structure

func (*UpdateSettings) WaitForBatchCompletion added in v1.1.0 

func (u *UpdateSettings) WaitForBatchCompletion(batchWait bool) *UpdateSettings

Wait for all instances in a group to be done before moving on.

func (*UpdateSettings) WatchTime added in v1.1.0 

func (u *UpdateSettings) WatchTime(ms int32) *UpdateSettings

Minimum number of seconds a shard must remain in RUNNING state before considered a success.

Source Files

View all Source files

Directories

Path Synopsis
gen-go
apache/aurora
apache/aurora/aurora_admin-remote
apache/aurora/aurora_scheduler_manager-remote
apache/aurora/read_only_scheduler-remote
Helper functions to process aurora.Response
 Helper functions to process aurora.Response

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.