job

Documentation

Overview

Package job provides job-related interfaces, data structures, and errors. To avoid an import cycle, this package must not have external dependencies because everything else depends on it.

Index

• Variables
• type ErrArgNotSet
  • func (e ErrArgNotSet) Error() string
• type ErrDataNotSet
  • func (e ErrDataNotSet) Error() string
• type ErrWrongArgType
  • func NewErrWrongArgType(key string, got, expect interface{}) ErrWrongArgType
  • func (e ErrWrongArgType) Error() string
• type ErrWrongDataType
  • func NewErrWrongDataType(key string, got, expect interface{}) ErrWrongDataType
  • func (e ErrWrongDataType) Error() string
• type Factory
• type Id
  • func NewId(jobType, jobName, jobId string) Id
  • func NewIdWithRequestId(jobType, jobName, jobId, reqId string) Id
• type Job
• type Return

Variables

var (
	// ErrUnknownJobType is returned by a job.Factory when the factory cannot
	// make the requested job type.
	ErrUnknownJobType = errors.New("unknown job type")

	// ErrRunTimeout should be returned by a job when it times out.
	ErrRunTimeout = errors.New("run timeout")

	// ErrWrongJobId is returned when the external job factory (EJF) creates a
	// job that does not return the same Id values. This is a bug in the EJF,
	// not Spin Cycle.
	ErrWrongJobId = errors.New("wrong job.Id")
)

Types

type ErrArgNotSet

type ErrArgNotSet struct {
	Arg string
}

ErrArgNotSet should be returned by a job when a required key is not set in jobArgs. For example:

val, ok := jobArgs["srcHost"]
if !ok {
  return job.ErrArgNotSet{"srcHost"}
}

func (ErrArgNotSet) Error

func (e ErrArgNotSet) Error() string

type ErrDataNotSet

type ErrDataNotSet struct {
	Key string
}

ErrDataNotSet should be returned by a job when a required key is not set in jobData.

func (ErrDataNotSet) Error

func (e ErrDataNotSet) Error() string

type ErrWrongArgType

type ErrWrongArgType struct {
	Key        string
	GotType    reflect.Type
	ExpectType reflect.Type
}

ErrWrongArgType should be returned by a job when a jobArgs value is not the expected type. For example:

v := jobArgs["sliceOfStrings"]
sliceOfStrings, ok := v.([]string])
if !ok {
  return job.NewErrWrongArgType("sliceOfStrings", v, []string{})
}

func NewErrWrongArgType

func NewErrWrongArgType(key string, got, expect interface{}) ErrWrongArgType

func (ErrWrongArgType) Error

func (e ErrWrongArgType) Error() string

type ErrWrongDataType

type ErrWrongDataType struct {
	Key        string
	GotType    reflect.Type
	ExpectType reflect.Type
}

ErrWrongDataType should be returned by a job when a jobData value is not the expected type. For example:

type Cluster struct { ... }
v := jobData["cluster"]
cluster, ok := v.(Cluster)
if !ok {
  return job.NewErrWrongDataType("cluster", v, Cluster{})
}

func NewErrWrongDataType

func NewErrWrongDataType(key string, got, expect interface{}) ErrWrongDataType

func (ErrWrongDataType) Error

func (e ErrWrongDataType) Error() string

type Factory

type Factory interface {
	Make(id Id) (Job, error)
}

A Factory instantiates a Job of the given type. A factory only instantiates a new Job object, it must not call any Job interface methods on the newly created job. If an error is returned, the returned Job should be ignored.

Spin Cycle does not and should not know how to instantiate jobs because they are external (imported in external/factory.go). Once a job is instantiated with external, implementation-specific details, Spin Cycle only needs to know and use the Job interface methods.

Spin Cycle calls the job's Id method to ensure that it returns the same Id values that it was created with. If not, ErrWrongJobId is returned.

type Id

type Id struct {
	// User-defined low-level job type implemented in code and made by external
	// job factory (EJF). Any value that the EJF is coded to make is valid. Job
	// types are heavily reused by being given unique names in request specs.
	Type string

	// User-defined job name in a request spec. Names must be unique per sequence.
	// During sequence expansion, job names remain the same and become uniquely
	// identified from one another by job ID.
	Name string

	// Spin Cycle-defined job ID unique per request. Job IDs distinguish named
	// jobs in expanded sequences.
	Id string

	// RequestId of the request that created the job. This is only informational
	// for reporting/loggging/tracing.
	RequestId string
}

Id represents how jobs are uniquely identified per request. Type and Name are user-defined in the external job factory (EJF) and request spec, respectively. Id is defined per request by Spin Cycle. An example for each value:

Type:  net/down-ip
Name:  down-sip-on-eth0
Id:    9m4e2mr0

Job types are defined in code and in the external job factory (EJF). They represent what the job is and does. Job names are defined in request specs; they are named instantiantions of job types. The name should be short but meaningful in the context of the request. Job IDs are short, random strings generated by Spin Cycle. Id is only guaranteed to be unique per request, not globally.

func NewId

func NewId(jobType, jobName, jobId string) Id

NewId is a convenience function for creating a new Id with the given values. It's used by job-runner/runner/Factory.Make to create an Id from a proto.Job.

func NewIdWithRequestId

func NewIdWithRequestId(jobType, jobName, jobId, reqId string) Id

type Job

type Job interface {
	// Create allows the job to get and save internal data needed to run later.
	// The job can save jobArgs and set new ones for other jobs.
	//
	// This method is only called once by the Request Manager when constructing
	// a job chain. Construction of the job chain fails if an error is returned.
	Create(jobArgs map[string]interface{}) error

	// Serialize returns all internal data needed to run later. The reciprocal
	// method is Deserialize.
	//
	// This method is only called once by the Request Manager when constructing
	// a job chain. Construction of the job chain fails if an error is returned.
	Serialize() ([]byte, error)

	// Deserialize sets internal data needed to run later. The reciprocal
	// method is Serialize.
	//
	// This method is only called once by the Job Runner when reconstructing
	// a job chain. Reconstruction of the job chain fails if an error is
	// returned.
	Deserialize([]byte) error

	// Run runs the job using its interal data and the run-time jobData from
	// previously-ran (upstream) jobs. Run can modify jobData. Run is expected
	// to block, but the job must respond to Stop and Status while running.
	// The returned error, if any, indicates a problem before or after running
	// the job. The final state of the job is returned in the Return structure,
	// along with other things like the error and exit code (if there was one).
	// The final state is the most important field because it is used by the JR
	// to determine how to handle a job when it's done running.
	//
	// Currently, the Job Runner only calls this method once. Resuming a job is
	// not currently supported.
	Run(jobData map[string]interface{}) (Return, error)

	// Stop stops a job. The Job Runner calls this method when stopping a job
	// chain before it has completed. The job must respond to Stop while Run
	// is executing. Stop is expected to block but also return quickly.
	Stop() error

	// Status returns the real-time status of the job. The job must respond
	// to Status while Run is executing. Status is expected to block but also
	// return quickly.
	Status() string

	// Id returns the Id structure the job was created with. This is used primarily
	// for status and logging by Spin Cycle. The Id values for the job must exactly
	// match the Id values passed to the external job factory which created the job,
	// else Spin Cycle returns ErrWrongJobId.
	Id() Id
}

A Job is the smallest, reusable building block in Spin Cycle that has meaning by itself. A job should do one thing and be reusable. For example, job type "net/down-ip" removes an IP address from a network interface. This job is meaningful by itself and reusable in different requests.

All jobs are user-defined and instantiated by the external job factory (EJF), which is also user-defined (job/external.JobFactory). Spin Cycle only defines the Job interface; jobs are written by the user. A job must implement the Job interface and be able to accomplish its purpose only through this interface. Spin Cycle is completely unaware of job implementation details.

The Job interface has two sides: one for the Request Manager (RM), the other for the Job Runner (JR). The RM calls Create and Serialize, and the JR calls the other methods. The call sequence is: Create, Serialize, Deserialize, Run.

type Return

type Return struct {
	State  byte   // proto/STATE_ const
	Exit   int64  // Unix exit code
	Error  error  // Go error
	Stdout string // stdout output
	Stderr string // stderr output
}

Return represents return values and output from a job. State indicates how the job completed. If State == proto.STATE_COMPLETE, the job completed successfully. Anything else indicates that the job failed or didn't complete, which will probably cause the job chain to stop. The job is responsible for adhering to this convention.

State, Exit, and Error are not mutually exclusive. A job can return State = STATE_COMPLETE to indicate success but also a non-nil Error or a non-zero Exit. This is useful for idempotent jobs and logging that the job was successful because it handled being re-ran. For example, a job could delete a record, but when re-ran the record has already been deleted, so the job is successful but reports Error = ErrRecordNotFound for logging.