libdrmaa

package module
Version: v0.0.0-...-2162eb3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 2, 2021 License: Apache-2.0 Imports: 11 Imported by: 2

README

JobTracker implementation on top of libdrmaa.so

Basic implementation of a JobTracker wrapper for libdrmaa.so. This is the DRMAA version 1 c library which is shipped with many workload managers. The JobTracker implementation can be used by DRMAA2OS to provide a Go DRMAA2 interface for drmaa version 1.

It is currently tested with Grid Engine using the Docker image in this directory. The Jobtracker uses github.com/dgruber/drmaa Go wrapper for job submission.

JobTemplate Mapping

DRMAA2 JobTemplate Internal Go drmaa job template
RemoteCommand SetRemoteCommand
Args SetArgs
InputPath SetInputPath(":"+InputPath)
OutputPath SetOutputPath(":"+OutputPath)
ErrorPath SetErrorPath(":"+ErrorPath)
JobName "cdrmaatrackerjob" if not set / SetJobName
JoinFiles SetJoinFiles
Email SetEmail
JobEnviornment map[key]value SetJobEnviornment("key=value", ...)

JobState Mapping

The following table shows how DRMAA2 job states are mapped to DRMAA version 1 job states.

DRMAA2 Job State Internal Go drmaa job state
drmaa2interface.Undetermined drmaa.PsUndetermined
drmaa2interface.Queued drmaa.PsQueuedActive
drmaa2interface.QueuedHeld drmaa.PsSystemOnHold
drmaa2interface.QueuedHeld drmaa.PsUserOnHold
drmaa2interface.QueuedHeld drmaa.PsUserSystemOnHold
drmaa2interface.Running drmaa.PsRunning
drmaa2interface.Suspended drmaa.PsSystemSuspended
drmaa2interface.Suspended drmaa.PsUserSuspended
drmaa2interface.Suspended drmaa.PsUserSystemSuspended
drmaa2interface.Done drmaa.PsDone
drmaa2interface.Failed drmaa.PsFailed

JobInfo Mapping

JobInfo is set when the job is in an end state.

DRMAA2 JobInfo Internal Go drmaa job info
ExitStatus Only meaningful when drmaa job HasExited()
ID JobID()
SubmissionTime Resource usage map submission_time value
DispatchTime Resource usage map start_time value
FinishTime Resource usage map end_time value

TODO add more

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConvertDRMAA2JobTemplateToDRMAAJobTemplate

func ConvertDRMAA2JobTemplateToDRMAAJobTemplate(jt drmaa2interface.JobTemplate, t *drmaa.JobTemplate) error

ConvertDRMAA2JobTemplateToDRMAAJobTemplate transforms a Go DRMAA2 job template into a C drmaa job template.

func ConvertDRMAAJobInfoToDRMAA2JobInfo

func ConvertDRMAAJobInfoToDRMAA2JobInfo(ji *drmaa.JobInfo) (info drmaa2interface.JobInfo)

ConvertDRMAAJobInfoToDRMAA2JobInfo takes a drmaa v1 JobInfo object and converts it into a DRMAA2 JobInfo object.

func ConvertDRMAAJobTemplateToDRMAA2JobTemplate

func ConvertDRMAAJobTemplateToDRMAA2JobTemplate(jt *drmaa.JobTemplate) (drmaa2interface.JobTemplate, error)

ConvertDRMAAJobTemplateToDRMAA2JobTemplate transforms a C DRMAA job template into a Go DRMAA2 job template.

func ConvertDRMAAStateToDRMAA2State

func ConvertDRMAAStateToDRMAA2State(pt drmaa.PsType) drmaa2interface.JobState

ConvertDRMAAStateToDRMAA2State takes a DRMAA v1 state and converts it in a DRMAA2 job state.

func ConvertUnixToTime

func ConvertUnixToTime(t string) time.Time

ConvertUnixToTime converts something like 1590752891.0000 to time. Some systems report ms since epoch others just seconds.

func NewAllocator

func NewAllocator() *allocator

Types

type DRMAATracker

type DRMAATracker struct {
	sync.Mutex
	// contains filtered or unexported fields
}

DRMAATracker implements the JobTracker interface with drmaa.so as backend for job management. That allows to user drmaa.so with a DRMAA2 compatible interface.

func NewDRMAATracker

func NewDRMAATracker() (*DRMAATracker, error)

NewDRMAATracker creates a new JobTracker interface implementation which manages jobs through the drmaa (version 1) interface.

func (*DRMAATracker) AddArrayJob

func (t *DRMAATracker) AddArrayJob(template drmaa2interface.JobTemplate, begin int, end int, step int, maxParallel int) (string, error)

AddArrayJob submits an array job through the underlying drmaa.so RunBulkJobs function.

func (*DRMAATracker) AddJob

func (t *DRMAATracker) AddJob(template drmaa2interface.JobTemplate) (string, error)

AddJob makes a new job submission through the underlying drmaa.so RunJob function.

func (*DRMAATracker) DeleteJob

func (t *DRMAATracker) DeleteJob(jobID string) error

DeleteJob removes a job from the internal DB. It can only be removed when it is in an end state (failed or done.

func (*DRMAATracker) DestroySession

func (t *DRMAATracker) DestroySession() error

DestroySession is not part of the interface but neccessary for shutting down the connection to the workload manager.

func (*DRMAATracker) JobControl

func (t *DRMAATracker) JobControl(jobID, action string) error

JobControl allows the job to be executed.

func (*DRMAATracker) JobInfo

func (t *DRMAATracker) JobInfo(jobID string) (drmaa2interface.JobInfo, error)

JobInfo returns more detailed information about a job when the job is finished.

func (*DRMAATracker) JobState

func (t *DRMAATracker) JobState(jobID string) (drmaa2interface.JobState, string, error)

JobState returns the current state of the given job.

func (*DRMAATracker) ListArrayJobs

func (t *DRMAATracker) ListArrayJobs(arrayJobID string) ([]string, error)

ListArrayJobs returns all job IDs of the job array task.

func (*DRMAATracker) ListJobCategories

func (t *DRMAATracker) ListJobCategories() ([]string, error)

ListJobCategories returns the job categories available at the workload manager. Since this is not a drmaa v1 concept we ignore it for now.

func (*DRMAATracker) ListJobs

func (t *DRMAATracker) ListJobs() ([]string, error)

ListJobs returns all jobs previously submitted and still locally cached.

func (*DRMAATracker) Wait

func (t *DRMAATracker) Wait(jobid string, timeout time.Duration, state ...drmaa2interface.JobState) error

Wait blocks until the job reached one of the given states or the timeout is reached.

type WorkloadManagerType

type WorkloadManagerType int

WorkloadManagerType is related to a specific drmaa.so backend as there are minor differences in terms of capabilities

const (
	// UnivaGridEngine as recogized drmaa.so backend
	UnivaGridEngine WorkloadManagerType = iota
	// SonOfGridEngine as recogized drmaa.so backend
	SonOfGridEngine
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL