Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func TaskNamespace ¶
func TaskNamespace(pipelineInfo *pps.PipelineInfo) string
TaskNamespace returns the namespace used by the task package for this pipeline.
Types ¶
type Driver ¶
type Driver interface { Jobs() col.PostgresCollection Pipelines() col.PostgresCollection NewTaskSource() task.Source NewTaskDoer(string, task.Cache) task.Doer // Returns the PipelineInfo for the pipeline that this worker belongs to PipelineInfo() *pps.PipelineInfo // Returns the kubernetes namespace that the worker is deployed in Namespace() string // Returns the path that will contain the input filesets for the job InputDir() string // Returns the pachd API client for the driver PachClient() *client.APIClient // Returns the number of workers to be used ExpectedNumWorkers() (int64, error) // WithContext clones the current driver and applies the context to its // pachClient. The pachClient context will be used for other blocking // operations as well. WithContext(context.Context) Driver // WithActiveData swaps the given scratch directory into the 'active' input // directory used when running user code. This also locks a mutex so that no // two datums can be active concurrently. WithActiveData([]*common.Input, string, func() error) error // UserCodeEnv returns the set of environment variables to construct when // launching the configured user process. UserCodeEnv(string, *pfs.Commit, []*common.Input, string) []string RunUserCode(context.Context, logs.TaggedLogger, []string) error RunUserErrorHandlingCode(context.Context, logs.TaggedLogger, []string) error // TODO: provide a more generic interface for modifying jobs, and // some quality-of-life functions for common operations. DeleteJob(*pachsql.Tx, *pps.JobInfo) error UpdateJobState(*pps.Job, pps.JobState, string) error // TODO: figure out how to not expose this - currently only used for a few // operations in the map spawner NewSQLTx(func(*pachsql.Tx) error) error // Returns the image ID associated with a container running in the worker pod GetContainerImageID(context.Context, string) (string, error) }
Driver provides an interface for common functions needed by worker code, and captures the relevant objects necessary to provide these functions so that users do not need to keep track of as many variables. In addition, this interface can be used to mock out external calls to make unit-testing simpler.
func NewDriver ¶
func NewDriver( ctx context.Context, env serviceenv.ServiceEnv, pachClient *client.APIClient, pipelineInfo *pps.PipelineInfo, rootPath string, ) (Driver, error)
NewDriver constructs a Driver object using the given clients and pipeline settings. It makes blocking calls to determine the user/group to use with the user code on the current worker node, as well as determining if enterprise features are activated (for exporting stats).