jobExecutor

README

jobExecutor

go module to assist in running jobs in multiple goroutines and print their output

features:

• Can set the max concurrent jobs with: SetMaxConcurrentJobs, default to runtime. GOMAXPROCS ()
• Can run commands and "runnable" functions (they must return a string and an error)
• Can register handlers for the following events:
  • OnJobsStart: called before any job start
  • OnJobStart: called before each job start
  • OnJobDone: called after each job terminated
  • OnJobsDone: called after all jobs are terminated
• Fluent interface, you can chain methods
• Can add jobs programmatically
• Can display a progress report of ongoing jobs
• Can display output using custom templates

Usage:

Adding some jobs and executing them

package main

import (
	"errors"
	"fmt"
	"math/rand"
	"os"
	"strings"
	"time"

	"github.com/software-t-rex/go-jobExecutor"
)

func longFunction() (string, error) {
	duration := time.Duration(rand.Intn(5)) * time.Millisecond
	time.Sleep(duration)
	if rand.Intn(10) <= 7 { // random failure
		return fmt.Sprintf("- runnable succeed in %v\n", duration), nil
	}
	return fmt.Sprintf("- runnable Failed in %v\n", duration), errors.New("error while asleep")
}

func longFunction2() (string, error) {
	res, err := longFunction()
	if err == nil {
		res = strings.Replace(res, "runnable", "runnable2", -1)
	}
	return res, err
}

func main() {
	// set max concurrent jobs (not required default to GOMAXPROCS)
	jobExecutor.SetMaxConcurrentJobs(8)
	executor := jobExecutor.NewExecutor()
	// add some "runnable" functions
	executor.AddJobFns(longFunction, longFunction2)
	// add a single command
	executor.AddJobCmd("ls", "-l")
	// or multiple command at once
	executor.AddJobCmds([][]string{
		{"sleep", "5"},
		{"sleep", "2"},
	}...)

	// execute them and get errors if any
	jobErrors := executor.Execute()
	if len(jobErrors) > 0 {
		fmt.Fprintln(os.Stderr, jobErrors)
	}
}

Binding some event handlers:

func main () {
	executor := jobExecutor.NewExecutor()

	// add a simple command
	executor.AddJobCmd("sleep", "5")

	// binding some event handlers (can be done anytime before calling Execute)
	// you can call the same method multiple times to bind more than one handler
	// they will be called in order
	executor.
		OnJobsStart(func(jobs jobExecutor.JobList) {
			fmt.Printf("Starting %d jobs\n", len(jobs))
		}).
		OnJobStart(func (jobs jobExecutor.JobList, jobId int) {
			fmt.Printf("Starting jobs %d\n", jobId)
		}).
		OnJobDone(func (jobs jobExecutor.JobList, jobId int) {
			job:=jobs[jobId]
			if job.IsState(jobExecutor.JobStateFailed) {
				fmt.Printf("job %d terminanted with error: %s\n", jobId, job.Err)
			}
		}).
		OnJobsDone(func (jobExecutor.JobList) {
			fmt.Println("Done")
		})

	// add some "runnable" functions and execute
	executor.AddJobFns( longFunction, longFunction2).Execute()
}

Display state of running jobs:

func main() {
	jobExecutor.SetMaxConcurrentJobs(5)
	executor := jobExecutor.NewExecutor().WithProgressOutput()
	executor.AddJobCmds([][]string{
		{"sleep", "10"},
		{"sleep", "9"},
		{"sleep", "8"},
		{"sleep", "7"},
		{"sleep", "6"},
		{"sleep", "5"},
		{"sleep", "4"},
		{"sleep", "3"},
		{"sleep", "2"},
		{"sleep", "1"},
	}...).Execute()
}

Other outputs methods:

• WithOrderedOutput: output ordered res and errors at the end
• WithFifoOutput: output res and errors as they arrive
• WithStartOutput: output a line when launching a job
• WithStartSummary: output a summary of jobs to do

All output methods use a go template which you can override by calling the method

jobExecutor.SetTemplateString(myTemplateString)

the template string must contains following templates definition:

• startSummary
• jobStatusLine
• jobStatusFull
• doneReport
• startProgressReport
• progressReport You can look at output.gtpl file for an example

Documentation

Overview

Copyright © 2023 Jonathan Gotti <jgotti at jgotti dot org> SPDX-FileType: SOURCE SPDX-License-Identifier: MIT SPDX-FileCopyrightText: 2023 Jonathan Gotti <jgotti@jgotti.org>

Copyright © 2023 Jonathan Gotti <jgotti at jgotti dot org> SPDX-FileType: SOURCE SPDX-License-Identifier: MIT SPDX-FileCopyrightText: 2023 Jonathan Gotti <jgotti@jgotti.org>

Copyright © 2023 Jonathan Gotti <jgotti at jgotti dot org> SPDX-FileType: SOURCE SPDX-License-Identifier: MIT SPDX-FileCopyrightText: 2023 Jonathan Gotti <jgotti@jgotti.org>

Copyright © 2023 Jonathan Gotti <jgotti at jgotti dot org> SPDX-FileType: SOURCE SPDX-License-Identifier: MIT SPDX-FileCopyrightText: 2023 Jonathan Gotti <jgotti@jgotti.org>

Index

• Constants
• func SetMaxConcurrentJobs(n int)
• func SetTemplateString(templateString string)
• type ExecuteOptions
• type JobError
  • func NewJobError(jobId int, err error) JobError
  • func (e *JobError) Error() string
  • func (e *JobError) String() string
• type JobExecutor
  • func NewExecutor() *JobExecutor
  • func (e *JobExecutor) AddJobCmd(cmd string, args ...string) *JobExecutor
  • func (e *JobExecutor) AddJobCmds(cmdsAndArgs ...[]string) *JobExecutor
  • func (e *JobExecutor) AddJobFns(fns ...runnableFn) *JobExecutor
  • func (e *JobExecutor) Execute() JobsError
  • func (e *JobExecutor) Len() int
  • func (e *JobExecutor) OnJobDone(fn jobEventHandler) *JobExecutor
  • func (e *JobExecutor) OnJobStart(fn jobEventHandler) *JobExecutor
  • func (e *JobExecutor) OnJobsDone(fn jobsEventHandler) *JobExecutor
  • func (e *JobExecutor) OnJobsStart(fn jobsEventHandler) *JobExecutor
  • func (e *JobExecutor) WithFifoOutput() *JobExecutor
  • func (e *JobExecutor) WithOrderedOutput() *JobExecutor
  • func (e *JobExecutor) WithProgressOutput() *JobExecutor
  • func (e *JobExecutor) WithStartOutput() *JobExecutor
  • func (e *JobExecutor) WithStartSummary() *JobExecutor
• type JobList
• type JobsError
  • func (es JobsError) Error() string
  • func (es JobsError) String() string

Constants

const (
	JobStatePending = 0
	JobStateRunning = 1
	JobStateDone    = 2
	JobStateSucceed = 4
	JobStateFailed  = 8
)

Functions

func SetMaxConcurrentJobs

func SetMaxConcurrentJobs(n int)

set the default number of concurrent jobs to run default to GOMAXPROCS

func SetTemplateString

func SetTemplateString(templateString string)

Template for all outputs related to jobs it must define the following templates: - startSummary: which will receive a JobList - jobStatus: which will receive a single job - progressReport: which will receive a jobList - doneReport: which will receive a jobList

Types

type ExecuteOptions

type ExecuteOptions struct {
	// contains filtered or unexported fields
}

type JobError

type JobError struct {
	Id            int
	OriginalError error
}

func NewJobError

func NewJobError(jobId int, err error) JobError

func (*JobError) Error

func (e *JobError) Error() string

func (*JobError) String

func (e *JobError) String() string

type JobExecutor

type JobExecutor struct {
	// contains filtered or unexported fields
}

func NewExecutor

func NewExecutor() *JobExecutor

instanciate a new JobExecutor

func (*JobExecutor) AddJobCmd

func (e *JobExecutor) AddJobCmd(cmd string, args ...string) *JobExecutor

func (*JobExecutor) AddJobCmds

func (e *JobExecutor) AddJobCmds(cmdsAndArgs ...[]string) *JobExecutor

Add miltiple job command to execute

func (*JobExecutor) AddJobFns

func (e *JobExecutor) AddJobFns(fns ...runnableFn) *JobExecutor

Add one or more job function to run (func() (string, error))

func (*JobExecutor) Execute

func (e *JobExecutor) Execute() JobsError

effectively execute jobs and return collected errors as JobsError

func (*JobExecutor) Len

func (e *JobExecutor) Len() int

return the total number of jobs in the pool

func (*JobExecutor) OnJobDone

func (e *JobExecutor) OnJobDone(fn jobEventHandler) *JobExecutor

Add an handler which will be call after a jobs is terminated

func (*JobExecutor) OnJobStart

func (e *JobExecutor) OnJobStart(fn jobEventHandler) *JobExecutor

Add an handler which will be call before a jobs is started

func (*JobExecutor) OnJobsDone

func (e *JobExecutor) OnJobsDone(fn jobsEventHandler) *JobExecutor

Add an handler which will be call after all jobs are terminated

func (*JobExecutor) OnJobsStart

func (e *JobExecutor) OnJobsStart(fn jobsEventHandler) *JobExecutor

Add an handler which will be call before any jobs is started

func (*JobExecutor) WithFifoOutput

func (e *JobExecutor) WithFifoOutput() *JobExecutor

Display full jobStatus as they arrive

func (*JobExecutor) WithOrderedOutput

func (e *JobExecutor) WithOrderedOutput() *JobExecutor

display doneReport when all jobs are Done

func (*JobExecutor) WithProgressOutput

func (e *JobExecutor) WithProgressOutput() *JobExecutor

will override onJobStarts / onJobStart / onJobDone handlers previsously defined generally you should avoid using these method with other handlers bound to the JobExecutor instance

func (*JobExecutor) WithStartOutput

func (e *JobExecutor) WithStartOutput() *JobExecutor

Output a line to say a job is starting

func (*JobExecutor) WithStartSummary

func (e *JobExecutor) WithStartSummary() *JobExecutor

Output a summary of the job that will be run

type JobList

type JobList []*job

type JobsError

type JobsError map[int]JobError

func (JobsError) Error

func (es JobsError) Error() string

func (JobsError) String

func (es JobsError) String() string