jobExecutor

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 2, 2023 License: MIT Imports: 11 Imported by: 0

README

jobExecutor

go module to assist in running jobs in multiple goroutines and print their output

features:

  • Can set the max concurrent jobs with: SetMaxConcurrentJobs, default to runtime. GOMAXPROCS ()
  • Can run commands and "runnable" functions (they must return a string and an error)
  • Can register handlers for the following events:
    • OnJobsStart: called before any job start
    • OnJobStart: called before each job start
    • OnJobDone: called after each job terminated
    • OnJobsDone: called after all jobs are terminated
  • Fluent interface, you can chain methods
  • Can add jobs programmatically
  • Can display a progress report of ongoing jobs
  • Can display output using custom templates

Usage:

Adding some jobs and executing them

package main

import (
	"errors"
	"fmt"
	"math/rand"
	"os"
	"os/exec"
	"strings"
	"time"

	"github.com/software-t-rex/go-jobExecutor"
)

func longFunction() (string, error) {
	duration := time.Duration(rand.Intn(5)) * time.Millisecond
	time.Sleep(duration)
	if rand.Intn(10) <= 7 { // random failure
		return fmt.Sprintf("- runnable succeed in %v\n", duration), nil
	}
	return fmt.Sprintf("- runnable Failed in %v\n", duration), errors.New("error while asleep")
}

func longFunction2() (string, error) {
	res, err := longFunction()
	if err == nil {
		res = strings.Replace(res, "runnable", "runnable2", -1)
	}
	return res, err
}

func main() {
	// set max concurrent jobs (not required default to GOMAXPROCS)
	jobExecutor.SetMaxConcurrentJobs(8)
	executor := jobExecutor.NewExecutor()
	// add some "runnable" functions
	executor.AddJobFns(longFunction, longFunction2)
	// add a single command
	executor.AddJobCmds(exec.Command("ls", "-l"))
	// or multiple command at once
	executor.AddJobCmds(
		exec.Command("sleep", "5"),
		exec.Command("sleep", "2"),
	)

	// execute them and get errors if any
	jobErrors := executor.Execute()
	if len(jobErrors) > 0 {
		fmt.Fprintln(os.Stderr, jobErrors)
	}
}

Binding some event handlers:

func main () {
	executor := jobExecutor.NewExecutor()

	// add a simple command
	executor.AddJobCmds(exec.Command("sleep", "5"))

	// binding some event handlers (can be done anytime before calling Execute)
	// you can call the same method multiple times to bind more than one handler
	// they will be called in order
	executor.
		OnJobsStart(func(jobs jobExecutor.JobList) {
			fmt.Printf("Starting %d jobs\n", len(jobs))
		}).
		OnJobStart(func (jobs jobExecutor.JobList, jobId int) {
			fmt.Printf("Starting jobs %d\n", jobId)
		}).
		OnJobDone(func (jobs jobExecutor.JobList, jobId int) {
			job:=jobs[jobId]
			if job.IsState(jobExecutor.JobStateFailed) {
				fmt.Printf("job %d terminanted with error: %s\n", jobId, job.Err)
			}
		}).
		OnJobsDone(func (jobExecutor.JobList) {
			fmt.Println("Done")
		})

	// add some "runnable" functions and execute
	executor.AddJobFns( longFunction, longFunction2).Execute()
}

Display state of running jobs:


func main() {
	jobExecutor.SetMaxConcurrentJobs(5)
	executor := jobExecutor.NewExecutor().WithProgressOutput()
	// add a command and set its display name in output templates (there's a AddNamedJobFn too)
	executor.AddNamedJobCmd("Wait fot 2 seconds", exec.Command("sleep", "2"))

	executor.AddJobCmds(
		exec.Command("sleep", "10"),
		exec.Command("sleep", "9"),
		exec.Command("sleep", "8"),
		exec.Command("sleep", "7"),
		exec.Command("sleep", "6"),
		exec.Command("sleep", "5"),
		exec.Command("sleep", "4"),
		exec.Command("sleep", "3"),
		exec.Command("sleep", "2"),
		exec.Command("sleep", "1"),
	).Execute()
}

Other outputs methods:

  • WithOrderedOutput: output ordered res and errors at the end
  • WithFifoOutput: output res and errors as they arrive
  • WithStartOutput: output a line when launching a job
  • WithStartSummary: output a summary of jobs to do

All output methods use a go template which you can override by calling the method

jobExecutor.SetTemplateString(myTemplateString)

the template string must contains following templates definition:

  • startSummary
  • jobStatusLine
  • jobStatusFull
  • doneReport
  • startProgressReport
  • progressReport You can look at output.gtpl file for an example

Documentation

Overview

Copyright © 2023 Jonathan Gotti <jgotti at jgotti dot org> SPDX-FileType: SOURCE SPDX-License-Identifier: MIT SPDX-FileCopyrightText: 2023 Jonathan Gotti <jgotti@jgotti.org>

Index

Constants

View Source 
const (
	JobStatePending = 0
	JobStateRunning = 1
	JobStateDone    = 2
	JobStateSucceed = 4
	JobStateFailed  = 8
)

Variables

This section is empty.

Functions

func SetMaxConcurrentJobs 

func SetMaxConcurrentJobs(n int)

set the default number of concurrent jobs to run default to GOMAXPROCS

func SetTemplateString 

func SetTemplateString(templateString string)

Template for all outputs related to jobs It must define the following templates:

  • startSummary: which will receive a JobList
  • jobStatus: which will receive a single job
  • progressReport: which will receive a jobList
  • doneReport: which will receive a jobList

Types

type JobExecutor 

type JobExecutor struct {
	// contains filtered or unexported fields
}

func NewExecutor 

func NewExecutor() *JobExecutor

Instanciate a new JobExecutor

func (*JobExecutor) AddJobCmds 

func (e *JobExecutor) AddJobCmds(cmds ...*exec.Cmd) *JobExecutor

Add multiple job commands to run

func (*JobExecutor) AddJobFns 

func (e *JobExecutor) AddJobFns(fns ...runnableFn) *JobExecutor

Add one or more job function to run (func() (string, error))

func (*JobExecutor) AddNamedJobCmd added in v1.0.0 

func (e *JobExecutor) AddNamedJobCmd(name string, cmd *exec.Cmd) *JobExecutor

Add a job command and set its output display name

func (*JobExecutor) AddNamedJobFn added in v1.0.0 

func (e *JobExecutor) AddNamedJobFn(name string, fn runnableFn) *JobExecutor

Add a job function and set its output display name

func (*JobExecutor) Execute 

func (e *JobExecutor) Execute() JobsError

Effectively execute jobs and return collected errors as JobsError

func (*JobExecutor) Len 

func (e *JobExecutor) Len() int

Return the total number of jobs added to the jobExecutor

func (*JobExecutor) OnJobDone 

func (e *JobExecutor) OnJobDone(fn jobEventHandler) *JobExecutor

Add a handler which will be called after a job is terminated

func (*JobExecutor) OnJobStart 

func (e *JobExecutor) OnJobStart(fn jobEventHandler) *JobExecutor

Add a handler which will be called before a job is started

func (*JobExecutor) OnJobsDone 

func (e *JobExecutor) OnJobsDone(fn jobsEventHandler) *JobExecutor

Add a handler which will be called after all jobs are terminated

func (*JobExecutor) OnJobsStart 

func (e *JobExecutor) OnJobsStart(fn jobsEventHandler) *JobExecutor

Add a handler which will be called before any jobs is started

func (*JobExecutor) WithFifoOutput 

func (e *JobExecutor) WithFifoOutput() *JobExecutor

Display full jobStatus as they arrive

func (*JobExecutor) WithOrderedOutput 

func (e *JobExecutor) WithOrderedOutput() *JobExecutor

Display doneReport when all jobs are Done

func (*JobExecutor) WithProgressOutput 

func (e *JobExecutor) WithProgressOutput() *JobExecutor

Display a job status report updated each time a job start or end be carefull when dealing with other handler that generate output as it will potentially break progress output

func (*JobExecutor) WithStartOutput 

func (e *JobExecutor) WithStartOutput() *JobExecutor

Output a line to say a job is starting

func (*JobExecutor) WithStartSummary 

func (e *JobExecutor) WithStartSummary() *JobExecutor

Output a summary of jobs that will be run

type JobList 

type JobList []*job

type JobsError 

type JobsError map[int]error

Map indexes correspond to job index in the queue

func (JobsError) Error 

func (es JobsError) Error() string

func (JobsError) String 

func (es JobsError) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.