worker

package module
v0.0.0-...-c60b214 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 25, 2019 License: GPL-3.0, LGPL-3.0 Imports: 0 Imported by: 0

README

worker

Worker is a worker pool thought to be easily observable. You can plug it with whatever you want, but we provide out of the box a worker pool integrated with Prometheus

Usage

package main

import (
	"log"
	"net/http"
	"time"

	// Include these!
	"github.com/bcmi-labs/worker.go"
	"github.com/bcmi-labs/worker.go/promworker"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

func main() {
	// Start the prometheus handler!
	http.Handle("/metrics", promhttp.Handler())
	log.Printf("Listening prometheus metrics on %s/metrics\n", ":8888")
	go func() {
		log.Fatal(http.ListenAndServe(":8888", nil))
	}()

	// Instantiate a new worker with the name test!
	pool := promworker.New("test")

	// Run a job that sleeps for 10 seconds!
	pool.Run(func() {
		time.Sleep(time.Second * 10)
	})

	// Run another job that registers the duration of actions inside it!
	pool.RunTraced("trace", func(trace worker.Trace) {
		time.Sleep(time.Second * 10)
		// When you call Mark it's like you stopped a stopwatch and recorded
		// the time from the last Mark or the start of the trace!
		trace.Mark("first action")
		time.Sleep(time.Second * 5)
		trace.Mark("second action")
		time.Sleep(time.Second * 1)
		trace.Mark("third action")
	})

	// Block forever!
	select {}
}

This program will expose a prometheus endpoint with the following metrics:

# HELP test_jobs_count The number of jobs currently running in the worker pool
# TYPE test_jobs_count gauge
test_jobs_count 2
# HELP test_jobs_sum The number of jobs processed since the start
# TYPE test_jobs_sum counter
test_jobs_sum 2
# HELP test_trace_nanoseconds The duration in nanoseconds of arbitrary actions performed by jobs in the worker pool
# TYPE test_trace_nanoseconds summary
test_trace_nanoseconds{action="first action",scope="trace",quantile="0.5"} 1.0000253174e+10
test_trace_nanoseconds{action="first action",scope="trace",quantile="0.9"} 1.0000253174e+10
test_trace_nanoseconds{action="first action",scope="trace",quantile="0.99"} 1.0000253174e+10
test_trace_nanoseconds_sum{action="first action",scope="trace"} 1.0000253174e+10
test_trace_nanoseconds_count{action="first action",scope="trace"} 1
test_trace_nanoseconds{action="second action",scope="trace",quantile="0.5"} 5.000191237e+09
test_trace_nanoseconds{action="second action",scope="trace",quantile="0.9"} 5.000191237e+09
test_trace_nanoseconds{action="second action",scope="trace",quantile="0.99"} 5.000191237e+09
test_trace_nanoseconds_sum{action="second action",scope="trace"} 5.000191237e+09
test_trace_nanoseconds_count{action="second action",scope="trace"} 1
test_trace_nanoseconds{action="third action",scope="trace",quantile="0.5"} 1.000101584e+09
test_trace_nanoseconds{action="third action",scope="trace",quantile="0.9"} 1.000101584e+09
test_trace_nanoseconds{action="third action",scope="trace",quantile="0.99"} 1.000101584e+09
test_trace_nanoseconds_sum{action="third action",scope="trace"} 1.000101584e+09
test_trace_nanoseconds_count{action="third action",scope="trace"} 1

Documentation

Overview

Example
package main

import (
	"fmt"
	"time"

	"github.com/bcmi-labs/worker.go"
)

func main() {
	fmt.Println("worker usage")
	mockJobs := mockJobs{}

	pool := worker.Pool{
		Jobs: &mockJobs,
	}

	pool.Run(func() {
		time.Sleep(1 * time.Second)
	})

	fmt.Println("Jobs running:", mockJobs.N)

	pool.Run(func() {
		time.Sleep(1 * time.Second)
	})

	fmt.Println("Jobs running:", mockJobs.N)

	time.Sleep(2 * time.Second)

	fmt.Println("Jobs running:", mockJobs.N)

}

type mockJobs struct {
	N int
}

func (m *mockJobs) Inc() { m.N++ }
func (m *mockJobs) Dec() { m.N-- }
Output:
worker usage
Jobs running: 1
Jobs running: 2
Jobs running: 0

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Pool

type Pool struct {
	// Jobs keep tracks of the number of jobs.
	// Whenever a job is started, Inc is called. Whenever a job finishes, Dec is called
	Jobs interface {
		Inc()
		Dec()
	}
	// Recover is an optional function that's launched to recover a panicing goroutine
	Recover func(interface{})
	// Tracer is used by RunTraced to spawn traces to keep track of the duration of actions during the running job
	Tracer Tracer
}

Pool is a worker pool thought to be easily observable

func (*Pool) Run

func (p *Pool) Run(fn func())

Run spawns a new goroutine with the given function. Jobs.Inc() is called whenever it's enqueued, Jobs.Dec() is called when it finishes. Recover is called whenever the goroutine panics

func (*Pool) RunTraced

func (p *Pool) RunTraced(scope string, fn func(trace Trace))

RunTraced behaves like Run, but provides to the running goroutine a trace to keep track of the duration of actions Of course, if you don't provide a Tracer to the Pool, it will panic

type Trace

type Trace interface {
	Mark(action string)
}

Trace is an object that keep tracks of the duration of an action. When the object is created a timer starts, and whenever Mark is called the object will record the duration of the action described

type Tracer

type Tracer interface {
	New(scope string) Trace
}

Tracer is an object that can spawn traces. Scope is a generic string to assign to the trace in order to organize them

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL