Version: v0.0.0-...-3ca95c8 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Dec 7, 2013 License: MIT


relyq Build Status

A relatively simple Redis-backed reliable task queue and state machine.

Its made up of four simpleq's: todo, doing, failed, and done. Tasks will never be dropped on the floor even if a processing server crashes because all operations are atomic. Tasks can be represented as any data type.

Note: relyq assumes all tasks have IDs. Thus you must pass in a pointer to an object that has an Id() method (or its element has an Id() method.)

There are a few redis clients for Go but this package uses redigo


Documentation on godoc.org


go get github.com/Rafflecopter/golang-relyq/relyq


import (
  redisstorage "github.com/Rafflecopter/golang-relyq/storage/redis"

func CreateRelyQ(pool *redis.Pool) *relyq.Queue {
  cfg := &relyq.Config{
    Prefix: "my-relyq", // Required
    Delimiter: ":", // Defaults to :
    IdField: "id", // ID field for tasks
    UseDoneQueue: false, // Whether to keep list of "done" tasks (default false)
    KeepDoneTasks: false, // Whether to keep the backend storage of "done" tasks (default false)

  storage := redisstorage.New(redisstorage.JSONMarshaller, pool, cfg.Prefix, cfg.Delimiter)

  return relyq.New(pool, storage, cfg)

func QuickCreateRelyQ(pool *redis.Pool) *relyq.Queue {
  return relyq.NewRedisJson(pool, &relyq.Config{Prefix: "my-relyq"})


Create your task types:

type Task struct {

  IdField string
  SomethingElse *OtherStruct

Basic use:

q := CreateRelyQ(redisPool)

// Push a task on to Todo queue
err := q.Push(&Task{SomethingElse: &OtherStruct{}})

// Move a task from Todo to Doing
task := new(Task)
ok, err := q.Process(task) // ok=false means nothing in task

// Block and process a task
err := q.BProcess(task)

// Finish a task
err := q.Finish(task)

// Or fail it
err := q.Fail(task)

// Remove a task from the Failed queue
err := q.Remove(q.Failed, task)

// Eventually
err := q.Close()

Or use a listener:

q := CreateRelyQ(redisPool)

var example *Task
l := q.Listen(example)

go func() {
  for err := range l.Errors {
    // Do something with errors

go func() {
  for task := range l.Tasks {
    // Do something with tasks
    err := q.Finish(task.(*Task))

// Eventually
err := q.Close()


go test

Storage Options

Normal operation stores the full task description or object in the queue itself. This can be inefficient for LREM operations. Sometimes one might even want to store task descriptions in a separate datastore than redis to save memory. Custom backends have been created for this purpose.


The Redis backend stores serialized task objects in Redis. Each options object also accepts the storage_prefix field to set the prefix for where task objects are stored. Documentation

Create a storage object:

storage := redisstorage.New(marshallers.JsonMarshaller, pool, cfg.Prefix, cfg.Delimiter)

Or, for skipping the storage step, use this handy shortcut

q := relyq.NewRedisJson(pool, cfg)


  • deferred tasks
  • recurring tasks


See LICENSE file.


Path Synopsis
Package marshallers provides objects for marshalling tasks into bytes
Package marshallers provides objects for marshalling tasks into bytes
Package relyq provides a reliable queue backed by redis
Package relyq provides a reliable queue backed by redis
Package redis provides a Storage backend for relyq
Package redis provides a Storage backend for relyq

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL