README
¶
Murder of crows
Miniature extendable queue orchestration library, capable of creating multiple equal sized queues that can be consumed in parallel without needing to lock the entire queue for processing from a single worker.
Usage
Create a Murder object to manage queues, assigning a Crow object as an adapter to communicate with any atomic-enabled system, a RedisCrow is built into the library.
import "github.com/mohamed-essam/murder-of-crows"
import "redis.v5"
import "fmt"
func main() {
redisClient := redis.NewClient(&redis.Options{Addr: "127.0.0.1:6379", Password: "", DB: 0})
redisCrow := &murder.RedisCrow{Redis: redisClient}
// Create a new murder instance with 100 as a bulk size, 1 second TTL and a worker_group_name
// Worker group name separates different queue systems by prefixing them
// 100 is a rough number as no locking is used in enqueueing to speed this operation up, the actual queue size will be >= 100
murderInstance := murder.NewMurder(100, 1, redisCrow, "worker_group_name")
for (i := 0; i < 100; i++) {
murderInstance.Add(i)
}
lockKey, locked := murderInstance.Lock() // This locks a full queue giving only the worker with lockKey access to it
if (locked) { // this may be false if there are no full queues
jobs := murderInstance.Get(lockKey) // jobs are always returned as an array of strings
for _, obj := range jobs { // Prints out 0 through 99
fmt.Printf("%s\n", obj)
}
}
}
Documentation
¶
Index ¶
- type Crow
- type Murder
- type RedisCrow
- func (c *RedisCrow) AddToQueue(groupName string, obj interface{}, ageConfigured bool)
- func (c *RedisCrow) ClearQueue(queueName string, groupID string) error
- func (c *RedisCrow) CreateLockKey(queueName string, lockKey string, TTL int) bool
- func (c *RedisCrow) CurrentQueue(groupName string) string
- func (c *RedisCrow) ExtendLockKey(lockKey string, TTL int)
- func (c *RedisCrow) FindQueueByKey(lockKey string) (string, bool)
- func (c *RedisCrow) GetQueueContents(queueName string) []string
- func (c *RedisCrow) GetReadyQueues(groupID string) []string
- func (c *RedisCrow) IsLocked(queueName string) bool
- func (c *RedisCrow) MoveToReady(groupName, newName string)
- func (c *RedisCrow) QueueSize(groupName string) int
- func (c *RedisCrow) QueueTimeSinceCreation(groupName string) int
- func (c *RedisCrow) RemoveLockKey(lockKey string)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Crow ¶
type Crow interface {
QueueSize(string) int // Query main queue size
QueueTimeSinceCreation(string) int // Query time since queue creation
AddToQueue(string, interface{}, bool) // Add object to queue
GetQueueContents(string) []string // Retrieve all contents of queue
ClearQueue(string, string) error // Clear all queue contents
CreateLockKey(string, string, int) bool // Create lock key for a queue, confirm if lock acquired, and set TTL
IsLocked(string) bool // Check if a queue is locked
FindQueueByKey(string) (string, bool) // Get the queue for a lock key if exists
ExtendLockKey(string, int) // Extend TTL of lock key to value provided
RemoveLockKey(string) // Removes a lock key if exists
MoveToReady(string, string) // Move a queue to ready to process queues
GetReadyQueues(string) []string // Get full queues
}
Crow : Interface for any storage system for the orchestrator to use
type Murder ¶
type Murder struct {
// contains filtered or unexported fields
}
Murder : Orchestra for queueing systems Provides availability via introducing multiple queues, locking and clearing
func NewMurderWithAge ¶
func (*Murder) Ack ¶
Ack : Acknowledge processing of a queue lock extending its time to kill Useful for long running jobs
func (*Murder) AgeConfigured ¶
func (*Murder) Get ¶
Get : Get contents of a queue given its lock key Ensuring the worker locked the queue and acquired the lock key
func (*Murder) Lock ¶
Lock : Lock a queue returning a lock key that is needed for acknowledging the processing of the queue If no queue is ready to process, returns empty string and false
Source Files