Documentation
¶
Overview ¶
Package pqueue provides privitives for processing a simple persistent queue backed by the local filesystem. In many ways, the approach is similar to Maildir. Submitting jobs can be done easily from any language by creating a directory and moving it atomically into a directory.
A pqueue is a directory with several subdirectories:
- New files and directories are created in the `tmp` subdirectory before being moved atomically to elsewhere in the directory structure.
- The `new` subdirectory contains jobs that workers should process (see the `Take` method).
- The `cur` subdirectory contains a subdirectory for each worker (named by its process ID) where jobs are placed while the worker processes them.
- When jobs fail or finish successfully, they are moved to the `failed` or `done` subdirectories, respectively. See the `Fail` and `Finish` methods.
Jobs have state in the form of properties, which are really just files inside the job's directory. The `Get` and `Set` methods read these properties and set them atomically. Properties are set both by the process submitting a job (to specify the work that is to be done) and by workers (to checkpoint its progress so the job can continue if interrupted).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Job ¶
type Job struct {
Basename string
// contains filtered or unexported fields
}
type Queue ¶
type Queue struct {
// contains filtered or unexported fields
}
func OpenQueue ¶
Open a pqueue. The directory `dir` must already exist. The subdirectories (`new`, `cur`, etc.) will be created if they are missing.
func (*Queue) CreateJob ¶
Create a job in the `tmp` directory of the queue. After you finish preparing the job with `Set`, call the `Submit` method to make the job available to workers.
func (*Queue) RescueDeadJobs ¶
Go through the `cur` subdirectory, determine which workers are no longer alove, and resubmit the jobs they were processing when they died.
Source Files