Documentation ¶
Overview ¶
Package sej implements file-based segmented journal (queue)
Index ¶
- Variables
- func JournalDirPath(dir string) string
- func OffsetDirPath(dir string) string
- func ReadOffset(r io.ReadSeeker) (offset uint64, err error)
- func WatchRootDir(rootDir string, watchInterval time.Duration, open OpenFunc) error
- func WriteMessage(w io.Writer, buf []byte, m *Message) (int64, error)
- type CorruptionError
- type DefaultOffset
- type JournalDir
- type JournalFile
- type JournalFiles
- type Message
- type Offset
- type OpenFunc
- type ScanOffsetError
- type ScanTruncatedError
- type Scanner
- type Test
- type Writer
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCRC is returned when the CRC of an message value does not match the stored CRC ErrCRC = errors.New("CRC mismatch") // ErrTimeout is returned when no message can be obtained within Reader.Timeout ErrTimeout = errors.New("read timeout") )
var ( // ErrLocked is returned when another writer has already gotten the lock ErrLocked = errors.New("file is already locked") )
var ( // NotifyTimeout is the timeout value in rare cases that the OS notification fails // to capture the file/directory change events NotifyTimeout = time.Hour )
Functions ¶
func JournalDirPath ¶
func OffsetDirPath ¶
func ReadOffset ¶
func ReadOffset(r io.ReadSeeker) (offset uint64, err error)
ReadOffset reads the offset stored in an ofs file (r)
func WatchRootDir ¶
WatchRootDir watches the directory and calls open only once for each shard
Types ¶
type CorruptionError ¶
type CorruptionError struct { File string Offset uint64 Timestamp time.Time Message []byte Err error FixErr error }
CorruptionError is returned when the last message of a segmented journal file is corrupted
func (*CorruptionError) Error ¶
func (e *CorruptionError) Error() string
type JournalDir ¶
type JournalDir struct { Files JournalFiles // contains filtered or unexported fields }
func OpenJournalDir ¶
func OpenJournalDir(dir string) (*JournalDir, error)
func (*JournalDir) First ¶
func (d *JournalDir) First() *JournalFile
func (*JournalDir) Last ¶
func (d *JournalDir) Last() *JournalFile
type JournalFile ¶
JournalFile represents a unopened journal file
func ParseJournalFileName ¶
func ParseJournalFileName(dir, name string) (*JournalFile, error)
ParseJournalFileName parses a journal file name and returns an JournalFile object
func (*JournalFile) FirstMessage ¶
func (journalFile *JournalFile) FirstMessage() (*Message, error)
func (*JournalFile) LastMessage ¶
func (journalFile *JournalFile) LastMessage() (*Message, error)
func (*JournalFile) LastOffset ¶
func (journalFile *JournalFile) LastOffset() (uint64, error)
LatestOffset returns the offset after the last message in a journal file
func (*JournalFile) LastReadableOffset ¶
func (journalFile *JournalFile) LastReadableOffset() (uint64, error)
type JournalFiles ¶
type JournalFiles []JournalFile
func (JournalFiles) Len ¶
func (a JournalFiles) Len() int
func (JournalFiles) Less ¶
func (a JournalFiles) Less(i, j int) bool
func (JournalFiles) Swap ¶
func (a JournalFiles) Swap(i, j int)
type Offset ¶
type Offset struct { Syncing bool // contains filtered or unexported fields }
Offset is used to manage a disk-persisted offset
func NewOffset ¶
func NewOffset(dir, name string, defaultOffset DefaultOffset) (*Offset, error)
NewOffset creates a new Offset object persisted to dir/ofs/name.ofs
func OpenReadonlyOffset ¶
type ScanOffsetError ¶
func (*ScanOffsetError) Error ¶
func (e *ScanOffsetError) Error() string
type ScanTruncatedError ¶
func (*ScanTruncatedError) Error ¶
func (e *ScanTruncatedError) Error() string
type Scanner ¶
type Scanner struct { Timeout time.Duration // read timeout when no data arrived, default 0 // contains filtered or unexported fields }
Scanner implements reading of messages from segmented journal files
func NewScanner ¶
NewScanner creates a scanner for reading dir/jnl starting from offset Default Timeout is 1 second
type Test ¶
Test is a collection of test utility methods
func (Test) NewDir ¶
NewDir creates a new test directory the path will be deleted automatically after the tests
func (Test) VerifyMessageValues ¶
func (Test) VerifyMessages ¶
type Writer ¶
type Writer struct { SegmentSize int // contains filtered or unexported fields }
Writer writes to segmented journal files
func NewWriter ¶
NewWriter creates a new writer for writing to dir/jnl with file size at least segmentSize
func (*Writer) Close ¶
Close closes the writer, flushes the buffer and syncs the file to the hard drive
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package hub is a generated protocol buffer package.
|
Package hub is a generated protocol buffer package. |
internal
|
|
reader
Package bufio implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or Writer) that also implements the interface but provides buffering and some help for textual I/O.
|
Package bufio implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or Writer) that also implements the interface but provides buffering and some help for textual I/O. |