Documentation
¶
Overview ¶
Package orderedwriter implements an unbounded buffer for ordering concurrent writes against a non-seekable io.Writer.
Concurrent downloaders (e.g. the S3 manager) often produce chunks out of order but expect them written in offset order. OrderedWriterAt buffers out-of-order chunks in a linked list and flushes them to the underlying writer as soon as the next expected offset becomes available.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type OrderedWriterAt ¶
type OrderedWriterAt struct {
// contains filtered or unexported fields
}
OrderedWriterAt wraps an io.Writer and accepts WriteAt calls from multiple goroutines, flushing them in offset order.
func New ¶
func New(w io.Writer) *OrderedWriterAt
New creates an OrderedWriterAt that writes to w in offset order.
func (*OrderedWriterAt) WriteAt ¶
func (w *OrderedWriterAt) WriteAt(p []byte, offset int64) (int, error)
WriteAt writes p at the given offset. If offset is the next expected byte, p is written straight through to the underlying writer; otherwise p is copied (because callers may reuse the slice before it is flushed) and queued. After queueing, any prefix of the queue that is now contiguous with written is flushed.
Bytes at offsets below the written watermark are dropped (or the chunk is trimmed when it straddles the watermark): the SDK retries parts on error and a retried WriteAt may re-issue bytes that were already flushed. Re-buffering them would stall the flush loop forever, because the stale chunk's offset can never equal the watermark again.
Source Files