Documentation
¶
Overview ¶
Package mempool provides fixed-capacity, page-aligned mmap buffer pools for storage I/O.
Pools ¶
SlabPool divides one contiguous allocation into fixed-size slots. Its SlabPool.Acquire method is non-blocking, and the complete slab can be registered as one io_uring fixed buffer.
MmapPool owns a fixed set of individually allocated buffers. Its Acquire method blocks until a buffer is available, providing token-bucket backpressure; MmapPool.TryAcquire is the non-blocking variant. Acquired MmapBuffer values have a reference-counted lifecycle.
Both pools allocate and pre-warm their memory at construction time so the hot path does not use the kernel allocator. MmapPool stores raw byte slices internally and wraps one in a fresh MmapBuffer on each acquisition. A stale *MmapBuffer held by a racing reader therefore remains distinct from the next MmapBuffer for the same underlying memory.
Index ¶
- Constants
- Variables
- func SetPanicOnMisuse(v bool)
- type MmapBuffer
- type MmapPool
- func (p *MmapPool) Acquire() *MmapBuffer
- func (p *MmapPool) AcquireAligned(size int64) *MmapBuffer
- func (p *MmapPool) Capacity() int
- func (p *MmapPool) Close()
- func (p *MmapPool) Name() string
- func (p *MmapPool) Outstanding() int64
- func (p *MmapPool) SetSlowAcquireWarning(d time.Duration)
- func (p *MmapPool) TryAcquire() (*MmapBuffer, bool)
- type SlabPool
- type Slot
Constants ¶
const ( // SlabBitsPerShard is the number of slots tracked per shard (one uint64 mask). SlabBitsPerShard = 64 // SlabMaxShards caps the pool at 65 536 slots. SlabMaxShards = 1024 // SlabMaxSlots is the maximum total slot count (SlabMaxShards × SlabBitsPerShard). SlabMaxSlots = SlabMaxShards * SlabBitsPerShard )
Slab geometry constants.
Variables ¶
var ErrSlabExhausted = errors.New("mempool: slab exhausted")
ErrSlabExhausted is returned by Acquire when all slots are in use.
Functions ¶
func SetPanicOnMisuse ¶
func SetPanicOnMisuse(v bool)
SetPanicOnMisuse enables or disables panicking on detected API misuse. The default is false — misuse is silently tolerated, which is safe in the presence of context-cancellation races where multiple paths may attempt to release the same buffer.
Currently detected misuse:
- MmapBuffer.Unpin called more times than MmapPool.Acquire + MmapBuffer.TryInc (reference count driven below zero).
- MmapBuffer.Unpin somehow triggering resetAndRelease on a buffer whose leased flag was already cleared (secondary defence; unreachable in normal usage).
Call with true in your program's init() or TestMain to catch bugs during development and testing.
Types ¶
type MmapBuffer ¶
type MmapBuffer struct {
// contains filtered or unexported fields
}
MmapBuffer is a reference-counted, page-aligned memory buffer.
Obtain one via MmapPool.Acquire or MmapPool.TryAcquire; release each reference with MmapBuffer.Unpin. The underlying memory returns to the pool when the reference count reaches zero.
Multiple concurrent readers can safely share a buffer by calling MmapBuffer.TryInc before accessing it and MmapBuffer.Unpin when done.
func NewMmapBuffer ¶
func NewMmapBuffer(size int64) *MmapBuffer
NewMmapBuffer allocates a standalone, unpooled page-aligned mmap buffer of at least size bytes. Unlike pool-acquired buffers, it is unmapped directly when Unpin reduces the reference count to zero — it never returns to a pool.
Use this for one-off allocations that exceed a pool's slab size.
func (*MmapBuffer) AlignedBytes ¶
func (b *MmapBuffer) AlignedBytes(off int64) []byte
AlignedBytes returns the smallest align.BlockSize-aligned prefix covering off bytes of valid data: raw[:align.PageAlign(off)]. Returns nil for off ≤ 0.
Use this when preparing a buffer for O_DIRECT write, where both the buffer length and the write size must be page-aligned.
func (*MmapBuffer) Bytes ¶
func (b *MmapBuffer) Bytes() []byte
Bytes returns the full, page-aligned underlying byte slice.
func (*MmapBuffer) Cap ¶
func (b *MmapBuffer) Cap() int
Cap returns the size of the underlying allocation in bytes.
func (*MmapBuffer) IsPooled ¶
func (b *MmapBuffer) IsPooled() bool
IsPooled reports whether the buffer was acquired from a pool (true) or allocated as a standalone one-off buffer via NewMmapBuffer (false).
func (*MmapBuffer) TryInc ¶
func (b *MmapBuffer) TryInc() bool
TryInc atomically increments the reference count. Returns false if the buffer has already been released (refCount ≤ 0). Safe for concurrent use without external locks.
Typical pattern for a reader racing an eviction:
if !buf.TryInc() {
// buffer was reclaimed; handle miss
}
defer buf.Unpin()
func (*MmapBuffer) Unpin ¶
func (b *MmapBuffer) Unpin()
Unpin decrements the reference count. When the count reaches zero, the raw memory is returned to the pool (or unmapped for unpooled buffers).
If the count was already zero (i.e. this is an extra release), the call is silently ignored by default. Enable SetPanicOnMisuse to turn over-releases into panics during development.
type MmapPool ¶
type MmapPool struct {
// contains filtered or unexported fields
}
MmapPool is a fixed-capacity pool of pre-allocated, page-aligned mmap buffers. Create with NewMmapPool; close with [Close] after all outstanding buffers have been returned via MmapBuffer.Unpin.
func NewMmapPool ¶
NewMmapPool creates a pool of capacity page-aligned buffers, each of bufferSize bytes (rounded up to the nearest 4 KiB page).
All buffers are pre-allocated and pre-warmed at construction time. Typical usage: NewMmapPool("writes", 1<<20, 32) — 32 × 1 MiB slabs.
func (*MmapPool) Acquire ¶
func (p *MmapPool) Acquire() *MmapBuffer
Acquire blocks until a buffer is available and returns it with refCount=1. The returned *MmapBuffer must eventually be released via MmapBuffer.Unpin.
Acquire panics if called on a closed pool. If [SetSlowAcquireWarning] has been called, a warning is logged each time the pool stays empty for longer than the configured duration.
func (*MmapPool) AcquireAligned ¶
func (p *MmapPool) AcquireAligned(size int64) *MmapBuffer
AcquireAligned returns a buffer large enough to hold at least size bytes. If size fits within the pool's slab size, a pooled buffer is returned. Otherwise an unpooled NewMmapBuffer is allocated and returned directly.
Use this when the required size varies and may occasionally exceed the pool's standard slab (e.g. writing an oversized record).
func (*MmapPool) Close ¶
func (p *MmapPool) Close()
Close drains and releases all buffers remaining in the pool. Must be called only after all outstanding buffers have been returned.
func (*MmapPool) Outstanding ¶
Outstanding returns the number of buffers currently checked out of the pool.
func (*MmapPool) SetSlowAcquireWarning ¶
SetSlowAcquireWarning enables a diagnostic log when Acquire blocks for longer than d. Each time the threshold is crossed, a slog.Warn message is emitted with the pool name and outstanding count, then Acquire resumes waiting. Pass d=0 to disable (the default).
This is invaluable for detecting an undersized pool in production: if Acquire regularly blocks, the pool capacity or buffer size needs tuning.
func (*MmapPool) TryAcquire ¶
func (p *MmapPool) TryAcquire() (*MmapBuffer, bool)
TryAcquire returns a buffer immediately if one is available, or (nil, false) if the pool is currently empty. It is the non-blocking counterpart of [Acquire].
type SlabPool ¶
type SlabPool struct {
// contains filtered or unexported fields
}
SlabPool is a contiguous, page-aligned slab allocator.
A single anonymous mmap region is divided into fixed-size slots tracked by per-shard 64-bit atomic bitmasks. Acquire and Release are wait-free under low contention and spin briefly on the same shard under high contention.
On Linux, if the rounded total size is a multiple of align.HugepageSize (2 MiB), align.AllocAligned will attempt MAP_HUGETLB transparently and fall back to standard 4 KiB pages if huge pages are unavailable.
Create with NewSlabPool; close with [Close] after all Slots have been returned and any io_uring ring or scheduler that registered this pool has been closed. Registration retains the pool but does not transfer ownership.
func NewSlabPool ¶
NewSlabPool creates a SlabPool covering at least totalSize bytes, divided into slots of at least slotSize bytes.
slotSize is rounded up to the nearest align.BlockSize (4 KiB) page boundary automatically. totalSize is rounded up to the nearest align.HugepageSize (2 MiB) boundary.
Returns an error if slotSize ≤ 0, if the pool would require more than SlabMaxSlots slots, or if the total size is too small for even one slot.
func (*SlabPool) Acquire ¶
Acquire returns a free Slot. Returns ErrSlabExhausted if all slots are in use. Non-blocking; the caller is expected to retry or back off.
Starts at a random shard to distribute load across shards under concurrent use, then probes linearly (with wrap-around) until a free slot is found.
func (*SlabPool) Close ¶
func (p *SlabPool) Close()
Close unmaps the slab. Must be called only after all Slots have been released and after any io_uring ring that registered this pool has exited.
func (*SlabPool) Contains ¶
Contains reports whether buf's base pointer lies within this slab. Useful for validating that a buffer was obtained from this pool.
func (*SlabPool) RawData ¶
RawData returns the underlying contiguous slab. The caller must not modify the returned slice. Intended for registering the buffer with io_uring (io_uring_register_buffers).
type Slot ¶
type Slot struct {
Data []byte // usable slice; do not reassign the backing array
// contains filtered or unexported fields
}
Slot is a fixed-size view into a SlabPool slab.
Data is the usable byte slice for the slot. The caller may read and write Data freely, but must not reassign it (e.g. via append that reallocates the backing array). Slot.Release validates the base pointer and panics if Data has been tampered with.
Source Files