Documentation
¶
Overview ¶
Package sys provides OS-level file descriptor primitives for page-cache bypass.
KeepAlive mandate ¶
Every exported function that calls a raw syscall via an fd extracted from *os.File calls runtime.KeepAlive(f) unconditionally immediately after the syscall — before any error checking. This prevents the Go GC from finalizing f (and closing its fd) while the kernel still has it open:
err := unix.Fdatasync(int(f.Fd()))
runtime.KeepAlive(f) // <— unconditional, before error check
if err != unix.EINTR { … }
Cross-platform ¶
Build-tagged files (io_linux.go, io_darwin.go) supply platform implementations for Fdatasync, Fallocate, PunchHole, Fadvise, CreateDirect, OpenDirect, and CopyFileRange. This file supplies shared types and pure-computation helpers.
Index ¶
- Constants
- Variables
- func AlignForHolePunch(offset, length int64) (int64, int64, bool)
- func CopyFileRange(srcFile, dstFile *os.File, srcOff, dstOff *int64, length int) (int, error)
- func CreateAndAllocate(path string, flags OpenFlag, allocSize int64) (*os.File, error)
- func CreateDirect(path string, flags OpenFlag) (*os.File, error)
- func Fadvise(f *os.File, offset Offset_t, length int64, hint FadviseHint) error
- func Fallocate(f *os.File, size int64) error
- func Fdatasync(f *os.File) error
- func Ftruncate(f *os.File, size int64) error
- func IsAligned(block []byte) bool
- func IsTransientIOError(err error) bool
- func OpenDirect(path string, flags OpenFlag) (*os.File, error)
- func PreadAligned(f *os.File, buf []byte, offset int64, flags OpenFlag) (int, error)
- func PunchHole(f *os.File, offset, length int64) (int64, error)
- func SyncFile(f *os.File, flags OpenFlag) error
- func WriteFile(path string, data []byte, flags OpenFlag) (retErr error)
- type FadviseHint
- type Offset_t
- type OpenFlag
Constants ¶
const ( SyncNone OpenFlag = 0 SyncData = FlDSync SyncFull = FlSync )
Common flag combinations.
const ( // UseFadvise reports whether posix_fadvise is effective on this platform. UseFadvise = true // RequiresAlignment reports whether O_DIRECT requires 4 KiB-aligned // buffers, offsets, and transfer sizes. Always true on Linux. RequiresAlignment = true // RequiresExplicitSync reports whether explicit sync calls are needed // for durable writes. On Linux, O_DSYNC/O_SYNC at open time suffices. RequiresExplicitSync = false )
Platform capability constants.
Variables ¶
var ErrAlignment = errors.New("dio/sys: buffer not aligned for O_DIRECT")
ErrAlignment is returned when a buffer is not properly aligned for O_DIRECT.
Functions ¶
func AlignForHolePunch ¶
AlignForHolePunch computes filesystem-block-aligned boundaries for FALLOC_FL_PUNCH_HOLE / F_PUNCHHOLE so that adjacent blobs are not punched.
Returns (alignedOffset, alignedLength, canPunch). canPunch is false when the adjusted range is smaller than one block.
func CopyFileRange ¶
CopyFileRange copies length bytes from srcFile at *srcOff to dstFile at *dstOff using the copy_file_range(2) syscall. On XFS with reflinks this is a metadata-only operation (zero physical I/O). Retries automatically on EINTR.
func CreateAndAllocate ¶
CreateAndAllocate creates path with flags and reserves allocSize bytes via Fallocate.
func CreateDirect ¶
CreateDirect creates path for writing. Always uses O_CREATE|O_WRONLY|O_TRUNC. Pass FlDirectIO to add O_DIRECT; FlDSync / FlSync to add O_DSYNC / O_SYNC. Falls back to buffered I/O if the filesystem rejects O_DIRECT.
func Fadvise ¶
Fadvise advises the kernel of the expected access pattern for f. Maps the cross-platform FadviseHint to Linux posix_fadvise constants. Retries automatically on EINTR.
func Fallocate ¶
Fallocate pre-allocates disk space for a file to reduce fragmentation and improve sequential write performance. Retries automatically on EINTR.
func Fdatasync ¶
Fdatasync syncs file data to storage without syncing metadata. Prefers fdatasync(2) over fsync(2) for better performance on Linux. Retries automatically on EINTR (Go runtime SIGURG preemption).
func Ftruncate ¶
Ftruncate sets the logical size of f to size bytes, releasing any extents beyond that point back to the filesystem without moving the file position.
The canonical use case is snapping an O_DIRECT file back to its true size after the tail-padding trick: the last chunk is written block-aligned (padded with zeros to the next 4 KiB boundary), and Ftruncate then sets the logical EOF to the actual byte count, discarding the padding.
Prefer this over syscall.Ftruncate(int(f.Fd()), size) directly. When the caller has extracted a raw fd via f.Fd() for use with iosched.WriteAt, the GC may finalise f — closing the file descriptor — before the call site if nothing explicitly references f after f.Fd(). os.File.Truncate is a method call on f, so the compiler guarantees f is live until Truncate returns; no runtime.KeepAlive bookkeeping is required at the call site.
func IsAligned ¶
IsAligned reports whether the first byte of block is aligned to align.BlockSize. Delegates to align.IsAligned; provided here so callers need only import sys.
func IsTransientIOError ¶
IsTransientIOError reports whether err represents a temporary condition that may succeed on retry.
func OpenDirect ¶
OpenDirect opens an existing file for reading with optional O_DIRECT. Falls back to buffered I/O if the filesystem rejects O_DIRECT.
func PreadAligned ¶
PreadAligned reads len(buf) bytes from f at offset into buf. When FlDirectIO is set, validates that buf, offset, and len(buf) all satisfy the align.BlockSize alignment requirements of O_DIRECT.
func PunchHole ¶
PunchHole deallocates a byte range within a file, creating a sparse file. Uses FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE to reclaim disk space without changing the logical file size.
The range is aligned inward to filesystem block boundaries so that adjacent data is never punched. Returns the number of bytes reclaimed after alignment, or (0, nil) if the range is smaller than one block. Retries automatically on EINTR.
Types ¶
type FadviseHint ¶
type FadviseHint int
FadviseHint describes the expected access pattern for posix_fadvise.
const ( FadvSequential FadviseHint = iota FadvDontNeed FadvRandom )
type OpenFlag ¶
type OpenFlag int
OpenFlag controls file opening and sync behavior.
const ( // FlDirectIO bypasses the page cache (O_DIRECT on Linux, F_NOCACHE on Darwin). FlDirectIO OpenFlag = 1 << iota // FlDSync syncs data before each write returns (O_DSYNC on Linux). // On Darwin, explicit Fdatasync is needed after writes. FlDSync // FlSync syncs data and metadata before each write returns (O_SYNC on Linux). // On Darwin, explicit f.Sync is needed after writes. FlSync )
Source Files