container

package
v1.13.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 28, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ContainerMagic  = "ColdKeep" // must be exactly 8 bytes
	ContainerHdrLen = 64         // fixed header size in bytes
)
View Source 
const (
	ContainerFormatVersionMajor uint16 = 1
	ContainerFormatVersionMinor uint16 = 0
)

Container format versioning is intentionally decoupled from the app version. This keeps on-disk compatibility stable while CLI releases evolve.

View Source 
const (
	ContainerCodecUnknown uint16 = 0
	ContainerCodecPlain   uint16 = 1
	ContainerCodecAESGCM  uint16 = 2
)

Optional container-level codec hint for future migrations. Current write path keeps this as unknown because blocks already persist codec per payload, and mixed-codec containers are allowed.

View Source 
const (
	FLAG_BLOCK_LAYOUT  = 1 << 0 // container uses block-based layout
	FLAG_HAS_BLOCK_TBL = 1 << 1 // optional block index present
)

Reserved future flags (not used yet)

View Source 
const LegacyContainerFormatVersionMajor uint16 = 0

Supported legacy format major from pre-v1 headers that used app semantic version fields in bytes 8..11.

Variables

View Source 
var ContainersDir = utils_env.GetenvOrDefault("COLDKEEP_STORAGE_DIR", "./storage/containers")
View Source 
var ErrContainerFull = errors.New("container full")

ErrContainerFull is returned by Container.Append when the payload would exceed the container's max size.

View Source 
var ErrContainerLockContention = errors.New("container row lock contention")

Functions

func CheckContainerHashFile added in v0.4.0 

func CheckContainerHashFile(id int, filename, storedHash string) error

func CheckContainerHashFileInDir added in v0.8.0 

func CheckContainerHashFileInDir(id int, filename, storedHash string, containersDir string) error

func GetContainerMaxSize 

func GetContainerMaxSize() int64

GetContainerMaxSize returns the current container max size

func QuarantineContainer added in v0.10.0 

func QuarantineContainer(dbconn *sql.DB, containerID int64) error

func QuarantineContainerInDir added in v1.4.1 

func QuarantineContainerInDir(dbconn *sql.DB, containerID int64, containersDir string) error

func ReadPayloadAt added in v0.7.0 

func ReadPayloadAt(c Container, offset int64, size int64) ([]byte, error)

func SafeContainerPath added in v1.10.2 

func SafeContainerPath(containersDir, filename string) (string, error)

SafeContainerPath validates a container filename before joining it under a container root.

func SealContainer 

func SealContainer(tx db.DBTX, containerID int64, filename string) error

func SealContainerInDir added in v0.8.0 

func SealContainerInDir(tx db.DBTX, containerID int64, filename string, containersDir string) error

func SetContainerMaxSize 

func SetContainerMaxSize(size int64)

SetContainerMaxSize sets the container max size (for testing)

func UpdateContainerSize 

func UpdateContainerSize(tx db.DBTX, containerID int64, newSize int64) error

Types

type ActiveContainer added in v0.6.0 

type ActiveContainer struct {
	ID        int64
	Filename  string
	Container Container
	MaxSize   int64
}

func GetOrCreateOpenContainerInDirExcluding added in v0.8.0 

func GetOrCreateOpenContainerInDirExcluding(db db.DBTX, containersDir string, excludeID int64) (ActiveContainer, error)

type BrokenOpenContainerError added in v0.10.0 

type BrokenOpenContainerError struct {
	ContainerID int64
	Err         error
}

func (*BrokenOpenContainerError) Error added in v0.10.0 

func (e *BrokenOpenContainerError) Error() string

func (*BrokenOpenContainerError) Unwrap added in v0.10.0 

func (e *BrokenOpenContainerError) Unwrap() error

type Container added in v0.6.0 

type Container interface {
	Append(data []byte) (offset int64, err error)
	ReadAt(offset int64, size int64) ([]byte, error)
	Size() int64
	Truncate(size int64) error
	Sync() error
	Close() error
}

type ContainerWriter added in v0.8.0 

type ContainerWriter interface {
	FinalizeContainer() error
}

ContainerWriter is the storage-context ownership boundary: callers can always finalize and release writer-owned resources, while append-oriented behavior is discovered through optional interfaces in internal/storage.

type FileContainer added in v0.6.0 

type FileContainer struct {
	// contains filtered or unexported fields
}

func OpenReadOnlyContainer added in v0.8.0 

func OpenReadOnlyContainer(path string, maxSize int64) (*FileContainer, error)

OpenReadOnlyContainer opens an existing container in read-only mode.

This wrapper avoids ambiguous boolean call sites like openExistingContainer(true, ...) and makes intent explicit.

func OpenWritableContainer added in v0.8.0 

func OpenWritableContainer(path string, maxSize int64) (*FileContainer, error)

OpenWritableContainer opens an existing container in writable mode.

This wrapper avoids ambiguous boolean call sites like openExistingContainer(false, ...) and makes intent explicit.

func (*FileContainer) Append added in v0.6.0 

func (c *FileContainer) Append(data []byte) (int64, error)

func (*FileContainer) Close added in v0.6.0 

func (c *FileContainer) Close() error

func (*FileContainer) ReadAt added in v0.6.0 

func (c *FileContainer) ReadAt(offset int64, size int64) ([]byte, error)

func (*FileContainer) SetSize added in v1.7.0 

func (c *FileContainer) SetSize(size int64)

func (*FileContainer) Size added in v0.8.0 

func (c *FileContainer) Size() int64

func (*FileContainer) Sync added in v0.6.0 

func (c *FileContainer) Sync() error

func (*FileContainer) Truncate added in v0.10.0 

func (c *FileContainer) Truncate(size int64) error
type Header struct {
	FormatMajor uint16
	FormatMinor uint16
	HeaderLen   uint32
	Flags       uint32
	CreatedAt   int64
	MaxSize     int64
	CodecID     uint16
}

type LocalPlacement added in v0.8.0 

type LocalPlacement struct {
	ContainerID      int64
	Filename         string
	Offset           int64
	StoredSize       int64
	NewContainerSize int64
	Rotated          bool
	PreviousID       int64
	PreviousFilename string
	PreviousSize     int64
	Full             bool
}

LocalPlacement describes where a payload was physically appended.

type LocalWriter added in v0.8.0 

type LocalWriter struct {
	// contains filtered or unexported fields
}

func NewLocalWriter added in v0.8.0 

func NewLocalWriter(maxSize int64) *LocalWriter

func NewLocalWriterWithDir added in v0.8.0 

func NewLocalWriterWithDir(dir string, maxSize int64) *LocalWriter

func NewLocalWriterWithDirAndDB added in v0.10.0 

func NewLocalWriterWithDirAndDB(dir string, maxSize int64, dbconn *sql.DB) *LocalWriter

NewLocalWriterWithDirAndDB creates a LocalWriter that commits a durable sealing marker (via dbconn) before physical finalization of each container. Passing a non-nil dbconn is required for the full sealing-safety guarantee; passing nil is still correct but skips the pre-finalization DB marker.

func (*LocalWriter) AcknowledgeAppendCommitted added in v0.10.0 

func (w *LocalWriter) AcknowledgeAppendCommitted()

AcknowledgeAppendCommitted clears the rollback bookkeeping after the enclosing DB transaction has successfully committed. This completes the commit acknowledgment path of the state machine: pendingAppend and the pre-write size/file fields are zeroed so a future rollback call cannot accidentally truncate already-committed bytes. Must be called exactly once after each successful commit that followed an AppendPayload success. Safe to call when no append is pending (no-op).

func (*LocalWriter) ActiveContainerState added in v0.8.0 

func (w *LocalWriter) ActiveContainerState() (ActiveContainer, int64, bool)

ActiveContainerState returns the currently opened local container state, if any.

func (*LocalWriter) AppendPayload added in v0.8.0 

func (w *LocalWriter) AppendPayload(tx db.DBTX, payload []byte) (LocalPlacement, error)

AppendPayload appends already-encoded payload bytes to the active local container. DB lifecycle decisions (size update/seal/chunk linking) remain outside this writer. If there is no active container (including after FinalizeContainer), this method lazily opens one. Canonical lifecycle contract: internal/storage/store.go (Append lifecycle state machine).

func (*LocalWriter) BindDB added in v0.10.0 

func (w *LocalWriter) BindDB(dbconn *sql.DB)

func (*LocalWriter) DB added in v0.10.0 

func (w *LocalWriter) DB() *sql.DB

DB returns the underlying *sql.DB held by this writer (may be nil). Used when cloning a writer per worker to propagate the DB connection.

func (*LocalWriter) Dir added in v0.8.0 

func (w *LocalWriter) Dir() string

func (*LocalWriter) FinalizeContainer added in v0.8.0 

func (w *LocalWriter) FinalizeContainer() error

FinalizeContainer performs physical sync/close for the active container and clears local active state. If physical finalization fails, the quarantine path is executed before returning so no future writes can reuse a potentially unsafe file.

func (*LocalWriter) MaxSize added in v0.8.0 

func (w *LocalWriter) MaxSize() int64

func (*LocalWriter) QuarantineActiveContainer added in v0.10.0 

func (w *LocalWriter) QuarantineActiveContainer() error

QuarantineActiveContainer is the quarantine path used after failed cleanup boundaries (for example rollback/finalize/sync cleanup failure). It closes current handles, clears pending append state, and marks the DB row as quarantined to prevent future reuse.

func (*LocalWriter) RollbackLastAppend added in v0.10.0 

func (w *LocalWriter) RollbackLastAppend() error

RollbackLastAppend truncates the active container file back to its pre-append offset on the rollback path when the enclosing DB transaction was rolled back or failed to commit. Safe to call even if no unresolved append is pending (no-op). After a successful rollback path cleanup, the writer's active state is reset so the next AppendPayload selects a fresh open container from the database. If rollback path cleanup itself fails, caller must trigger quarantine for the active container before any further writes.

type SimulatedWriter added in v0.8.0 

type SimulatedWriter struct {
	// contains filtered or unexported fields
}

func NewSimulatedWriter added in v0.8.0 

func NewSimulatedWriter(maxSize int64) *SimulatedWriter

func (*SimulatedWriter) AcknowledgeAppendCommitted added in v0.10.0 

func (w *SimulatedWriter) AcknowledgeAppendCommitted()

AcknowledgeAppendCommitted is a no-op for SimulatedWriter: there is no physical rollback state to clear on the commit acknowledgment path. Canonical lifecycle contract: internal/storage/store.go (Append lifecycle state machine).

func (*SimulatedWriter) AppendPayload added in v0.8.0 

func (w *SimulatedWriter) AppendPayload(tx db.DBTX, payload []byte) (LocalPlacement, error)

func (*SimulatedWriter) FinalizeContainer added in v0.8.0 

func (w *SimulatedWriter) FinalizeContainer() error

func (*SimulatedWriter) MaxSize added in v0.8.0 

func (w *SimulatedWriter) MaxSize() int64

func (*SimulatedWriter) QuarantineActiveContainer added in v0.10.0 

func (w *SimulatedWriter) QuarantineActiveContainer() error

func (*SimulatedWriter) RollbackLastAppend added in v0.10.0 

func (w *SimulatedWriter) RollbackLastAppend() error

RollbackLastAppend is a no-op for SimulatedWriter: no physical bytes are written, so rollback path cleanup has nothing to truncate. Canonical lifecycle contract: internal/storage/store.go (Append lifecycle state machine).

func (*SimulatedWriter) SealContainer added in v0.8.0 

func (w *SimulatedWriter) SealContainer(tx db.DBTX, containerID int64, _ string, _ string) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.