segment

package
v0.0.0-...-17cad6e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	MaxDocumentPayloadSize = 64 << 20
)

Variables

View Source 
var (
	ErrSegmentCorrupt   = errors.New("db: corrupt segment")
	ErrSegmentSealed    = errors.New("db: write segment is sealed")
	ErrSegmentFull      = errors.New("db: write segment is full")
	ErrSegmentNotFound  = errors.New("db: segment not found")
	ErrDocumentNotFound = errors.New("db: document not found")
)

Functions

This section is empty.

Types

type FetchResult 

type FetchResult struct {
	PrimaryKey string
	Document   *StoredDocument
	Err        error
}

FetchResult preserves request order. A nil Document with nil Err means the primary key is absent or logically deleted, matching the pinned baseline.

type ImmutableSegment 

type ImmutableSegment struct {
	// contains filtered or unexported fields
}

ImmutableSegment is a verified read-only segment snapshot.

func OpenImmutableSegment 

func OpenImmutableSegment(ctx context.Context, collectionDir string, metadata common.SegmentMetadata) (*ImmutableSegment, error)

OpenImmutableSegment loads and verifies the first data file in metadata.

func (*ImmutableSegment) Document 

func (s *ImmutableSegment) Document(docID uint64) (StoredDocument, bool)

Document returns an independent document copy.

func (*ImmutableSegment) Documents 

func (s *ImmutableSegment) Documents() []StoredDocument

Documents returns all immutable documents in ascending ID order.

func (*ImmutableSegment) ID 

func (s *ImmutableSegment) ID() uint64

ID returns the immutable segment ID.

func (*ImmutableSegment) MemoryUsageBytes 

func (s *ImmutableSegment) MemoryUsageBytes() uint64

MemoryUsageBytes returns the encoded record bytes retained by the segment.

func (*ImmutableSegment) Metadata 

func (s *ImmutableSegment) Metadata() common.SegmentMetadata

Metadata returns a deep copy.

type SegmentManager 

type SegmentManager struct {
	// contains filtered or unexported fields
}

SegmentManager owns sorted immutable segments, one optional write segment, the primary-key map, and the logical deletion snapshot.

func NewSegmentManager 

func NewSegmentManager(primaryKey *common.PrimaryKeyMap, deletes *common.DeleteStore) *SegmentManager

NewSegmentManager constructs an empty manager. Nil stores are replaced with empty instances.

func (*SegmentManager) AddImmutable 

func (m *SegmentManager) AddImmutable(segment *ImmutableSegment) error

AddImmutable adds a verified immutable segment.

func (*SegmentManager) ClearWriting 

func (m *SegmentManager) ClearWriting() *WriteSegment

ClearWriting removes and returns the current write segment.

func (*SegmentManager) Deletes 

func (m *SegmentManager) Deletes() *common.DeleteStore

Deletes returns the manager's shared logical deletion set.

func (*SegmentManager) Document 

func (m *SegmentManager) Document(docID uint64) (StoredDocument, bool)

Document returns a live document by global ID.

func (*SegmentManager) DocumentByPrimaryKey 

func (m *SegmentManager) DocumentByPrimaryKey(key string) (StoredDocument, bool, error)

DocumentByPrimaryKey resolves the IDMap and verifies the target document's primary key so a corrupt mapping cannot return another document.

func (*SegmentManager) Fetch 

func (m *SegmentManager) Fetch(ctx context.Context, primaryKeys []string) ([]FetchResult, error)

Fetch resolves primary keys in input order. Missing keys are successful nil results; context cancellation is returned at batch level and attached to all unprocessed entries.

func (*SegmentManager) ImmutableMetadata 

func (m *SegmentManager) ImmutableMetadata() []common.SegmentMetadata

ImmutableMetadata returns independent metadata sorted like segments.

func (*SegmentManager) ImmutableSegments 

func (m *SegmentManager) ImmutableSegments() []*ImmutableSegment

ImmutableSegments returns segments sorted by minimum document ID and ID.

func (*SegmentManager) LiveDocumentPrimaryKeys 

func (m *SegmentManager) LiveDocumentPrimaryKeys(ctx context.Context) (map[uint64]string, error)

LiveDocumentPrimaryKeys returns the authoritative primary key for every live document ID without cloning stored document payloads.

func (*SegmentManager) LiveDocuments 

func (m *SegmentManager) LiveDocuments(ctx context.Context) ([]StoredDocument, error)

LiveDocuments returns independent copies of every current document in ascending global document-ID order. Superseded and deleted versions are omitted. The caller is responsible for excluding concurrent multi-step writes when it needs a transactionally stable query snapshot.

func (*SegmentManager) PrimaryKeys 

func (m *SegmentManager) PrimaryKeys() *common.PrimaryKeyMap

PrimaryKeys returns the manager's shared primary-key map.

func (*SegmentManager) RemoveImmutable 

func (m *SegmentManager) RemoveImmutable(segmentID uint64) (*ImmutableSegment, error)

RemoveImmutable removes and returns a segment without deleting its files.

func (*SegmentManager) ReplacePrimaryKeys 

func (m *SegmentManager) ReplacePrimaryKeys(primaryKey *common.PrimaryKeyMap) error

ReplacePrimaryKeys swaps the logical IDMap while the collection-level lock excludes readers and writers. It is used only after CURRENT commits a new immutable checkpoint.

func (*SegmentManager) RotateWriting 

func (m *SegmentManager) RotateWriting(currentID uint64, immutable *ImmutableSegment, next *WriteSegment) error

RotateWriting atomically replaces the current write segment with its immutable snapshot and installs the next empty write segment. The caller must have already durably published matching manifest metadata.

func (*SegmentManager) SetWriting 

func (m *SegmentManager) SetWriting(segment *WriteSegment) error

SetWriting installs a write segment after checking ID and document-range conflicts.

func (*SegmentManager) StorageStats 

func (m *SegmentManager) StorageStats() StorageStats

StorageStats returns a stable snapshot of retained segment resources.

func (*SegmentManager) Writing 

func (m *SegmentManager) Writing() *WriteSegment

Writing returns the current write segment.

type StorageStats 

type StorageStats struct {
	ImmutableSegmentCount uint64
	MutableDocumentCount  uint64
	DeletedDocumentCount  uint64
	MemoryUsageBytes      uint64
}

StorageStats describes retained segment data without allocating document copies. MemoryUsageBytes counts encoded record headers, keys, payloads, and logical-deletion IDs.

type StoredDocument 

type StoredDocument struct {
	DocID      uint64
	PrimaryKey string
	Payload    []byte
}

StoredDocument is the schema-independent representation held by segments. Payload is produced and interpreted by the collection's schema codec.

func CloneDocuments 

func CloneDocuments(docs []StoredDocument) []StoredDocument

func (StoredDocument) Clone 

func (d StoredDocument) Clone() StoredDocument

Clone returns a deep copy of d.

type WriteSegment 

type WriteSegment struct {
	// contains filtered or unexported fields
}

WriteSegment accepts sequential documents until it is sealed.

func NewWriteSegment 

func NewWriteSegment(id, minDocID, maxDocs uint64) (*WriteSegment, error)

NewWriteSegment constructs an empty segment with a fixed global ID range start and capacity.

func (*WriteSegment) Append 

func (s *WriteSegment) Append(ctx context.Context, primaryKey string, payload []byte) (StoredDocument, error)

Append stores a cloned payload and assigns the next contiguous document ID.

func (*WriteSegment) AppendExpected 

func (s *WriteSegment) AppendExpected(ctx context.Context, expectedDocID uint64, primaryKey string, payload []byte) (StoredDocument, error)

AppendExpected appends only if expectedDocID is still next. It lets the WAL record and in-memory application agree on the assigned global ID.

func (*WriteSegment) Document 

func (s *WriteSegment) Document(docID uint64) (StoredDocument, bool)

Document returns an independent document copy.

func (*WriteSegment) Documents 

func (s *WriteSegment) Documents() []StoredDocument

Documents returns all documents in ascending ID order.

func (*WriteSegment) ID 

func (s *WriteSegment) ID() uint64

ID returns the segment ID.

func (*WriteSegment) MemoryUsageBytes 

func (s *WriteSegment) MemoryUsageBytes() uint64

MemoryUsageBytes returns the encoded record bytes retained by the segment.

func (*WriteSegment) Metadata 

func (s *WriteSegment) Metadata() common.SegmentMetadata

Metadata returns the current in-memory range without file references.

func (*WriteSegment) NextDocumentID 

func (s *WriteSegment) NextDocumentID() (uint64, error)

NextDocumentID returns the ID that the next append will receive.

func (*WriteSegment) ReservedRange 

func (s *WriteSegment) ReservedRange() (uint64, uint64)

ReservedRange returns the inclusive document-ID range owned by the segment.

func (*WriteSegment) Seal 

func (s *WriteSegment) Seal(ctx context.Context, collectionDir, relativeName string) (*ImmutableSegment, error)

Seal writes one immutable segment file relative to collectionDir and makes this write segment reject further appends.

func (*WriteSegment) Snapshot 

func (s *WriteSegment) Snapshot(ctx context.Context, collectionDir, relativeName string) (*ImmutableSegment, error)

Snapshot writes the current non-empty contents as an immutable segment without sealing the write segment. Collection flush uses this before the manifest commit point so a failed publication can safely keep accepting WAL backed writes and retry with fresh immutable artifacts.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.