backup

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 22, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package backup provides backup and restore functionality for ObaDB.

Package backup provides backup and restore functionality for ObaDB.

Package backup provides backup and restore functionality for ObaDB.

Overview

The backup package implements database backup and restore operations for the ObaDB storage engine. It supports:

  • Full database backups with consistent snapshots
  • Incremental backups (changes since last backup)
  • Compression for reduced storage
  • LDIF export/import for interoperability
  • Checksum verification for data integrity

Backup Formats

Two backup formats are supported:

  • Native: Binary format optimized for fast backup/restore
  • LDIF: Text format for interoperability with other LDAP servers

Creating Backups

Create a full backup: 

opts := &backup.BackupOptions{
    OutputPath: "/backup/oba-20260218.bak",
    Compress:   true,
    Format:     backup.FormatNative,
}

stats, err := backup.Full(engine, opts)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Backed up %d entries in %v\n", stats.EntryCount, stats.Duration)

LDIF Export

Export to LDIF format: 

opts := &backup.BackupOptions{
    OutputPath: "/backup/data.ldif",
    Format:     backup.FormatLDIF,
    BaseDN:     "dc=example,dc=com",
}

stats, err := backup.Full(engine, opts)

Restoring Backups

Restore from a backup: 

opts := &backup.RestoreOptions{
    InputPath: "/backup/oba-20260218.bak",
    Verify:    true, // Verify checksums before restore
    Format:    backup.FormatNative,
}

err := backup.Restore(engine, opts)
if err != nil {
    log.Fatal(err)
}

LDIF Import

Import from LDIF: 

opts := &backup.RestoreOptions{
    InputPath: "/backup/data.ldif",
    Format:    backup.FormatLDIF,
}

err := backup.Restore(engine, opts)

Backup Header

Native backups include a header with metadata: 

header := backup.NewBackupHeader()
header.PageSize = 4096
header.TotalPages = 1000
header.EntryCount = 5000
header.SetCompressed(true)

Backup Statistics

BackupStats provides information about the backup: 

stats := &backup.BackupStats{
    TotalPages:      1000,
    TotalBytes:      4096000,
    CompressedBytes: 1024000,
    Duration:        5 * time.Second,
    EntryCount:      5000,
}

ratio := stats.CompressionRatio() // 0.75 (75% reduction)

Error Handling

Common backup errors:

  • ErrInvalidBackup: Backup file is malformed
  • ErrInvalidMagic: Not a valid ObaDB backup
  • ErrChecksumMismatch: Data corruption detected
  • ErrUnsupportedFormat: Unknown backup version

Package backup provides backup and restore functionality for ObaDB.

Package backup provides backup and restore functionality for ObaDB.

Package backup provides LDIF export and import functionality for ObaDB.

Package backup provides backup and restore functionality for ObaDB.

Index

Constants

View Source 
const (
	// BackupMagic is the magic number for ObaDB backup files.
	BackupMagicByte0 = 'O'
	BackupMagicByte1 = 'B'
	BackupMagicByte2 = 'A'
	BackupMagicByte3 = 'B'

	// BackupVersion is the current backup format version.
	BackupVersion uint32 = 1

	// BackupHeaderSize is the size of the backup header in bytes.
	BackupHeaderSize = 64
)

Backup format constants.

View Source 
const (
	// BackupFlagCompressed indicates the backup is compressed.
	BackupFlagCompressed uint32 = 1 << iota
	// BackupFlagIncremental indicates the backup is incremental.
	BackupFlagIncremental
	// BackupFlagMultiFile indicates the backup contains multiple files.
	BackupFlagMultiFile
)

Backup flags.

View Source 
const (
	// MinMatchLength is the minimum length for a match.
	MinMatchLength = 4

	// MaxMatchLength is the maximum length for a match.
	MaxMatchLength = 255 + MinMatchLength

	// MaxOffset is the maximum offset for a match (16-bit).
	MaxOffset = 65535

	// HashTableSize is the size of the hash table for compression.
	HashTableSize = 1 << 14 // 16384 entries

	// CompressBlockSize is the size of each compression block.
	CompressBlockSize = 64 * 1024 // 64KB blocks

	// LiteralRunMask is the mask for literal run length in token.
	LiteralRunMask = 0xF0

	// MatchLengthMask is the mask for match length in token.
	MatchLengthMask = 0x0F
)

Compression constants for LZ4-style compression.

View Source 
const (
	// IncrementalMagicByte0-3 form the magic number for incremental backups.
	IncrementalMagicByte0 = 'O'
	IncrementalMagicByte1 = 'B'
	IncrementalMagicByte2 = 'A'
	IncrementalMagicByte3 = 'I'

	// IncrementalHeaderSize is the size of the incremental backup header in bytes.
	IncrementalHeaderSize = 80

	// MetadataFileName is the name of the backup metadata file.
	MetadataFileName = "backup_metadata.oba"
)

Incremental backup constants.

Variables

View Source 
var (
	ErrNilEngine         = errors.New("storage engine is nil")
	ErrNilPageManager    = errors.New("page manager is nil")
	ErrBackupFailed      = errors.New("backup failed")
	ErrRestoreFailed     = errors.New("restore failed")
	ErrInvalidBackup     = errors.New("invalid backup file")
	ErrInvalidMagic      = errors.New("invalid backup magic number")
	ErrUnsupportedFormat = errors.New("unsupported backup format")
	ErrChecksumMismatch  = errors.New("backup checksum mismatch")
	ErrOutputPathEmpty   = errors.New("output path is empty")
	ErrInputPathEmpty    = errors.New("input path is empty")
	ErrBackupCorrupted   = errors.New("backup file is corrupted")
	ErrImportFailed      = errors.New("import failed")
	ErrExportFailed      = errors.New("export failed")
)

Backup errors.

View Source 
var (
	ErrNoBaseBackup            = errors.New("no base backup found, run full backup first")
	ErrInvalidIncrementalMagic = errors.New("invalid incremental backup magic number")
	ErrIncrementalCorrupted    = errors.New("incremental backup is corrupted")
	ErrWALNotAvailable         = errors.New("WAL is not available")
	ErrMetadataNotFound        = errors.New("backup metadata not found")
	ErrInvalidMetadata         = errors.New("invalid backup metadata")
)

Incremental backup errors.

View Source 
var (
	ErrInvalidLDIF     = errors.New("invalid LDIF format")
	ErrMissingDN       = errors.New("missing DN in LDIF entry")
	ErrInvalidBase64   = errors.New("invalid base64 encoding")
	ErrEmptyReader     = errors.New("empty reader")
	ErrTransactionFail = errors.New("transaction failed")
)

LDIF errors.

View Source 
var (
	ErrDataDirEmpty       = errors.New("data directory is empty")
	ErrDataDirNotExist    = errors.New("data directory does not exist")
	ErrUnknownBackupType  = errors.New("unknown backup format")
	ErrRestoreInProgress  = errors.New("restore already in progress")
	ErrInvalidBackupChain = errors.New("invalid backup chain: LSN mismatch")
	ErrNoBackupsToRestore = errors.New("no backups to restore")
)

Restore errors.

View Source 
var BackupMagic = [4]byte{BackupMagicByte0, BackupMagicByte1, BackupMagicByte2, BackupMagicByte3}

BackupMagic is the magic number for ObaDB backup files.

View Source 
var IncrementalMagic = [4]byte{IncrementalMagicByte0, IncrementalMagicByte1, IncrementalMagicByte2, IncrementalMagicByte3}

IncrementalMagic is the magic number for incremental backup files.

View Source 
var StorageFiles = []string{"data.oba", "index.oba", "wal.oba"}

Storage file names.

Functions

func ParseLDIF 

func ParseLDIF(r io.Reader) ([]*storage.Entry, error)

ParseLDIF is a convenience function to parse LDIF content without an engine.

func WriteLDIF 

func WriteLDIF(w io.Writer, entries []*storage.Entry) error

WriteLDIF is a convenience function to write entries to LDIF format.

Types

type BackupChainInfo 

type BackupChainInfo struct {
	// FullBackup is the path to the full backup.
	FullBackup string

	// IncrementalBackups is the list of incremental backups in order.
	IncrementalBackups []string

	// TotalBackups is the total number of backups in the chain.
	TotalBackups int

	// StartLSN is the LSN of the full backup.
	StartLSN uint64

	// EndLSN is the LSN of the last incremental backup.
	EndLSN uint64

	// IsComplete indicates if the chain is complete (no gaps).
	IsComplete bool
}

BackupChainInfo contains information about a backup chain.

type BackupFormat 

type BackupFormat string

BackupFormat represents the backup file format. 

const (
	// FormatNative is the native binary backup format.
	FormatNative BackupFormat = "native"
	// FormatLDIF is the LDIF text backup format.
	FormatLDIF BackupFormat = "ldif"
)

type BackupHeader 

type BackupHeader struct {
	Magic      [4]byte
	Version    uint32
	Timestamp  int64
	Flags      uint32
	PageSize   uint32
	TotalPages uint64
	EntryCount uint64
	Checksum   uint32
	Reserved   [20]byte
}

BackupHeader represents the header of a native backup file. Layout (64 bytes):

  • Bytes 0-3: Magic number ("OBAB")
  • Bytes 4-7: Version (uint32)
  • Bytes 8-15: Timestamp (int64, Unix timestamp)
  • Bytes 16-19: Flags (uint32)
  • Bytes 20-23: PageSize (uint32)
  • Bytes 24-31: TotalPages (uint64)
  • Bytes 32-39: EntryCount (uint64)
  • Bytes 40-43: Checksum (uint32, CRC32 of all page data)
  • Bytes 44-63: Reserved

func NewBackupHeader 

func NewBackupHeader() *BackupHeader

NewBackupHeader creates a new backup header with default values.

func (*BackupHeader) Deserialize 

func (h *BackupHeader) Deserialize(buf []byte) error

Deserialize reads the backup header from a byte slice.

func (*BackupHeader) IsCompressed 

func (h *BackupHeader) IsCompressed() bool

IsCompressed returns true if the backup is compressed.

func (*BackupHeader) IsIncremental 

func (h *BackupHeader) IsIncremental() bool

IsIncremental returns true if the backup is incremental.

func (*BackupHeader) IsMultiFile 

func (h *BackupHeader) IsMultiFile() bool

IsMultiFile returns true if the backup contains multiple files.

func (*BackupHeader) Serialize 

func (h *BackupHeader) Serialize() ([]byte, error)

Serialize writes the backup header to a byte slice.

func (*BackupHeader) SerializeTo 

func (h *BackupHeader) SerializeTo(buf []byte) error

SerializeTo writes the backup header to an existing byte slice.

func (*BackupHeader) SetCompressed 

func (h *BackupHeader) SetCompressed(compressed bool)

SetCompressed sets the compressed flag.

func (*BackupHeader) SetIncremental 

func (h *BackupHeader) SetIncremental(incremental bool)

SetIncremental sets the incremental flag.

func (*BackupHeader) SetMultiFile 

func (h *BackupHeader) SetMultiFile(multiFile bool)

SetMultiFile sets the multi-file flag.

func (*BackupHeader) Validate 

func (h *BackupHeader) Validate() error

Validate validates the backup header.

type BackupManager 

type BackupManager struct {
	// contains filtered or unexported fields
}

BackupManager manages backup and restore operations for ObaDB.

func NewBackupManager 

func NewBackupManager(pageManager *storage.PageManager) *BackupManager

NewBackupManager creates a new BackupManager with the given page manager.

func NewBackupManagerWithEngine 

func NewBackupManagerWithEngine(pageManager *storage.PageManager, engine storage.StorageEngine) *BackupManager

NewBackupManagerWithEngine creates a new BackupManager with a storage engine. This is useful for LDIF exports that need access to entries.

func (*BackupManager) Backup 

func (bm *BackupManager) Backup(opts *BackupOptions) (*BackupStats, error)

Backup performs a backup operation based on the provided options.

func (*BackupManager) GetBackupInfo 

func (bm *BackupManager) GetBackupInfo(path string) (*BackupHeader, error)

GetBackupInfo returns information about a backup file.

func (*BackupManager) Restore 

func (bm *BackupManager) Restore(opts *RestoreOptions) (*BackupStats, error)

Restore restores a database from a backup file.

func (*BackupManager) VerifyBackup 

func (bm *BackupManager) VerifyBackup(path string) error

VerifyBackup verifies the integrity of a backup file.

type BackupMetadata 

type BackupMetadata struct {
	LastBackupLSN  uint64
	LastBackupTime int64
	BackupType     string // "full" or "incremental"
	BackupPath     string
}

BackupMetadata stores information about the last backup for incremental backups.

type BackupOptions 

type BackupOptions struct {
	// OutputPath is the path to the backup file.
	OutputPath string

	// DataDir is the data directory containing all storage files.
	// When set, all storage files (data.oba, index.oba, wal.oba) are backed up.
	DataDir string

	// Compress enables compression for the backup.
	Compress bool

	// Incremental enables incremental backup (only changes since last backup).
	Incremental bool

	// Format specifies the backup format ("native" or "ldif").
	Format BackupFormat

	// BaseDN is the base DN for LDIF export (optional, defaults to root).
	BaseDN string
}

BackupOptions configures the backup operation.

func (*BackupOptions) Validate 

func (o *BackupOptions) Validate() error

Validate validates the backup options.

type BackupStats 

type BackupStats struct {
	// TotalPages is the total number of pages backed up.
	TotalPages uint64

	// TotalBytes is the total size of the backup in bytes.
	TotalBytes int64

	// CompressedBytes is the compressed size (if compression enabled).
	CompressedBytes int64

	// Duration is the time taken to complete the backup.
	Duration time.Duration

	// EntryCount is the number of entries backed up.
	EntryCount uint64
}

BackupStats contains statistics about a backup operation.

func (*BackupStats) CompressionRatio 

func (s *BackupStats) CompressionRatio() float64

CompressionRatio returns the compression ratio (0-1). Returns 0 if compression is not enabled or no data was written.

type CompressWriter 

type CompressWriter struct {
	// contains filtered or unexported fields
}

CompressWriter wraps an io.Writer and compresses data using LZ4-style compression.

func NewCompressWriter 

func NewCompressWriter(w io.Writer) *CompressWriter

NewCompressWriter creates a new compression writer.

func (*CompressWriter) Close 

func (cw *CompressWriter) Close() error

Close flushes any remaining data and closes the writer.

func (*CompressWriter) TotalInput 

func (cw *CompressWriter) TotalInput() int64

TotalInput returns the total uncompressed bytes received.

func (*CompressWriter) Write 

func (cw *CompressWriter) Write(p []byte) (n int, err error)

Write writes data to the compression buffer. Data is compressed and written when the buffer is full.

func (*CompressWriter) Written 

func (cw *CompressWriter) Written() int64

Written returns the total compressed bytes written.

type DecompressReader 

type DecompressReader struct {
	// contains filtered or unexported fields
}

DecompressReader wraps an io.Reader and decompresses LZ4-style compressed data.

func NewDecompressReader 

func NewDecompressReader(r io.Reader) *DecompressReader

NewDecompressReader creates a new decompression reader.

func (*DecompressReader) Read 

func (dr *DecompressReader) Read(p []byte) (n int, err error)

Read reads decompressed data.

func (*DecompressReader) TotalRead 

func (dr *DecompressReader) TotalRead() int64

TotalRead returns the total decompressed bytes read.

type FileEntry 

type FileEntry struct {
	Name string
	Size int64
	Data []byte
}

FileEntry represents a file entry in a multi-file backup.

type IncrementalBackupManager 

type IncrementalBackupManager struct {
	*BackupManager
	// contains filtered or unexported fields
}

IncrementalBackupManager extends BackupManager with incremental backup support.

func NewIncrementalBackupManager 

func NewIncrementalBackupManager(pageManager *storage.PageManager, wal *storage.WAL, metadataDir string) *IncrementalBackupManager

NewIncrementalBackupManager creates a new IncrementalBackupManager.

func (*IncrementalBackupManager) GetIncrementalBackupInfo 

func (ibm *IncrementalBackupManager) GetIncrementalBackupInfo(path string) (*IncrementalHeader, error)

GetIncrementalBackupInfo returns information about an incremental backup file.

func (*IncrementalBackupManager) GetLastBackupInfo 

func (ibm *IncrementalBackupManager) GetLastBackupInfo() (*BackupMetadata, error)

GetLastBackupInfo returns information about the last backup.

func (*IncrementalBackupManager) IncrementalBackup 

func (ibm *IncrementalBackupManager) IncrementalBackup(opts *BackupOptions) (*BackupStats, error)

IncrementalBackup performs an incremental backup that only captures changes since the last backup.

func (*IncrementalBackupManager) IncrementalRestore 

func (ibm *IncrementalBackupManager) IncrementalRestore(opts *RestoreOptions) (*BackupStats, error)

IncrementalRestore restores from an incremental backup file.

func (*IncrementalBackupManager) RecordFullBackup 

func (ibm *IncrementalBackupManager) RecordFullBackup(lsn uint64, backupPath string) error

RecordFullBackup records a full backup in the metadata for incremental backup chain.

func (*IncrementalBackupManager) VerifyIncrementalBackup 

func (ibm *IncrementalBackupManager) VerifyIncrementalBackup(path string) error

VerifyIncrementalBackup verifies the integrity of an incremental backup file.

type IncrementalHeader 

type IncrementalHeader struct {
	Magic      [4]byte
	Version    uint32
	Timestamp  int64
	Flags      uint32
	PageSize   uint32
	BaseLSN    uint64
	CurrentLSN uint64
	PageCount  uint64
	TotalBytes uint64
	Checksum   uint32
	Reserved   [20]byte
}

IncrementalHeader represents the header of an incremental backup file. Layout (80 bytes):

  • Bytes 0-3: Magic number ("OBAI")
  • Bytes 4-7: Version (uint32)
  • Bytes 8-15: Timestamp (int64, Unix timestamp)
  • Bytes 16-19: Flags (uint32)
  • Bytes 20-23: PageSize (uint32)
  • Bytes 24-31: BaseLSN (uint64) - LSN of the base backup
  • Bytes 32-39: CurrentLSN (uint64) - Current LSN at backup time
  • Bytes 40-47: PageCount (uint64) - Number of modified pages
  • Bytes 48-55: TotalBytes (uint64) - Total size of page data
  • Bytes 56-59: Checksum (uint32, CRC32 of all page data)
  • Bytes 60-79: Reserved

func NewIncrementalHeader 

func NewIncrementalHeader() *IncrementalHeader

NewIncrementalHeader creates a new incremental backup header with default values.

func (*IncrementalHeader) Deserialize 

func (h *IncrementalHeader) Deserialize(buf []byte) error

Deserialize reads the incremental header from a byte slice.

func (*IncrementalHeader) IsCompressed 

func (h *IncrementalHeader) IsCompressed() bool

IsCompressed returns true if the backup is compressed.

func (*IncrementalHeader) Serialize 

func (h *IncrementalHeader) Serialize() ([]byte, error)

Serialize writes the incremental header to a byte slice.

func (*IncrementalHeader) SerializeTo 

func (h *IncrementalHeader) SerializeTo(buf []byte) error

SerializeTo writes the incremental header to an existing byte slice.

func (*IncrementalHeader) SetCompressed 

func (h *IncrementalHeader) SetCompressed(compressed bool)

SetCompressed sets the compressed flag.

func (*IncrementalHeader) Validate 

func (h *IncrementalHeader) Validate() error

Validate validates the incremental header.

type LDIFExporter 

type LDIFExporter struct {
	// contains filtered or unexported fields
}

LDIFExporter exports entries from ObaDB to LDIF format.

func NewLDIFExporter 

func NewLDIFExporter(engine storage.StorageEngine) *LDIFExporter

NewLDIFExporter creates a new LDIFExporter with the given storage engine.

func (*LDIFExporter) Export 

func (e *LDIFExporter) Export(w io.Writer, baseDN string) error

Export exports all entries under the given baseDN to the writer in LDIF format. It uses subtree scope to include all descendants.

func (*LDIFExporter) ExportEntry 

func (e *LDIFExporter) ExportEntry(w io.Writer, entry *storage.Entry) error

ExportEntry exports a single entry to the writer in LDIF format.

type LDIFImporter 

type LDIFImporter struct {
	// contains filtered or unexported fields
}

LDIFImporter imports entries from LDIF format into ObaDB.

func NewLDIFImporter 

func NewLDIFImporter(engine storage.StorageEngine) *LDIFImporter

NewLDIFImporter creates a new LDIFImporter with the given storage engine.

func (*LDIFImporter) Import 

func (i *LDIFImporter) Import(r io.Reader) error

Import imports entries from the reader in LDIF format. Each entry is imported in its own transaction.

func (*LDIFImporter) ImportBatch 

func (i *LDIFImporter) ImportBatch(r io.Reader) error

ImportBatch imports entries from the reader in a single transaction. This is more efficient for large imports but less safe if an error occurs.

func (*LDIFImporter) Parse 

func (i *LDIFImporter) Parse(r io.Reader) ([]*storage.Entry, error)

Parse parses LDIF content and returns entries without importing them.

type RestoreManager 

type RestoreManager struct {
	// contains filtered or unexported fields
}

RestoreManager manages database restore operations.

func NewRestoreManager 

func NewRestoreManager(dataDir string) *RestoreManager

NewRestoreManager creates a new RestoreManager with the given data directory.

func (*RestoreManager) CleanDataDir 

func (rm *RestoreManager) CleanDataDir() error

CleanDataDir removes all files in the data directory. Use with caution - this is destructive!

func (*RestoreManager) DataDir 

func (rm *RestoreManager) DataDir() string

DataDir returns the configured data directory.

func (*RestoreManager) DiscoverBackupChain 

func (rm *RestoreManager) DiscoverBackupChain(dir string) (*BackupChainInfo, error)

DiscoverBackupChain discovers and validates a backup chain in a directory. It finds the full backup and all related incremental backups.

func (*RestoreManager) GetBackupInfo 

func (rm *RestoreManager) GetBackupInfo(path string) (interface{}, error)

GetBackupInfo returns information about a backup file.

func (*RestoreManager) GetBackupType 

func (rm *RestoreManager) GetBackupType(path string) (string, error)

GetBackupType returns the type of a backup file ("full" or "incremental").

func (*RestoreManager) Restore 

func (rm *RestoreManager) Restore(opts *RestoreOptions) (*RestoreStats, error)

Restore restores a database from a backup file. It automatically detects the backup type (full or incremental) and performs the appropriate restore operation.

func (*RestoreManager) RestoreChain 

func (rm *RestoreManager) RestoreChain(backups []string, opts *RestoreOptions) (*RestoreStats, error)

RestoreChain restores from a full backup and applies a sequence of incremental backups. The backups slice should contain paths to backup files in order: first the full backup, then incremental backups in chronological order.

func (*RestoreManager) RestoreToPointInTime 

func (rm *RestoreManager) RestoreToPointInTime(dir string, targetTime time.Time, opts *RestoreOptions) (*RestoreStats, error)

RestoreToPointInTime restores the database to a specific point in time. It finds the appropriate full backup and applies incremental backups up to the specified timestamp.

func (*RestoreManager) SetDataDir 

func (rm *RestoreManager) SetDataDir(dataDir string)

SetDataDir sets the data directory.

func (*RestoreManager) VerifyBackup 

func (rm *RestoreManager) VerifyBackup(path string) error

VerifyBackup verifies the integrity of a backup file.

type RestoreOptions 

type RestoreOptions struct {
	// InputPath is the path to the backup file.
	InputPath string

	// Verify enables checksum verification before restore.
	Verify bool

	// Format specifies the backup format ("native" or "ldif").
	Format BackupFormat

	// DataDir is the target directory for restored data.
	// Used by RestoreManager for specifying the restore destination.
	DataDir string
}

RestoreOptions configures the restore operation.

func (*RestoreOptions) Validate 

func (o *RestoreOptions) Validate() error

Validate validates the restore options.

type RestoreStats 

type RestoreStats struct {
	// TotalPages is the total number of pages restored.
	TotalPages uint64

	// TotalBytes is the total size of restored data in bytes.
	TotalBytes int64

	// Duration is the time taken to complete the restore.
	Duration time.Duration

	// BackupType indicates the type of backup restored ("full" or "incremental").
	BackupType string

	// BackupsApplied is the number of backup files applied (for chain restore).
	BackupsApplied int
}

RestoreStats contains statistics about a restore operation.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.