mtf

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 17, 2026 License: MIT Imports: 10 Imported by: 0

README

go-mtf

Go Reference

A pure-Go library for reading Microsoft Tape Format (MTF) streams - the format produced by NTBACKUP.EXE and commonly found in .bkf backup files.

Features media spanning, transparent decompression, sparse reconstruction, Media Based Catalog parsing, and near-zero-allocation classification.

Features

  • Typed block iterator - Next yields media/set/entry/set-end blocks; a medium's role is self-evident from the sequence.
  • Faithful extraction - NTFS security descriptors, extended attributes, sparse maps, and every remaining stream are auto-materialized onto the Header; sparse files are reconstructed (holes zero-filled).
  • Transparent decompression - Stac LZS (MTF_LZS221) and the compression/encryption frame layer; encryption via a pluggable decryptor (the spec defines no cipher).
  • Multi-media spanning — reassembles a data set split across media, including mid-file splits. The Continuation callback gives the application full context (tape name, family ID, sequence) to prompt an operator.
  • Media Based Catalog - standard Type 1 Set Map / File/Directory Detail parsing, auto-detected Backup Exec XML catalogs, and a becatalog companion for vendor-specific payloads.
  • Media family identification - Family() combines the TAPE block and Set Map to tell you the family ID, total tape count, and which data sets live on which tapes. Essential for the "I have one tape, what do I need?" use case.

Quick start

r, err := mtf.Open("backup.bkf")
if err != nil { log.Fatal(err) }
defer r.Close()

for {
	b, err := r.Next()
	if err == io.EOF { break }
	if err != nil { log.Fatal(err) }

	switch b.Kind {
	case mtf.KindEntry:
		fmt.Println(b.Header.Name)
		if b.Header.Type == mtf.EntryFile {
			io.Copy(os.Stdout, r) // stream file content
		}
	case mtf.KindSetEnd:
		fmt.Println("data set ended; catalog:", b.Catalog != nil)
	}
}

Documentation

Full reference lives in docs/:

Command-line tools

These are small utilities built on the library, primarily for surveying archives:

Tool Purpose
cmd/bkfscan Parallel .bkf surveyor using Census.
cmd/bkfcensus Single-file Census reporter. 
bkfscan /mnt/archive/BEData          # survey

Project layout

go-mtf/
  mtf.go          public types & constants
  reader.go       the block iterator
  header.go       field accessors & offsets
  strings.go      MTF string decoding
  datetime.go     date/time decoding
  streams.go      data-stream materialization & sparse
  catalog.go      Media Based Catalog (standard)
  spanning.go     multi-media continuation
  compress.go     compression/encryption frames
  lzs.go          Stac LZS decompressor
  census.go       cartridge classification
  becatalog/      Backup Exec XML catalog parser
  cmd/            bkfscan, bkfcensus (survey utilities)
  docs/           detailed documentation

License

MIT. See LICENSE.

Documentation

Overview

Package mtf provides a reader for the Microsoft Tape Format (MTF) used by NTBACKUP.EXE (.bkf) files and LTO tapes.

Archives that span multiple physical media (tapes or .bkf files) are supported via Reader.SetContinuation, which supplies the next medium when an End Of Tape Marker (EOTM) is encountered. The callback receives a Continuation with the exhausted medium's name, sequence, and family ID, giving the application enough context to prompt an operator ("insert tape 2").

The primary entry point is the Reader. Reader.Next advances entry by entry and transparently parses each object's data streams, materializing the metadata a faithful extraction needs (NTFS security descriptors, extended attributes, sparse maps) into the returned Header and positioning file content for Reader.Read: 

r, err := mtf.Open("backup.bkf")
if err != nil { log.Fatal(err) }
defer r.Close()
for {
	b, err := r.Next()
	if err == io.EOF { break }
	if err != nil { log.Fatal(err) }
	if b.Kind == mtf.KindEntry {
		fmt.Println(b.Header.Name)
		if b.Header.Type == mtf.EntryFile {
			io.Copy(os.Stdout, r)
		}
	}
}

Index

Constants

View Source 
const (
	// AttrContinuation (MTF_CONTINUATION, BIT0) is set on descriptor blocks that
	// are repeated on a continuation medium to restore context after an End of
	// Media (EOTM). See MTF spec section 8 (End Of Media Processing).
	AttrContinuation uint32 = 0x00000001
	// AttrCompression indicates compression may be active.
	AttrCompression uint32 = 0x00000002
	// AttrEOSAtEOM indicates End Of Medium was hit during end-of-set processing.
	AttrEOSAtEOM uint32 = 0x00000004
)

Common Block Attributes (MTF_DB_HDR Block Attributes field, offset 4). These apply to the common header of any descriptor block.

View Source 
const (
	WinAttrReadOnly   uint32 = 0x00000001 // FILE_ATTRIBUTE_READONLY
	WinAttrHidden     uint32 = 0x00000002 // FILE_ATTRIBUTE_HIDDEN
	WinAttrSystem     uint32 = 0x00000004 // FILE_ATTRIBUTE_SYSTEM
	WinAttrDirectory  uint32 = 0x00000010 // FILE_ATTRIBUTE_DIRECTORY
	WinAttrArchive    uint32 = 0x00000020 // FILE_ATTRIBUTE_ARCHIVE
	WinAttrDevice     uint32 = 0x00000040 // FILE_ATTRIBUTE_DEVICE
	WinAttrNormal     uint32 = 0x00000080 // FILE_ATTRIBUTE_NORMAL
	WinAttrTemporary  uint32 = 0x00000100 // FILE_ATTRIBUTE_TEMPORARY
	WinAttrSparse     uint32 = 0x00000200 // FILE_ATTRIBUTE_SPARSE_FILE
	WinAttrReparse    uint32 = 0x00000400 // FILE_ATTRIBUTE_REPARSE_POINT
	WinAttrCompressed uint32 = 0x00000800 // FILE_ATTRIBUTE_COMPRESSED
	WinAttrOffline    uint32 = 0x00001000 // FILE_ATTRIBUTE_OFFLINE
	WinAttrEncrypted  uint32 = 0x00004000 // FILE_ATTRIBUTE_ENCRYPTED
)

Windows file attributes (dwFileAttributes). These are the standard Win32 file attribute flags stored in the DBLK Attributes field for FILE and DIRB entries when the OS ID is Windows.

View Source 
const (
	// NTFileLinkFlag (BIT0) is set when the file is a POSIX hard link. When
	// set, the data streams should contain exactly one LINK stream
	// (STRM_NTFS_LINK) pointing to the link target.
	NTFileLinkFlag uint32 = 0x00000001
	// NTFilePOSIX (BIT16) is set when the file is POSIX.
	NTFilePOSIX uint32 = 0x00010000
)

NT file flags from the OS-specific data section of a FILE DBLK (OS ID 14). These are stored in Header.NTFileFlags.

View Source 
const (
	ReparseTagSymlink uint32 = 0xA000000C // IO_REPARSE_TAG_SYMLINK
	ReparseTagMount   uint32 = 0xA0000003 // IO_REPARSE_TAG_MOUNT_POINT
)

Windows reparse tag values for NTRP streams. These identify the type of reparse point encoded in the Header.LinkTarget field.

View Source 
const (
	// StreamMediaContinue (STREAM_CONTINUE, BIT0) marks a stream whose data is a
	// continuation of a stream split across media at EOM. Its Stream Length holds
	// only the remaining (unwritten) portion and its data begins at the next
	// Format Logical Block boundary. See MTF spec section 6.1.
	StreamMediaContinue uint16 = 0x0001
	// StreamMediaEncrypted (STREAM_ENCRYPTED, BIT3) marks an encrypted stream.
	StreamMediaEncrypted uint16 = 0x0008
	// StreamMediaCompressed (STREAM_COMPRESSED, BIT4) marks a compressed stream.
	StreamMediaCompressed uint16 = 0x0010
)

Stream Media Format Attributes (MTF_STREAM_HDR Stream Media Format Attributes field, stream header offset 6).

View Source 
const (
	StreamSTAN uint32 = 0x4E415453 // standard data
	StreamPNAM uint32 = 0x4D414E50 // path
	StreamFNAM uint32 = 0x4D414E46 // file name
	StreamCSUM uint32 = 0x4D555343 // checksum
	StreamCRPT uint32 = 0x54505243 // corrupt
	StreamSPAD uint32 = 0x44415053 // padding (marks the last stream of an object)
	StreamSPAR uint32 = 0x52415053 // sparse

	StreamTSMP uint32 = 0x504D5354 // set map, media based catalog, type 1
	StreamTFDD uint32 = 0x44444654 // fdd, media based catalog, type 1
	StreamMAP2 uint32 = 0x3250414D // set map, media based catalog, type 2
	StreamFDD2 uint32 = 0x32444446 // fdd, media based catalog, type 2
	StreamSM2P uint32 = 0x32504D53 // set map, media based catalog (variant)

	StreamADAT uint32 = 0x54414441 // NT data
	StreamNTEA uint32 = 0x4145544E // NT extended attributes
	StreamNACL uint32 = 0x4C43414E // NT ACL
	StreamNTRP uint32 = 0x5052544E // NT reparse point (symlinks, mount points)
	StreamLINK uint32 = 0x4B4E494C // NT hard link target
	StreamNTED uint32 = 0x4445544E // NT EData
	StreamNTQU uint32 = 0x5551544E // NT quota
	StreamNTPR uint32 = 0x5250544E // NT property
	StreamNTOI uint32 = 0x494F544E // NT object id
	StreamNTQP uint32 = 0x5051544E // NT quota/property (vendor variant)

	StreamGERC uint32 = 0x43524547 // Win9x

	StreamN386 uint32 = 0x3638334E // Netware
	StreamNBND uint32 = 0x444E424E // Netware
	StreamSMSD uint32 = 0x44534D53 // Netware

	StreamOACL uint32 = 0x4C43414F // OS/2 ACL

	StreamMRSC uint32 = 0x4353524D // Macintosh resource
	StreamMPRV uint32 = 0x5652504D // Macintosh private
	StreamMINF uint32 = 0x464E494D // Macintosh info
)

Stream data type identifiers. These are the four-byte stream type codes read as little-endian uint32 values (e.g. "STAN" -> 0x4E415453). They are exported so callers can interpret the stream types associated with an entry.

View Source 
const (

	// MTF_LZS221: the single registered software compression algorithm (Appendix C).
	AlgLZS221 = uint16(0x0ABE)
)
View Source 
const (
	// StreamFSSparse (STREAM_IS_SPARSE, BIT3) marks a stream whose data is
	// sparse. See MTF spec section 6.1.
	StreamFSSparse uint16 = 0x0008
)

Stream File System Attributes (MTF_STREAM_HDR Stream File System Attributes field, stream header offset 4).

Variables

View Source 
var ErrEncrypted = errors.New("mtf: stream is encrypted; register a decryptor with SetDecryptor")

ErrEncrypted is returned when a stream is encrypted but no decryptor has been registered with Reader.SetDecryptor.

Functions

func FormatSID 

func FormatSID(sid []byte) string

FormatSID converts a raw binary SID to its string representation (e.g. "S-1-5-21-..."). It returns an empty string if the SID is invalid.

func StreamTypeName 

func StreamTypeName(t uint32) string

StreamTypeName returns a human-readable name for a stream data type identifier.

Types

type Block 

type Block struct {
	Kind    BlockKind
	Tape    *TapeInfo // populated when Kind == KindMedia
	Set     *SetInfo  // populated when Kind == KindSet
	Header  *Header   // populated when Kind == KindEntry
	ESet    *ESetInfo // populated when Kind == KindSetEnd
	Catalog *Catalog  // populated when Kind == KindSetEnd (nil if the set had no catalog)
}

Block is a single structural element yielded by Reader.Next. Its Kind discriminates which field is populated. A single Block (and Header) are reused across calls to Next, so all fields are overwritten on the next call; callers should copy any values they need to retain.

type BlockKind 

type BlockKind uint8

BlockKind identifies the kind of MTF structure a Block represents. 

const (
	// KindMedia is yielded for an MTF_TAPE descriptor block: the start of a
	// physical medium. With media spanning enabled, one iteration yields a
	// KindMedia block for each medium as it is consumed.
	KindMedia BlockKind = iota
	// KindSet is yielded for an MTF_SSET descriptor block: the start of a data
	// set (one backup operation).
	KindSet
	// KindEntry is yielded for an extractable object descriptor (MTF_VOLB,
	// MTF_DIRB or MTF_FILE). The header is fully materialized; call [Reader.Read]
	// to stream the entry's standard data.
	KindEntry
	// KindSetEnd is yielded for an MTF_ESET descriptor block: a data set ended.
	// Any Media Based Catalog carried by the set's streams is attached as
	// Catalog (nil when the set recorded no catalog).
	KindSetEnd
)

type CartridgeRole 

type CartridgeRole int

CartridgeRole classifies the role a medium plays within its Media Family, inferred from the catalog type recorded on its TAPE block. 

const (
	// RoleUnknown is used when no TAPE block was seen or the catalog type is
	// unrecognized.
	RoleUnknown CartridgeRole = iota
	// RoleData is a cartridge that primarily carries file data
	// (Backup Exec CatalogType 64).
	RoleData
	// RoleCatalog is a consolidated catalog cartridge
	// (Backup Exec CatalogType 128).
	RoleCatalog
)

type Catalog 

type Catalog struct {
	// SetMap is the cumulative Media Family summary parsed from the 'TSMP'
	// stream. It lists one entry per data set in the family. It may be nil when
	// no standard Set Map stream was present.
	SetMap *SetMap
	// FDD is the per-data-set File/Directory Detail parsed from the 'TFDD'
	// stream: every volume, directory and file, in archive order, each carrying
	// the location (MediaSeq + FLA) of its descriptor block. It is empty when no
	// standard FDD stream was present or when the FDD payload is a Backup Exec
	// XML catalog (see BECatalog).
	FDD []CatalogEntry
	// BECatalog holds the parsed Backup Exec catalog, auto-detected from the
	// FDD stream. It is nil when the FDD payload is a standard MTF binary
	// catalog (in which case FDD is populated) or when no FDD stream was present.
	BECatalog *becatalog.Catalog
	// RawFDD is the unparsed payload of the FDD data stream. It is populated
	// whenever an FDD stream was captured, including vendor-specific payloads
	// the standard parser does not understand (so a vendor parser can take over).
	RawFDD []byte
	// RawSetMap is the unparsed payload of the Set Map data stream, populated
	// whenever a Set Map stream was captured.
	RawSetMap []byte
}

Catalog holds the parsed Media Based Catalog of the most recently completed data set, available via Reader.Catalog. It is nil when no MBC streams were present on the medium, or when the catalog was written in an unrecognized (vendor-specific) format that left no parseable standard records.

func (*Catalog) Raw 

func (c *Catalog) Raw() CatalogRaw

Raw returns the captured catalog stream payloads, implementing CatalogData. A vendor-specific catalog parser consumes CatalogRaw.FDD directly.

type CatalogData 

type CatalogData interface {
	// Raw returns the captured catalog stream payloads. FDD is the
	// File/Directory Detail ('TFDD'/'FDD2') payload and SetMap is the Set Map
	// ('TSMP'/'MAP2') payload; either may be nil if the stream was absent. For a
	// standard Type 1 catalog these are binary records; a writer may substitute a
	// vendor-specific payload (so a vendor parser takes over from Raw.FDD).
	Raw() CatalogRaw
}

CatalogData exposes the raw, uninterpreted catalog stream payloads captured from a data set's ESET block. It decouples vendor-specific catalog parsers (for example a Backup Exec XML parser) from this package's concrete Catalog type: a vendor parser accepts a CatalogData and works purely from the bytes.

Catalog satisfies CatalogData via its Catalog.Raw method.

type CatalogEntry 

type CatalogEntry struct {
	// Type is the kind of object.
	Type CatalogEntryType
	// MediaSeq is the MEDIA_SEQ_NUMBER: the 1-based sequence of the medium
	// within the Media Family that holds the object's descriptor block.
	MediaSeq uint16
	// FLA is the Format Logical Address of the object's descriptor block, the
	// byte offset within the medium at which the DBLK is written. Together with
	// MediaSeq it locates the object for random-access extraction.
	FLA uint64
	// Size is the DISPLAYABLE_SIZE copied from the descriptor block.
	Size uint64
	// Attributes is the type-specific attribute word (VOLB/DIRB/FILE
	// attributes) copied from the descriptor block.
	Attributes uint32
	// BlockAttributes is the MTF_DB_HDR common block attributes word.
	BlockAttributes uint32
	// Link is the FDD LINK field: for a directory, the offset of the next
	// sibling directory within the FDD; for a file, the stream offset of its
	// parent directory; for a volume, the offset of the next volume entry.
	Link int32
	// Name is the object name (directory/file name) or, for a volume, the
	// device name.
	Name string
	// VolumeLabel is the volume label (volume entries only).
	VolumeLabel string
	// MachineName is the source machine name (volume entries only). In a Set
	// Map it identifies the host that owns the data set.
	MachineName string
	// WriteTime is the media write date of a volume entry.
	WriteTime time.Time
	// ModTime, CreateTime, BackupTime and AccessTime are the object timestamps
	// copied from the descriptor block (directory and file entries).
	ModTime, CreateTime, BackupTime, AccessTime time.Time
}

CatalogEntry describes one object (volume, directory or file) recorded in the File/Directory Detail. Each entry carries the location of its descriptor block on medium, allowing a reader to seek directly to the object.

type CatalogEntryType 

type CatalogEntryType int

CatalogEntryType identifies the kind of object a CatalogEntry records. 

const (
	// EntryCatalogVolume is an FDD volume entry (corresponds to a VOLB DBLK).
	EntryCatalogVolume CatalogEntryType = iota
	// EntryCatalogDirectory is an FDD directory entry (corresponds to a DIRB
	// DBLK).
	EntryCatalogDirectory
	// EntryCatalogFile is an FDD file entry (corresponds to a FILE DBLK).
	EntryCatalogFile
)

type CatalogRaw 

type CatalogRaw struct {
	FDD    []byte
	SetMap []byte
}

CatalogRaw holds the raw, uninterpreted catalog stream payloads.

type Census 

type Census struct {
	// Tape is the metadata from the cartridge's TAPE block. It is nil when the
	// stream did not begin with a TAPE block (for example a bare continuation).
	Tape *TapeInfo
	// Set is the metadata from the data-set start (SSET) block, or nil.
	Set *SetInfo
	// Role is the inferred cartridge role.
	Role CartridgeRole
	// CatalogType is the catalog type recorded on the TAPE block
	// (0 when absent).
	CatalogType uint16
	// MediaSequence is the media sequence number (1-based within a family).
	// Derived from the TAPE Sequence field; a standalone cartridge reports 1.
	MediaSequence uint16
	// SetsClosed is the number of ESET blocks seen (data sets that end on this
	// cartridge). A mid-span continuation with no data-set end reports 0.
	SetsClosed int
	// HasCatalog reports whether any end-of-set block carried catalog streams
	// (Media Based Catalog: TFDD/FDD2/TSMP/MAP2).
	HasCatalog bool
	// CatalogBytes is the total catalog payload captured (TFDD + FDD2 +
	// SetMap), regardless of whether a standard parser could decode it.
	CatalogBytes int64

	// Volumes is the number of volume (VOLB) entries.
	Volumes int
	// Directories is the number of directory (DIRB) entries.
	Directories int
	// Files is the number of file (FILE) entries, including empty ones.
	Files int
	// EmptyFiles is the number of file entries with no data stream.
	EmptyFiles int
	// FileBytes is the sum of every file's on-media (stored) data size. For
	// uncompressed cartridges this equals the logical size; for compressed or
	// sparse ones it reflects the stored byte count.
	FileBytes int64
	// SparseFiles is the number of sparse file entries.
	SparseFiles int
	// CompressedFiles is the number of file entries whose data stream is
	// compressed.
	CompressedFiles int
	// EncryptedFiles is the number of file entries whose data stream is
	// encrypted.
	EncryptedFiles int
}

Census summarizes the shape of a single cartridge (one MTF stream, i.e. one .bkf file or one tape) at a glance, without reading file content. It is produced by Reader.Census, which walks every block but drains file data streams instead of delivering their bytes, so it is suitable for cheaply classifying large archives.

Census reads every block header and stream descriptor, so its file/byte counts are authoritative for the cartridge; only the content bytes are not materialized.

func (*Census) HasData 

func (c *Census) HasData() bool

HasData reports whether the cartridge carries any file content.

type Continuation 

type Continuation struct {
	// Sequence is the 1-based index of the medium that just ended within the
	// media family. The next medium to load therefore has Sequence+1.
	Sequence int
	// Media holds the descriptor of the exhausted medium, or nil if no TAPE
	// block was parsed for it.
	Media *TapeInfo
}

type Decryptor 

type Decryptor func(algo uint16, encrypted []byte) ([]byte, error)

Decryptor reverses the encryption of one frame's payload. algo is the stream's Data Encryption Algorithm ID (from the STAN header); encrypted holds the raw encrypted bytes. It must return the plaintext of exactly those bytes. The MTF spec defines no cipher, so the implementation is vendor-specific.

type ESetInfo 

type ESetInfo struct {
	Attributes       uint32 // ESET block attributes
	CorruptObjects   uint32 // number of corrupt files in the data set
	FDDMediaSequence uint16 // FDD media sequence number
	SetNumber        uint16 // data-set number being closed
	CreateTime       time.Time
}

ESetInfo holds metadata from the most recent end-of-data-set (ESET) block. It is available via Reader.ESet after a data set has ended.

type EntryType 

type EntryType int

EntryType describes the kind of entry a Header represents. 

const (
	// EntryFile is a regular file. Its contents are available via [Reader.Read].
	EntryFile EntryType = iota
	// EntryDirectory is a directory entry.
	EntryDirectory
	// EntryVolume is a source volume (device) the backup was taken from.
	EntryVolume
)
type Header struct {
	// Type is the kind of entry.
	Type EntryType
	// Name is the fully resolved path of the entry, formed from the source
	// volume, directory chain and the entry name using "/" separators.
	Name string
	// ModTime is the modification time of the entry.
	ModTime time.Time
	// AccessTime is the last access time, if recorded.
	AccessTime time.Time
	// CreateTime is the creation time, if recorded.
	CreateTime time.Time
	// BirthTime is the birth time of the entry, if recorded. Only emitted by
	// NT-based backups that populate the MTF birth-time field.
	BirthTime time.Time
	// Attributes is the block's type-specific attribute flags (the DBLK
	// attributes field that follows the common header). For FILE entries this is
	// the Windows dwFileAttributes field; for DIRB entries the directory
	// attributes. Test individual bits with the [WinAttr] constants.
	Attributes uint32
	// NTFileFlags is the Windows NT file-specific flags from the OS-specific
	// data section of a FILE DBLK (OSID 14). It is zero for non-NT entries.
	// Test individual bits with the [NTFile] constants.
	NTFileFlags uint32
	// BlockAttributes is the MTF_DB_HDR common Block Attributes field, which
	// carries bits such as MTF_CONTINUATION and MTF_COMPRESSION. Use the
	// Attr* constants to test it.
	BlockAttributes uint32
	// OSID is the operating system identifier recorded in the block.
	OSID uint8
	// SetNumber is the backup data-set number this entry belongs to.
	SetNumber uint16
	// Volume is the name of the source volume/device the entry resides on.
	Volume string
	// VolumeLabel is the source volume's label, if recorded (volume entries).
	VolumeLabel string
	// MachineName is the name of the source machine, if recorded (volume
	// entries).
	MachineName string
	// FileID is the MTF object identifier of the file (files only).
	FileID uint32
	// DirID is the identifier of the directory containing the entry.
	DirID uint32
	// LinkTarget is the target path for symbolic links and hard links.
	// It is populated from the NTRP (reparse) stream for symlinks and the
	// LINK stream for hard links. It is empty for regular entries.
	LinkTarget string
	// IsSymlink reports whether this entry is a symbolic link. Set when an
	// NTRP stream with reparse tag IO_REPARSE_TAG_SYMLINK is present.
	IsSymlink bool
	// IsHardLink reports whether this entry is a POSIX hard link. Set when
	// the NT file flags [NTFileLinkFlag] bit is set or a LINK stream is
	// present.
	IsHardLink bool

	// The following describe the file's standard (STAN) data stream and are
	// meaningful for file entries. They are populated by [Reader.Next].
	//
	// Size is the logical length in bytes of the file's content as delivered by
	// [Reader.Read]. For a plain file this is the STAN stream length; for a
	// sparse file it is the reconstructed (hole-filled) length; for a
	// compressed/encrypted file it is the on-media (stored) byte count (see
	// [Header.DisplayableSize] for the logical size in that case). It is zero
	// for non-file entries.
	Size int64
	// CompressionAlgorithm is the registered ID of the algorithm used to
	// compress the standard data stream, or zero if uncompressed.
	CompressionAlgorithm uint16
	// EncryptionAlgorithm is the registered ID of the algorithm used to
	// encrypt the standard data stream, or zero if unencrypted.
	EncryptionAlgorithm uint16
	// Compressed reports whether the standard data stream is compressed. The
	// bytes returned by [Reader.Read] are still compressed; decompression is
	// not performed.
	Compressed bool
	// Encrypted reports whether the standard data stream is encrypted. The
	// bytes returned by [Reader.Read] are still encrypted; decryption is not
	// performed.
	Encrypted bool
	// Sparse reports whether the file is sparse. For sparse files [Reader.Read]
	// transparently reconstructs the logical content (holes are zero-filled)
	// and [Header.SparseExtents] holds the parsed sparse map. The STREAM_IS_SPARSE
	// bit is documented in MTF spec section 6.1.
	Sparse bool
	// StreamChecksum is the checksum field of the standard (STAN) data stream
	// header (zero unless the stream is checksummed).
	StreamChecksum uint16
	// DisplayableSize is the object size recorded in the common descriptor
	// block's Displayable Size field. For uncompressed files it equals Size;
	// for compressed or sparse objects it reflects the logical (expanded) size.
	DisplayableSize uint64

	// SecurityDescriptor holds the raw NTFS security descriptor (NACL stream)
	// associated with the entry, if any. It is a self-relative security
	// descriptor as produced by the Win32 BackupRead API. Present on both file
	// and directory entries.
	SecurityDescriptor []byte
	// ExtendedAttributes holds the raw NT extended-attribute data (NTEA stream)
	// associated with the entry, if any.
	ExtendedAttributes []byte
	// SparseExtents describes the sparse layout of a sparse file (one entry per
	// SPAR stream), or nil for a non-sparse entry. Each extent carries the
	// non-hole bytes located at [SparseExtent.Offset] in the logical file;
	// [Reader.Read] fills the gaps with zero bytes. See MTF spec section 6.2.1.7.
	SparseExtents []SparseExtent
	// Streams holds every data stream associated with the entry that is not
	// carried by a named field above (e.g. NTOI object ids, NTQU quota, CSUM
	// checksums, ADAT alternate data, or vendor-specific streams). Each element
	// preserves the stream's four-byte type code and raw bytes so no metadata
	// is lost. Streams both preceding and following the standard (STAN) data
	// stream are captured. It is nil in [Reader.HeaderOnly] mode, which skips
	// stream data, and for entries that have no extra streams.
	Streams []StreamData
}

Header describes a single entry (file, directory or volume) within an MTF archive. It is returned by Reader.Next via Block.Header. A single Header is reused across entries, so its fields are overwritten on the next call to Next; callers that retain an entry across iterations must copy the fields they need.

func (*Header) GroupSID 

func (h *Header) GroupSID() []byte

GroupSID returns the group SID from the security descriptor stored in Header.SecurityDescriptor, or nil if no descriptor is present or it is too short to contain a group.

func (*Header) OwnerSID 

func (h *Header) OwnerSID() []byte

OwnerSID returns the owner SID from the security descriptor stored in Header.SecurityDescriptor, or nil if no descriptor is present or it is too short to contain an owner. The SID is returned in raw binary form (self-relative SECURITY_DESCRIPTOR format).

To convert a raw SID to string form (S-1-5-21-...), use FormatSID.

func (*Header) UnixMode 

func (h *Header) UnixMode() uint32

UnixMode returns a best-effort Unix permission mode derived from the Windows file attributes stored in Header.Attributes. The mapping follows common conventions:

  • Directories get mode 0755 (readable/traversable by all, writable by owner).
  • Regular files get mode 0644 (readable by all, writable by owner), or 0755 if the archive bit is set (a heuristic for executables on Unix-origin backups).
  • The readonly bit clears the owner-write bit.
  • The system bit is not mapped (Unix has no equivalent).

For precise permissions, parse Header.SecurityDescriptor and extract the DACL. The mode returned here is a reasonable default for migration tools that need a starting point.

type MediaFamily 

type MediaFamily struct {
	// ID is the Media Family ID (MFMID) from the TAPE block. All cartridges in
	// the same family share this ID, making it the key for grouping scattered
	// tapes into a restore set.
	ID uint32
	// TotalTapes is the total number of cartridges in this family, derived from
	// the Set Map (the maximum MediaSeq across all data-set entries). It is 0 when
	// no Set Map was present — for example a data-only cartridge that is not the
	// last in the family won't carry a Set Map.
	TotalTapes int
	// TapeSequence is this cartridge's 1-based position in the family, matching
	// [TapeInfo.Sequence].
	TapeSequence int
	// TapeName is the name/label of this cartridge.
	TapeName string
	// SetMap is the parsed Set Map from this cartridge, or nil if none was
	// present. Each [SetMapEntry] carries a [SetMapEntry.MediaSeq] that identifies
	// which tape the data set starts on; scanning them reveals the full set of
	// tapes in the family.
	SetMap *SetMap
}

MediaFamily summarizes what is known about the media family from a single cartridge. It combines the TAPE descriptor and the Set Map (if present) to answer questions like "which media family is this?" and "how many tapes do I need for a full restore?"

Call Reader.Family to obtain one.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader provides sequential access to the entries of an MTF/BKF stream.

Use [Next] to advance to the next entry and [Read] to read the standard data of the current file entry. The API intentionally mirrors archive/tar.

func NewReader 

func NewReader(r io.Reader) *Reader

NewReader returns a new Reader reading from r. When r implements io.Seeker (for example an *os.File), the reader skips data streams by seeking rather than reading, which is dramatically faster when only block headers are needed (listing, cataloging) on large archives.

func Open 

func Open(name string) (*Reader, error)

Open opens the named MTF/BKF file for reading.

func (*Reader) Catalog 

func (r *Reader) Catalog() *Catalog

Catalog returns the Media Based Catalog of the most recently completed data set, or nil if no MBC streams were captured. It is meaningful once Reader.Next has advanced past the data set's end (its first ESET block) or reached end-of-archive.

The catalog is parsed lazily on first call and cached.

func (*Reader) Census 

func (r *Reader) Census() (Census, error)

Census walks the cartridge, classifying it into a Census (returned by value, so it need not heap-allocate). File content is discarded rather than returned. It consumes the reader: after it returns the stream is exhausted (or stopped at the first read error). Use a fresh reader to extract content.

Census runs in header-only mode, so it performs zero per-entry allocations; the only allocations are the Reader itself and its block buffer.

Even when err is non-nil, the returned Census is populated with whatever was read before the error; this lets a caller classify partially readable cartridges.

func (*Reader) Checksum 

func (r *Reader) Checksum() (stored, computed uint16)

Checksum returns the MTF common-block header checksum field of the current block together with the value recomputed from the remaining header fields. Equal values indicate an intact header.

func (*Reader) Close 

func (r *Reader) Close() error

Close closes the underlying reader if it was opened by Open.

func (*Reader) CorruptObjects 

func (r *Reader) CorruptObjects() uint32

CorruptObjects returns the corrupt-object count reported by the most recent end-of-data-set (ESET) block, or zero if no set has ended yet.

func (*Reader) ESet 

func (r *Reader) ESet() *ESetInfo

ESet returns metadata from the most recent end-of-data-set (ESET) block, or nil if no data set has ended yet. The returned value is shared; callers must not retain it across subsequent calls to Reader.Next.

func (*Reader) Family 

func (r *Reader) Family() MediaFamily

Family returns what is known about the media family from the current cartridge. It combines the TAPE descriptor and the Set Map (if present) to answer questions like "which media family is this tape?" and "how many tapes do I need for a full restore?"

Call after the first KindMedia block has been processed (which fills TapeInfo) and, for the most complete family information, after the first KindSetEnd (which fills the Set Map).

The Set Map is cumulative — the one on the last cartridge of the family is the most complete. On a data-only cartridge (CatalogType 64) the Set Map may be nil; on a catalog cartridge (CatalogType 128) it is typically present.

func (*Reader) HeaderOnly 

func (r *Reader) HeaderOnly()

HeaderOnly puts the reader into header-only mode: metadata stream *data* (NTFS security descriptors, extended attributes, sparse maps) is skipped rather than read into Header fields, and standard (STAN) file content is never delivered. Entry *names* are also skipped (Header.Name is left empty) since string construction is the dominant per-entry allocation; block and stream *headers* are still parsed, so sizes and flags remain accurate. Combined with a seekable source this lets a caller walk a cartridge reading essentially only headers, with zero per-entry allocations. It is used by Reader.Census.

func (*Reader) MediaSequence 

func (r *Reader) MediaSequence() int

MediaSequence reports the 1-based index of the current physical medium. It is 1 for the initial medium and increments each time a continuation medium supplied via Reader.SetContinuation is switched to. It is always 1 for non-spanned archives.

func (*Reader) Next 

func (r *Reader) Next() (*Block, error)

Next advances through the MTF stream and returns the next structural block. The returned Block.Kind tells you what was encountered:

  • KindMedia: a medium (MTF_TAPE) started. Block.Tape holds its metadata.
  • KindSet: a data set (MTF_SSET) started. Block.Set holds its metadata.
  • KindEntry: an extractable object (MTF_VOLB/DIRB/FILE). Block.Header is fully materialized; call Reader.Read to stream its standard data.
  • KindSetEnd: a data set (MTF_ESET) ended. Block.ESet holds its metadata and Block.Catalog carries any Media Based Catalog (nil if none).

The medium's role is self-evident from the block sequence: a medium with KindEntry blocks but no trailing KindSetEnd is data-only (its data set continues on the next medium); one whose KindSetEnd carries a Catalog with no file-data entries is catalog-heavy; one with both is the normal case.

Media spanning is handled transparently: when a continuation is registered via Reader.SetContinuation, each consumed medium yields its own KindMedia. Next returns io.EOF when the archive is fully consumed.

func (*Reader) Position 

func (r *Reader) Position() int64

Position reports the byte offset already consumed from the underlying stream. It is intended for diagnostics and for building seek indexes for random access extraction.

func (*Reader) Read 

func (r *Reader) Read(p []byte) (int, error)

Read reads data from the current file entry into p. It returns the reconstructed file content: for a plain file this is the standard data (STAN) stream, transparently followed across continuation media; for a sparse file the holes are zero-filled according to the parsed sparse map.

Decompression and decryption are not performed: for a compressed or encrypted stream the raw stored bytes are returned.

Read returns io.EOF when the entry's content is exhausted. Calling Read on a non-file entry returns io.EOF immediately.

func (*Reader) Set 

func (r *Reader) Set() *SetInfo

Set returns metadata from the current (most recent) start-of-data-set block, or nil if none has been encountered yet.

func (*Reader) SetContinuation 

func (r *Reader) SetContinuation(next func(Continuation) (io.Reader, error))

SetContinuation registers a function that supplies the next physical medium when the current one ends (an MTF_EOTM block is reached).

This enables reading a single backup data set that spans multiple media (tapes or .bkf files). When the reader encounters an End Of Tape Marker (EOTM) — whether between entries or in the middle of a file's data stream — it calls next with a Continuation describing the medium that just ended, and resumes from the reader next returns.

The callback is the natural place to prompt an operator to load the next tape, e.g.: 

files := []string{"tape-1.bkf", "tape-2.bkf"}
r.SetContinuation(func(c mtf.Continuation) (io.Reader, error) {
    if c.Sequence >= len(files) {
        return nil, io.EOF // no more media
    }
    fmt.Printf("load %s (tape %d)\n", files[c.Sequence], c.Sequence+1)
    return os.Open(files[c.Sequence])
})

If next is nil (the default), an EOTM ends the archive like io.EOF. If next returns io.EOF or a nil reader, the archive ends.

func (*Reader) SetDecryptor 

func (r *Reader) SetDecryptor(d Decryptor)

SetDecryptor registers a callback used to decrypt encrypted data streams (§6.5). The MTF specification defines no data-encryption cipher, so the algorithm is vendor-specific; supply a Decryptor matching the writer. If a stream is encrypted and no decryptor is registered, [Read] returns ErrEncrypted.

func (*Reader) Tape 

func (r *Reader) Tape() *TapeInfo

Tape returns metadata from the most recent TAPE descriptor block, or nil if none has been encountered yet.

func (*Reader) TruncatedByEOTM 

func (r *Reader) TruncatedByEOTM() bool

TruncatedByEOTM reports whether the archive ended prematurely because an End-Of-Tape-Media marker was reached without a continuation medium being registered via Reader.SetContinuation. When true, the data set spans further media that were not supplied, and the returned snapshot is incomplete. Callers should warn the operator.

func (*Reader) VerifyChecksum 

func (r *Reader) VerifyChecksum() bool

VerifyChecksum reports whether the common-block header (MTF_DB_HDR) checksum of the current block matches the recomputed word-wise XOR over the remaining header fields (MTF spec, "Header Checksum"). This may be used to detect media corruption. It should be called immediately after [Next] returns (before [Read] consumes the entry). It always returns true when the current block buffer is too short to contain a checksum (nothing to verify).

Note: some writers emit a zero checksum; such blocks verify as valid only when every other header word is also zero, so a false "invalid" is possible for such archives. Treat the result as advisory.

type SetInfo 

type SetInfo struct {
	Name   string // set name
	Label  string // set label
	Owner  string // set owner/user
	Number uint16 // data-set number
	// Password is the data-set password, if recorded.
	Password string
	// PBA is the physical block address of the SSET block, used for seek
	// indexing in conjunction with a Media Based Catalog.
	PBA uint64
	// SoftwareVendorID is the registered vendor ID of the writing software.
	SoftwareVendorID uint16
	// SoftwareVersion is the writer's software version number.
	SoftwareVersion uint16
	Attributes      uint32
	Compression     uint16
	Encryption      uint16
	MajorVersion    uint8
	MinorVersion    uint8
	TimeZone        int8
	CreateTime      time.Time
}

SetInfo holds metadata from the MTF start-of-data-set (SSET) block.

type SetMap 

type SetMap struct {
	// MediaFamilyID is the MFMID of the Media Family this Set Map describes; it
	// matches TapeInfo.MFMID.
	MediaFamilyID uint32
	// Entries holds one entry per data set in the Media Family.
	Entries []SetMapEntry
}

SetMap is the cumulative Media Family summary parsed from a 'TSMP' stream. It lists one SetMapEntry per data set in the family, in the order the data sets were written.

type SetMapEntry 

type SetMapEntry struct {
	// MediaSeq is the Media Sequence Number of the medium this data set begins
	// on, copied from the TAPE block.
	MediaSeq uint16
	// FDDMediaSeq is the FDD Media Sequence Number from the ESET block.
	FDDMediaSeq uint16
	// SetNumber is the data-set number copied from the SSET block.
	SetNumber uint16
	// BlockAttributes is the MTF_DB_HDR common block attributes word.
	BlockAttributes uint32
	// SSETAttributes is the SSET attributes word.
	SSETAttributes uint32
	// SSETPBA is the Physical Block Address of the SSET block.
	SSETPBA uint64
	// FDDPBA is the Physical Block Address of this data set's FDD stream.
	FDDPBA uint64
	// FLA is the Format Logical Address of the SSET block.
	FLA uint64
	// NumDirectories is the count of directories in the data set.
	NumDirectories uint32
	// NumFiles is the count of files in the data set.
	NumFiles uint32
	// NumCorrupt is the count of corrupt files in the data set.
	NumCorrupt uint32
	// Size is the cumulative displayable size of the data set.
	Size uint64
	// Name is the data-set name.
	Name string
	// Description is the data-set description.
	Description string
	// Owner is the user name associated with the data set.
	Owner string
	// WriteTime is the media write date copied from the SSET block.
	WriteTime time.Time
	// TimeZone is the SSET time-zone offset (signed quarter-hours).
	TimeZone int8
	// Volumes are the volume entries (MTF_FDD_VOLB) following this Set Map
	// Entry, one per VOLB in the data set. Each carries the source device,
	// volume label and machine name.
	Volumes []CatalogEntry
}

SetMapEntry summarizes one data set (one SSET, typically one host's backup) within a Media Family, and is followed by its volume entries.

type SparseExtent 

type SparseExtent struct {
	// Offset is the logical byte offset within the file where Data begins.
	Offset int64
	// Data is the non-hole byte content located at Offset.
	Data []byte
}

SparseExtent describes one contiguous block of non-hole data within a sparse file, as parsed from a SPAR data stream. The logical file is reconstructed by placing each extent's Data at Offset and zero-filling the gaps between extents.

type StreamData 

type StreamData struct {
	Type uint32
	Data []byte
}

StreamData holds the raw bytes of a data stream not otherwise exposed by a named field on Header. Type is the four-byte stream type code (compare against the Stream* constants, e.g. StreamNTOI).

type TapeInfo 

type TapeInfo struct {
	Software   string // generator software string
	Name       string // tape name
	Label      string // tape label
	Password   string // media password, if recorded
	MFMID      uint32 // media family id
	Attributes uint32 // TAPE attributes
	Sequence   uint16 // media sequence number within the media family
	FLBSize    uint16 // logical block size used by the archive
	// PasswordAlgorithm is the registered ID of the password-encryption
	// algorithm used to protect the media password, or zero.
	PasswordAlgorithm uint16
	// SoftFilemarkBlockSize is the soft filemark (SFMB) block size in units of
	// 512 bytes; only meaningful when soft filemarks are used.
	SoftFilemarkBlockSize uint16
	// CatalogType is the Media Based Catalog format type recorded on the tape.
	CatalogType uint16
	// SoftwareVendorID is the registered vendor ID of the writing software.
	SoftwareVendorID uint16
	// MTFMajorVersion is the MTF major revision recorded in the TAPE block.
	MTFMajorVersion uint8
	CreateTime      time.Time
}

TapeInfo holds metadata from the MTF TAPE descriptor block.

Source Files

View all Source files

Directories

Path Synopsis
Package becatalog parses the Backup Exec catalog payload that some MTF writers place inside the standard File/Directory Detail (FDD) stream envelope.
 Package becatalog parses the Backup Exec catalog payload that some MTF writers place inside the standard File/Directory Detail (FDD) stream envelope.
cmd
bkfcensus command
Command bkfcensus runs mtf.Census on a single .bkf file and prints a compact one-line classification.
 Command bkfcensus runs mtf.Census on a single .bkf file and prints a compact one-line classification.
bkfscan command
Command bkfscan classifies every .bkf file in one or more directories using mtf.Census, printing a one-line summary and tallying aggregate stats.
 Command bkfscan classifies every .bkf file in one or more directories using mtf.Census, printing a one-line summary and tallying aggregate stats.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.