mtf

package module
v0.22.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 19, 2026 License: MIT Imports: 11 Imported by: 0

README

go-mtf

Go Reference

A pure-Go library for reading Microsoft Tape Format (MTF) streams - the format produced by NTBACKUP.EXE and commonly found in .bkf backup files.

Features media spanning, transparent decompression, sparse reconstruction, Media Based Catalog parsing, and near-zero-allocation classification.

Features

  • Typed block iterator - Next yields media/set/entry/set-end blocks; a medium's role is self-evident from the sequence.
  • Faithful extraction - NTFS security descriptors, extended attributes, sparse maps, and every remaining stream are auto-materialized onto the Header; sparse files are reconstructed (holes zero-filled).
  • Transparent decompression - Stac LZS (MTF_LZS221) and the compression/encryption frame layer; encryption via a pluggable decryptor (the spec defines no cipher).
  • Multi-media spanning — reassembles a data set split across media, including mid-file splits. The Continuation callback gives the application full context (tape name, family ID, sequence) to prompt an operator.
  • Media Based Catalog - standard Type 1 Set Map / File/Directory Detail parsing, auto-detected Backup Exec XML catalogs, and a becatalog companion for vendor-specific payloads.
  • Media family identification - Family() combines the TAPE block and Set Map to tell you the family ID, total tape count, and which data sets live on which tapes. Essential for the "I have one tape, what do I need?" use case.

Quick start

r, err := mtf.Open("backup.bkf")
if err != nil { log.Fatal(err) }
defer r.Close()

for {
	b, err := r.Next()
	if err == io.EOF { break }
	if err != nil { log.Fatal(err) }

	switch b.Kind {
	case mtf.KindEntry:
		fmt.Println(b.Header.Name)
		if b.Header.Type == mtf.EntryFile {
			io.Copy(os.Stdout, r) // stream file content
		}
	case mtf.KindSetEnd:
		fmt.Println("data set ended; catalog:", b.Catalog != nil)
	}
}

Documentation

Full reference lives in docs/:

Command-line tools

These are small utilities built on the library, primarily for surveying archives:

Tool Purpose
cmd/bkfscan Parallel .bkf surveyor using Census.
cmd/bkfcensus Single-file Census reporter. 
bkfscan /mnt/archive/BEData          # survey

Project layout

go-mtf/
  mtf.go          public types & constants
  reader.go       the block iterator
  header.go       field accessors & offsets
  strings.go      MTF string decoding
  datetime.go     date/time decoding
  streams.go      data-stream materialization & sparse
  catalog.go      Media Based Catalog (standard)
  spanning.go     multi-media continuation
  compress.go     compression/encryption frames
  lzs.go          Stac LZS decompressor
  census.go       cartridge classification
  becatalog/      Backup Exec XML catalog parser
  cmd/            bkfscan, bkfcensus (survey utilities)
  docs/           detailed documentation

License

MIT. See LICENSE.

Documentation

Overview

Package mtf provides a reader for the Microsoft Tape Format (MTF) used by NTBACKUP.EXE (.bkf) files and LTO tapes.

Archives that span multiple physical media (tapes or .bkf files) are supported via Reader.SetContinuation, which supplies the next medium when an End Of Tape Marker (EOTM) is encountered. The callback receives a Continuation with the exhausted medium's name, sequence, and family ID, giving the application enough context to prompt an operator ("insert tape 2").

The primary entry point is the Reader. Reader.Next advances entry by entry and transparently parses each object's data streams, materializing the metadata a faithful extraction needs (NTFS security descriptors, extended attributes, sparse maps) into the returned Header and positioning file content for Reader.Read: 

r, err := mtf.Open("backup.bkf")
if err != nil { log.Fatal(err) }
defer r.Close()
for {
	b, err := r.Next()
	if err == io.EOF { break }
	if err != nil { log.Fatal(err) }
	if b.Kind == mtf.KindEntry {
		fmt.Println(b.Header.Name)
		if b.Header.Type == mtf.EntryFile {
			io.Copy(os.Stdout, r)
		}
	}
}

Index

Constants

View Source 
const (
	// AttrContinuation (MTF_CONTINUATION, BIT0) is set on descriptor blocks that
	// are repeated on a continuation medium to restore context after an End of
	// Media (EOTM). See MTF spec section 8 (End Of Media Processing).
	AttrContinuation uint32 = 0x00000001
	// AttrCompression (MTF_COMPRESSION, BIT2) indicates compression may be active.
	// Per MTF spec Table 3 the common attributes reserve BIT1 and define
	// compression at BIT2 (NTBackup/MTF SDK value 0x00000004).
	AttrCompression uint32 = 0x00000004
	// AttrEOSAtEOM (MTF_EOS_AT_EOM, BIT3) indicates End Of Medium was hit during
	// end-of-set processing.
	AttrEOSAtEOM uint32 = 0x00000008
)

Common Block Attributes (MTF_DB_HDR Block Attributes field, offset 4). These apply to the common header of any descriptor block.

View Source 
const (
	// MTFAttrReadOnly (READ_ONLY_BIT, BIT8) is set if the file/directory is
	// marked read-only.
	MTFAttrReadOnly uint32 = 0x00000100
	// MTFAttrHidden (HIDDEN_BIT, BIT9) is set if hidden from the user.
	MTFAttrHidden uint32 = 0x00000200
	// MTFAttrSystem (SYSTEM_BIT, BIT10) is set if it is a system file/directory.
	MTFAttrSystem uint32 = 0x00000400
	// MTFAttrModified (MODIFIED_BIT / "archive" flag, BIT11) is set if the
	// object has been modified.
	MTFAttrModified uint32 = 0x00000800
	// MTFAttrEmpty (DIRB_EMPTY_BIT, BIT16) is set if the directory contained no
	// files or subdirectories. (FILE_IN_USE_BIT for FILE entries.)
	MTFAttrEmpty uint32 = 0x00010000
	// MTFAttrPathInStream (DIRB_PATH_IN_STREAM_BIT / FILE_NAME_IN_STREAM_BIT,
	// BIT17) is set when the path/name is stored in a stream rather than the
	// DBLK String Storage Area.
	MTFAttrPathInStream uint32 = 0x00020000
	// MTFAttrCorrupt (CORRUPT_BIT, BIT18) is set if the associated data could
	// not be read.
	MTFAttrCorrupt uint32 = 0x00040000
)

MTF descriptor-block attributes (Table 13 DIRB / Table 14 FILE). These are the bits of the DBLK Attributes field that follows the common header. DIRB and FILE share the same bit positions for the flags below; the only difference is the BIT16 meaning (DIRB_EMPTY vs FILE_IN_USE). Test these against Header.Attributes.

View Source 
const (
	WinAttrReadOnly   uint32 = 0x00000001 // FILE_ATTRIBUTE_READONLY
	WinAttrHidden     uint32 = 0x00000002 // FILE_ATTRIBUTE_HIDDEN
	WinAttrSystem     uint32 = 0x00000004 // FILE_ATTRIBUTE_SYSTEM
	WinAttrDirectory  uint32 = 0x00000010 // FILE_ATTRIBUTE_DIRECTORY
	WinAttrArchive    uint32 = 0x00000020 // FILE_ATTRIBUTE_ARCHIVE
	WinAttrDevice     uint32 = 0x00000040 // FILE_ATTRIBUTE_DEVICE
	WinAttrNormal     uint32 = 0x00000080 // FILE_ATTRIBUTE_NORMAL
	WinAttrTemporary  uint32 = 0x00000100 // FILE_ATTRIBUTE_TEMPORARY
	WinAttrSparse     uint32 = 0x00000200 // FILE_ATTRIBUTE_SPARSE_FILE
	WinAttrReparse    uint32 = 0x00000400 // FILE_ATTRIBUTE_REPARSE_POINT
	WinAttrCompressed uint32 = 0x00000800 // FILE_ATTRIBUTE_COMPRESSED
	WinAttrOffline    uint32 = 0x00001000 // FILE_ATTRIBUTE_OFFLINE
	WinAttrEncrypted  uint32 = 0x00004000 // FILE_ATTRIBUTE_ENCRYPTED
)

Windows file attributes (dwFileAttributes). These are the standard Win32 FILE_ATTRIBUTE_* flags stored in the OS-specific data area of FILE/DIRB descriptor blocks for Windows NT entries (OS ID 14, spec Structures 42/43). They are NOT the DBLK Attributes field (Header.Attributes); test them against Header.WinAttributes.

View Source 
const (
	// NTFileLinkFlag (BIT0) is set when the file is a POSIX hard link. When
	// set, the data streams should contain exactly one LINK stream
	// (STRM_NTFS_LINK) pointing to the link target.
	NTFileLinkFlag uint32 = 0x00000001
	// NTFilePOSIX (BIT16) is set when the file is POSIX.
	NTFilePOSIX uint32 = 0x00010000
)

NT file flags from the OS-specific data section of a FILE DBLK (OS ID 14). These are stored in Header.NTFileFlags.

View Source 
const (
	ReparseTagSymlink uint32 = 0xA000000C // IO_REPARSE_TAG_SYMLINK
	ReparseTagMount   uint32 = 0xA0000003 // IO_REPARSE_TAG_MOUNT_POINT
)

Windows reparse tag values for NTRP streams. These identify the type of reparse point encoded in the Header.LinkTarget field.

View Source 
const (
	// StreamFSModifiedByRead (STREAM_MODIFIED_BY_READ, BIT0) indicates the data
	// changed after reading; verify operations should not be attempted.
	StreamFSModifiedByRead uint16 = 0x0001
	// StreamFSContainsSecurity (STREAM_CONTAINS_SECURITY, BIT1) indicates the
	// stream contains security information.
	StreamFSContainsSecurity uint16 = 0x0002
	// StreamFSNonPortable (STREAM_IS_NON_PORTABLE, BIT2) indicates the data can
	// only be restored to the same OS it was saved from.
	StreamFSNonPortable uint16 = 0x0004
	// StreamFSSparse (STREAM_IS_SPARSE, BIT3) marks a stream whose data is
	// sparse. See MTF spec section 6.1.
	StreamFSSparse uint16 = 0x0008
)

Stream File System Attributes (MTF_STREAM_HDR Stream File System Attributes field, stream header offset 4). See MTF spec Table 17.

View Source 
const (
	// StreamMediaContinue (STREAM_CONTINUE, BIT0) marks a stream whose data is a
	// continuation of a stream split across media at EOM. Its Stream Length holds
	// only the remaining (unwritten) portion and its data begins at the next
	// Format Logical Block boundary. See MTF spec section 6.1.
	StreamMediaContinue uint16 = 0x0001
	// StreamMediaVariable (STREAM_VARIABLE, BIT1) marks a stream whose data is
	// segmented into variable-length pieces (section 6.3). Each segment has this
	// bit set; the last segment additionally sets StreamMediaVarEnd.
	StreamMediaVariable uint16 = 0x0002
	// StreamMediaVarEnd (STREAM_VAR_END, BIT2) marks the last segment of a
	// variable-length stream (section 6.3). When set, this is the final piece.
	StreamMediaVarEnd uint16 = 0x0004
	// StreamMediaEncrypted (STREAM_ENCRYPTED, BIT3) marks an encrypted stream.
	StreamMediaEncrypted uint16 = 0x0008
	// StreamMediaCompressed (STREAM_COMPRESSED, BIT4) marks a compressed stream.
	StreamMediaCompressed uint16 = 0x0010
	// StreamMediaChecksumed (STREAM_CHECKSUMED, BIT5) marks a stream that is
	// followed by a CSUM checksum stream (section 6.2.1.4 / Figure 19).
	StreamMediaChecksumed uint16 = 0x0020
	// StreamMediaEmbeddedLength (STREAM_EMBEDDED_LENGTH, BIT6) is an obsolete
	// bit provided for backwards compatibility with pre-1.00a drafts (section 6.1).
	StreamMediaEmbeddedLength uint16 = 0x0040
)

Stream Media Format Attributes (MTF_STREAM_HDR Stream Media Format Attributes field, stream header offset 6). See MTF spec Table 18.

View Source 
const (
	StreamSTAN uint32 = 0x4E415453 // standard data
	StreamPNAM uint32 = 0x4D414E50 // path
	StreamFNAM uint32 = 0x4D414E46 // file name
	StreamCSUM uint32 = 0x4D555343 // checksum
	StreamCRPT uint32 = 0x54505243 // corrupt
	StreamSPAD uint32 = 0x44415053 // padding (marks the last stream of an object)
	StreamSPAR uint32 = 0x52415053 // sparse

	StreamTSMP uint32 = 0x504D5354 // set map, media based catalog, type 1
	StreamTFDD uint32 = 0x44444654 // fdd, media based catalog, type 1
	StreamMAP2 uint32 = 0x3250414D // set map, media based catalog, type 2
	StreamFDD2 uint32 = 0x32444446 // fdd, media based catalog, type 2
	StreamSM2P uint32 = 0x32504D53 // 'SMP2' — Backup Exec Set Map variant (see docs/catalog.md). Kept under the SM2P name for compatibility; StreamSMP2 aliases it.
	StreamSMP2        = StreamSM2P

	StreamADAT uint32 = 0x54414441 // NT data
	StreamNTEA uint32 = 0x4145544E // NT extended attributes
	StreamNACL uint32 = 0x4C43414E // NT ACL
	StreamNTRP uint32 = 0x5052544E // NT reparse point (symlinks, mount points)
	StreamLINK uint32 = 0x4B4E494C // NT hard link target
	StreamNTED uint32 = 0x4445544E // NT EData
	StreamNTQU uint32 = 0x5551544E // NT quota
	StreamNTPR uint32 = 0x5250544E // NT property
	StreamNTOI uint32 = 0x494F544E // NT object id
	StreamNTQP uint32 = 0x5051544E // NT quota/property (vendor variant)

	StreamGERC uint32 = 0x43524547 // Win9x

	StreamN386 uint32 = 0x3638334E // Netware
	StreamNBND uint32 = 0x444E424E // Netware
	StreamSMSD uint32 = 0x44534D53 // Netware

	StreamOACL uint32 = 0x4C43414F // OS/2 ACL
	StreamO2EA uint32 = 0x4145324F // OS/2 EA (spec Table 22)

	StreamMRSC uint32 = 0x4353524D // Macintosh resource
	StreamMPRV uint32 = 0x5652504D // Macintosh private
	StreamMINF uint32 = 0x464E494D // Macintosh info
)

Stream data type identifiers. These are the four-byte stream type codes read as little-endian uint32 values (e.g. "STAN" -> 0x4E415453). They are exported so callers can interpret the stream types associated with an entry.

View Source 
const (

	// MTF_LZS221: the single registered software compression algorithm (Appendix C).
	AlgLZS221 = uint16(0x0ABE)
)
View Source 
const FDDCommonHeaderSize = fddHdrSize

FDDCommonHeaderSize is the byte size of the MTF_FDD_HDR common header that begins every FDD/Set-Map volume record.

View Source 
const FDDRecordLenOff = fddLenOff

FDDRecordLenOff is the offset of the UINT16 LENGTH field within an FDD common header (a volume record's own size).

View Source 
const FDDRecordStrTypeOff = fddStrTypeOff

FDDRecordStrTypeOff is the offset of the UINT8 STRING_TYPE field within an FDD common header.

View Source 
const SetMapHeaderSize = smHdrSize

SetMapHeaderSize is the byte size of the Set Map header (MFMID + entry count) that precedes the Set Map Entries in every Set Map stream payload.

View Source 
const (
	// VOLBNTDRCandidate (NT_VOLB_IS_DR_CANDIDATE, BIT0) is set when the data
	// following the VOLB is suitable for an NT system recovery.
	VOLBNTDRCandidate uint32 = 0x00000001
)

NT VOLB OS-specific flags (spec Structure 41 / Table 27). The NT Backup Set Attributes field sits at offset 4 of a volume entry's OS-specific data area.

Variables

View Source 
var ErrEncrypted = errors.New("mtf: stream is encrypted; register a decryptor with SetDecryptor")

ErrEncrypted is returned when a stream is encrypted but no decryptor has been registered with Reader.SetDecryptor.

View Source 
var ErrFilemark = errors.New("mtf: filemark")

ErrFilemark is returned by Tape.ReadBlock when the drive reports a filemark with no preceding data: the current recorded section has ended and the tape is positioned at the first block of the next section. A single filemark is a section delimiter, not end of data — the Reader skips it and keeps reading. Two filemarks in a row (no data between) mean end of recorded data and are reported as io.EOF.

This mirrors the Linux st driver contract: read() returns zero bytes at a filemark. Surfacing it as an explicit sentinel instead of hiding it behind io.EOF lets the Reader tell "section boundary" (keep going) apart from "no more data" (stop).

Functions

func Checksum32 added in v0.5.0 

func Checksum32(b []byte) uint32

Checksum32 returns the MTF data-stream checksum: a 32-bit XOR of the linear stream data (spec Figure 19 / section 6.2.1.4). The algorithm is independent of how the data is segmented: the bytes are consumed four at a time as little-endian 32-bit words and XOR-accumulated, with any trailing 1-3 bytes XORed into the low bytes of the accumulator. This yields the same value whether the data is chunked into 1, 2, 3 or 4-byte pieces. It is the value a Checksum Stream ('CSUM') carries for the preceding data stream.

func FormatSID 

func FormatSID(sid []byte) string

FormatSID converts a raw binary SID to its string representation (e.g. "S-1-5-21-..."). It returns an empty string if the SID is invalid.

func NewFileTape added in v0.12.0 

func NewFileTape(f *os.File) *fileTape

func ParseSetMapHeader added in v0.17.0 

func ParseSetMapHeader(raw []byte) (mediaFamilyID uint32, count int)

ParseSetMapHeader decodes a Set Map stream payload header, returning the Media Family ID and the declared Number-Of-Set-Map-Entries. It is exported so proprietary Set Map parsers (registered via RegisterSetMapParser) can share the header convention.

func ReadSetMapRaw added in v0.22.0 

func ReadSetMapRaw(tape Tape) (streamID uint32, payload []byte, err error)

ReadSetMap returns the cumulative Media Based Catalog Set Map from the final data set on the medium, using the spec's catalog-directed access path (§3.3.2.2 / §5.2.9 / §7.3.1): the trailing MTF_EOTM block stores the PBA of the last MTF_ESET; the Set Map stream is physical-block-aligned and is the last catalog stream preceding that ESET (Figure 24). This lets a caller enumerate every data set (name, SSETPBA, FLA, file count, size) in a handful of block reads near end-of-media instead of walking every file forward.

If the tape implements EOM() (e.g. DriveTape), ReadSetMap positions at end-of-recorded media first; otherwise the caller must have positioned it there. The tape's final position is unspecified on return — callers should Rewind/SeekBlock before further sequential use.

Returns (nil, nil) if no Set Map is present — e.g. a data-only continuation cartridge (the EOTM's MTF_NO_ESET_PBA / MTF_INVALID_ESET_PBA attribute bits are set, the medium has no EOTM, or no Set Map stream header is found in the trailing blocks). Callers should fall back to a forward walk (Reader.Next) in that case. ReadSetMapRaw locates and reads the raw Set Map stream payload (the bytes after the 22-byte stream header) from a tape's Media Based Catalog, returning the stream ID and payload. It is the building block of ReadSetMap; callers that want to inspect the raw layout (e.g. to verify field offsets against vendor media) can parse the payload themselves.

func RegisterSetMapParser added in v0.17.0 

func RegisterSetMapParser(streamID uint32, p SetMapParser)

RegisterSetMapParser registers a parser for a proprietary Set Map stream ID (e.g. the Backup Exec 'SMP2' variant, registered by the becatalog/besetmap subpackage). It is intended to be called from a package init() so that importing the package (blank import) opts in: the main mtf package itself contains no vendor-specific parsing. Registering an ID replaces any prior registration for that ID; it is not safe to call concurrently with parse.

func StreamTypeName 

func StreamTypeName(t uint32) string

StreamTypeName returns a human-readable name for a stream data type identifier.

Types

type Block 

type Block struct {
	Kind    BlockKind
	Tape    *TapeInfo // populated when Kind == KindMedia
	Set     *SetInfo  // populated when Kind == KindSet
	Header  *Header   // populated when Kind == KindEntry
	ESet    *ESetInfo // populated when Kind == KindSetEnd
	Catalog *Catalog  // populated when Kind == KindSetEnd (nil if the set had no catalog)
}

Block is a single structural element yielded by Reader.Next. Its Kind discriminates which field is populated. A single Block (and Header) are reused across calls to Next, so all fields are overwritten on the next call; callers should copy any values they need to retain.

type BlockKind 

type BlockKind uint8

BlockKind identifies the kind of MTF structure a Block represents. 

const (
	// KindMedia is yielded for an MTF_TAPE descriptor block: the start of a
	// physical medium. With media spanning enabled, one iteration yields a
	// KindMedia block for each medium as it is consumed.
	KindMedia BlockKind = iota
	// KindSet is yielded for an MTF_SSET descriptor block: the start of a data
	// set (one backup operation).
	KindSet
	// KindEntry is yielded for an extractable object descriptor (MTF_VOLB,
	// MTF_DIRB or MTF_FILE). The header is fully materialized; call [Reader.Read]
	// to stream the entry's standard data.
	KindEntry
	// KindSetEnd is yielded for an MTF_ESET descriptor block: a data set ended.
	// Any Media Based Catalog carried by the set's streams is attached as
	// Catalog (nil when the set recorded no catalog).
	KindSetEnd
)

type CartridgeRole 

type CartridgeRole int

CartridgeRole classifies the role a medium plays within its Media Family, inferred from the catalog type recorded on its TAPE block. 

const (
	// RoleUnknown is used when no TAPE block was seen or the catalog type is
	// unrecognized.
	RoleUnknown CartridgeRole = iota
	// RoleData is a cartridge that primarily carries file data
	// (Backup Exec CatalogType 64).
	RoleData
	// RoleCatalog is a consolidated catalog cartridge
	// (Backup Exec CatalogType 128).
	RoleCatalog
)

type Catalog 

type Catalog struct {
	// SetMap is the cumulative Media Family summary parsed from the 'TSMP'
	// stream. It lists one entry per data set in the family. It may be nil when
	// no standard Set Map stream was present.
	SetMap *SetMap
	// FDD is the per-data-set File/Directory Detail parsed from the 'TFDD'
	// stream: every volume, directory and file, in archive order, each carrying
	// the location (MediaSeq + FLA) of its descriptor block. It is empty when no
	// standard FDD stream was present or when the FDD payload is a Backup Exec
	// XML catalog (see BECatalog).
	FDD []CatalogEntry
	// BECatalog holds the parsed Backup Exec catalog, auto-detected from the
	// FDD stream. It is nil when the FDD payload is a standard MTF binary
	// catalog (in which case FDD is populated) or when no FDD stream was present.
	BECatalog *becatalog.Catalog
	// RawFDD is the unparsed payload of the FDD data stream. It is populated
	// whenever an FDD stream was captured, including vendor-specific payloads
	// the standard parser does not understand (so a vendor parser can take over).
	RawFDD []byte
	// RawSetMap is the unparsed payload of the Set Map data stream, populated
	// whenever a Set Map stream was captured.
	RawSetMap []byte
}

Catalog holds the parsed Media Based Catalog of the most recently completed data set, available via Reader.Catalog. It is nil when no MBC streams were present on the medium, or when the catalog was written in an unrecognized (vendor-specific) format that left no parseable standard records.

func (*Catalog) Raw 

func (c *Catalog) Raw() CatalogRaw

Raw returns the captured catalog stream payloads, implementing CatalogData. A vendor-specific catalog parser consumes CatalogRaw.FDD directly.

type CatalogData 

type CatalogData interface {
	// Raw returns the captured catalog stream payloads. FDD is the
	// File/Directory Detail ('TFDD'/'FDD2') payload and SetMap is the Set Map
	// ('TSMP'/'MAP2') payload; either may be nil if the stream was absent. For a
	// standard Type 1 catalog these are binary records; a writer may substitute a
	// vendor-specific payload (so a vendor parser takes over from Raw.FDD).
	Raw() CatalogRaw
}

CatalogData exposes the raw, uninterpreted catalog stream payloads captured from a data set's ESET block. It decouples vendor-specific catalog parsers (for example a Backup Exec XML parser) from this package's concrete Catalog type: a vendor parser accepts a CatalogData and works purely from the bytes.

Catalog satisfies CatalogData via its Catalog.Raw method.

type CatalogEntry 

type CatalogEntry struct {
	// Type is the kind of object.
	Type CatalogEntryType
	// MediaSeq is the MEDIA_SEQ_NUMBER: the 1-based sequence of the medium
	// within the Media Family that holds the object's descriptor block.
	MediaSeq uint16
	// FLA is the Format Logical Address of the object's descriptor block, the
	// byte offset within the medium at which the DBLK is written. Together with
	// MediaSeq it locates the object for random-access extraction.
	FLA uint64
	// Size is the DISPLAYABLE_SIZE copied from the descriptor block.
	Size uint64
	// Attributes is the type-specific attribute word (VOLB/DIRB/FILE
	// attributes) copied from the descriptor block.
	Attributes uint32
	// BlockAttributes is the MTF_DB_HDR common block attributes word.
	BlockAttributes uint32
	// Link is the FDD LINK field: for a directory, the offset of the next
	// sibling directory within the FDD; for a file, the stream offset of its
	// parent directory; for a volume, the offset of the next volume entry.
	Link int32
	// Name is the object name (directory/file name) or, for a volume, the
	// device name.
	Name string
	// VolumeLabel is the volume label (volume entries only).
	VolumeLabel string
	// MachineName is the source machine name (volume entries only). In a Set
	// Map it identifies the host that owns the data set.
	MachineName string
	// WriteTime is the media write date of a volume entry.
	WriteTime time.Time
	// ModTime, CreateTime, BackupTime and AccessTime are the object timestamps
	// copied from the descriptor block (directory and file entries).
	ModTime, CreateTime, BackupTime, AccessTime time.Time
}

CatalogEntry describes one object (volume, directory or file) recorded in the File/Directory Detail. Each entry carries the location of its descriptor block on medium, allowing a reader to seek directly to the object.

func ParseFDDVolume added in v0.17.0 

func ParseFDDVolume(rec []byte, strType uint8) *CatalogEntry

ParseFDDVolume decodes one FDD volume record (MTF_FDD_VOLB) from rec using the given string type. Exported for proprietary Set Map parsers that read volume records individually.

type CatalogEntryType 

type CatalogEntryType int

CatalogEntryType identifies the kind of object a CatalogEntry records. 

const (
	// EntryCatalogVolume is an FDD volume entry (corresponds to a VOLB DBLK).
	EntryCatalogVolume CatalogEntryType = iota
	// EntryCatalogDirectory is an FDD directory entry (corresponds to a DIRB
	// DBLK).
	EntryCatalogDirectory
	// EntryCatalogFile is an FDD file entry (corresponds to a FILE DBLK).
	EntryCatalogFile
)

type CatalogRaw 

type CatalogRaw struct {
	FDD    []byte
	SetMap []byte
}

CatalogRaw holds the raw, uninterpreted catalog stream payloads.

type Census 

type Census struct {
	// Tape is the metadata from the cartridge's TAPE block. It is nil when the
	// stream did not begin with a TAPE block (for example a bare continuation).
	Tape *TapeInfo
	// Set is the metadata from the data-set start (SSET) block, or nil.
	Set *SetInfo
	// Role is the inferred cartridge role.
	Role CartridgeRole
	// CatalogType is the catalog type recorded on the TAPE block
	// (0 when absent).
	CatalogType uint16
	// MediaSequence is the media sequence number (1-based within a family).
	// Derived from the TAPE Sequence field; a standalone cartridge reports 1.
	MediaSequence uint16
	// SetsClosed is the number of ESET blocks seen (data sets that end on this
	// cartridge). A mid-span continuation with no data-set end reports 0.
	SetsClosed int
	// HasCatalog reports whether any end-of-set block carried catalog streams
	// (Media Based Catalog: TFDD/FDD2/TSMP/MAP2).
	HasCatalog bool
	// CatalogBytes is the total catalog payload captured (TFDD + FDD2 +
	// SetMap), regardless of whether a standard parser could decode it.
	CatalogBytes int64

	// Volumes is the number of volume (VOLB) entries.
	Volumes int
	// Directories is the number of directory (DIRB) entries.
	Directories int
	// Files is the number of file (FILE) entries, including empty ones.
	Files int
	// EmptyFiles is the number of file entries with no data stream.
	EmptyFiles int
	// FileBytes is the sum of every file's on-media (stored) data size. For
	// uncompressed cartridges this equals the logical size; for compressed or
	// sparse ones it reflects the stored byte count.
	FileBytes int64
	// SparseFiles is the number of sparse file entries.
	SparseFiles int
	// CompressedFiles is the number of file entries whose data stream is
	// compressed.
	CompressedFiles int
	// EncryptedFiles is the number of file entries whose data stream is
	// encrypted.
	EncryptedFiles int
}

Census summarizes the shape of a single cartridge (one MTF stream, i.e. one .bkf file or one tape) at a glance, without reading file content. It is produced by Reader.Census, which walks every block but drains file data streams instead of delivering their bytes, so it is suitable for cheaply classifying large archives.

Census reads every block header and stream descriptor, so its file/byte counts are authoritative for the cartridge; only the content bytes are not materialized.

func (*Census) HasData 

func (c *Census) HasData() bool

HasData reports whether the cartridge carries any file content.

type Continuation 

type Continuation struct {
	// Sequence is the 1-based index of the medium that just ended within the
	// media family. The next medium to load therefore has Sequence+1.
	Sequence int
	// Media holds the descriptor of the exhausted medium, or nil if no TAPE
	// block was parsed for it.
	Media *TapeInfo
}

type Decryptor 

type Decryptor func(algo uint16, encrypted []byte) ([]byte, error)

Decryptor reverses the encryption of one frame's payload. algo is the stream's Data Encryption Algorithm ID (from the STAN header); encrypted holds the raw encrypted bytes. It must return the plaintext of exactly those bytes. The MTF spec defines no cipher, so the implementation is vendor-specific.

type DriveTape added in v0.13.0 

type DriveTape struct {
	// contains filtered or unexported fields
}

DriveTape adapts a tapedrive.Drive (the Linux st driver binding) to the Tape interface. It is the bridge from a real LTO/SCSI tape drive to a Reader.

The two libraries use opposite sentinel conventions for the two tape boundaries, so DriveTape translates them: 

go-tapedrive io.EOF        (a single filemark) -> mtf ErrFilemark
go-tapedrive ErrEndOfData  (two filemarks/EOD) -> mtf io.EOF

This keeps the mtf.Reader's contract intact — it skips ErrFilemark between recorded sections and stops cleanly at io.EOF — while go-tapedrive stays a pure driver binding with no mtf knowledge.

func NewDriveTape added in v0.13.0 

func NewDriveTape(d *tapedrive.Drive) *DriveTape

NewDriveTape wraps an open tapedrive.Drive. The caller transfers ownership: the tape's Close (and therefore Reader.Close) closes the underlying drive.

func (*DriveTape) Close added in v0.13.0 

func (t *DriveTape) Close() error

Close closes the underlying drive, satisfying io.Closer so Reader.Close releases the device.

func (*DriveTape) EOM added in v0.15.0 

func (t *DriveTape) EOM() error

EOM positions the drive at end-of-recorded media. Used by ReadSetMap to reach the trailing MTF_EOTM before reading the Media Based Catalog.

func (*DriveTape) ReadBlock added in v0.13.0 

func (t *DriveTape) ReadBlock(dst []byte) (int, error)

ReadBlock implements Tape. It delegates to tapedrive.Drive.ReadBlockInto and translates the boundary errors to mtf's convention. A data block is returned as (n, nil); a filemark as (0, ErrFilemark); end of recorded data as (0, io.EOF). The caller's dst must be sized for the drive's largest record (use maxTapeBlock).

func (*DriveTape) Rewind added in v0.15.0 

func (t *DriveTape) Rewind() error

Rewind rewinds the drive to beginning of tape.

func (*DriveTape) SeekBlock added in v0.13.0 

func (t *DriveTape) SeekBlock(block int64) error

SeekBlock implements Tape, positioning the drive at a physical block address via MTSEEK. The block number is the same device-level PBA that tapedrive.Drive.TellBlock reports and that MTF stores in MTF_SSET.

func (*DriveTape) TellBlock added in v0.13.0 

func (t *DriveTape) TellBlock() (int64, error)

TellBlock implements Tape, returning the drive's current physical block address via MTIOCPOS. This is the live PBA the Reader captures when it reads the MTF_SSET DBLK, anchoring the §3.4.3 FLA→PBA seek calculation.

type ESetInfo 

type ESetInfo struct {
	Attributes       uint32 // ESET block attributes
	CorruptObjects   uint32 // number of corrupt files in the data set
	FDDMediaSequence uint16 // FDD media sequence number
	SetNumber        uint16 // data-set number being closed
	CreateTime       time.Time
}

ESetInfo holds metadata from the most recent end-of-data-set (ESET) block. It is available via Reader.ESet after a data set has ended.

type EntryType 

type EntryType int

EntryType describes the kind of entry a Header represents. 

const (
	// EntryFile is a regular file. Its contents are available via [Reader.Read].
	EntryFile EntryType = iota
	// EntryDirectory is a directory entry.
	EntryDirectory
	// EntryVolume is a source volume (device) the backup was taken from.
	EntryVolume
)
type Header struct {
	// Type is the kind of entry.
	Type EntryType
	// Name is the fully resolved path of the entry, formed from the source
	// volume, directory chain and the entry name using "/" separators.
	Name string
	// ModTime is the modification time of the entry.
	ModTime time.Time
	// AccessTime is the last access time, if recorded.
	AccessTime time.Time
	// CreateTime is the creation time, if recorded.
	CreateTime time.Time
	// BirthTime is the birth time of the entry, if recorded. Only emitted by
	// NT-based backups that populate the MTF birth-time field.
	BirthTime time.Time
	// Attributes is the MTF descriptor-block (DBLK) attributes field that
	// follows the common header — Table 13 (DIRB) / Table 14 (FILE). It uses
	// the MTF bit layout (READ_ONLY=BIT8, HIDDEN=BIT9, SYSTEM=BIT10,
	// MODIFIED=BIT11, EMPTY/IN_USE=BIT16, PATH/NAME_IN_STREAM=BIT17,
	// CORRUPT=BIT18), NOT the Win32 layout. Test individual bits with the
	// [MTFAttr] constants. For the Windows FILE_ATTRIBUTE_* values use
	// [Header.WinAttributes].
	Attributes uint32
	// WinAttributes is the Windows dwFileAttributes value (the
	// FILE_ATTRIBUTE_* flags from the WIN32_FIND_DATA structure) loaded from
	// the OS-specific data area of the DBLK for Windows NT entries (OS ID 14,
	// spec Structures 42/43, offset 0). It is zero for non-NT entries or when
	// the OS-specific data is absent. Test individual bits with the [WinAttr]
	// constants; this is the source [Header.UnixMode] derives its mode from.
	WinAttributes uint32
	// NTFileFlags is the Windows NT file-specific flags from the OS-specific
	// data section of a FILE DBLK (OS ID 14, OS Version 1, spec Structure 43
	// offset 8). It is zero for non-NT entries, for OS Version 0 (Structure 40,
	// where offset 8 is Reserved) and for non-FILE blocks (DIRB Structure 42 has
	// no such field). Test individual bits with the [NTFile] constants.
	NTFileFlags uint32
	// BlockAttributes is the MTF_DB_HDR common Block Attributes field, which
	// carries bits such as MTF_CONTINUATION and MTF_COMPRESSION. Use the
	// Attr* constants to test it.
	BlockAttributes uint32
	// OSID is the operating system identifier recorded in the block.
	OSID uint8
	// SetNumber is the backup data-set number this entry belongs to.
	SetNumber uint16
	// Volume is the name of the source volume/device the entry resides on.
	Volume string
	// VolumeLabel is the source volume's label, if recorded (volume entries).
	VolumeLabel string
	// MachineName is the name of the source machine, if recorded (volume
	// entries).
	MachineName string
	// FileSystemFlags is the Win32 file-system flags (lpFileSystemFlags from
	// GetVolumeInformation) recorded in the OS-specific data area of a Windows
	// NT volume entry (OS ID 14, spec Structure 41, offset 0). It is zero for
	// non-volume entries or when the OS-specific data is absent.
	FileSystemFlags uint32
	// IsDRCandidate reports whether the volume's data is suitable for an NT
	// system recovery, per the NT_VOLB_IS_DR_CANDIDATE bit (spec Structure 41 /
	// Table 27, BIT0 of the NT Backup Set Attributes at OS-data offset 4). It
	// is meaningful only for volume (VOLB) entries of Windows NT origin.
	IsDRCandidate bool
	// FileID is the MTF object identifier of the file (files only).
	FileID uint32
	// DirID is the identifier of the directory containing the entry.
	DirID uint32
	// LinkTarget is the target path for symbolic links and hard links.
	// It is populated from the NTRP (reparse) stream for symlinks and the
	// LINK stream for hard links. It is empty for regular entries.
	LinkTarget string
	// IsSymlink reports whether this entry is a symbolic link. Set when an
	// NTRP stream with reparse tag IO_REPARSE_TAG_SYMLINK is present.
	IsSymlink bool
	// IsHardLink reports whether this entry is a POSIX hard link. Set when
	// the NT file flags [NTFileLinkFlag] bit is set or a LINK stream is
	// present.
	IsHardLink bool

	// The following describe the file's standard (STAN) data stream and are
	// meaningful for file entries. They are populated by [Reader.Next].
	//
	// Size is the logical length in bytes of the file's content as delivered by
	// [Reader.Read]. For a plain file this is the STAN stream length; for a
	// sparse file it is the reconstructed (hole-filled) length; for a
	// compressed/encrypted file it is the on-media (stored) byte count (see
	// [Header.DisplayableSize] for the logical size in that case). It is zero
	// for non-file entries.
	Size int64
	// CompressionAlgorithm is the registered ID of the algorithm used to
	// compress the standard data stream, or zero if uncompressed.
	CompressionAlgorithm uint16
	// EncryptionAlgorithm is the registered ID of the algorithm used to
	// encrypt the standard data stream, or zero if unencrypted.
	EncryptionAlgorithm uint16
	// Compressed reports whether the standard data stream is compressed. The
	// bytes returned by [Reader.Read] are still compressed; decompression is
	// not performed.
	Compressed bool
	// Encrypted reports whether the standard data stream is encrypted. The
	// bytes returned by [Reader.Read] are still encrypted; decryption is not
	// performed.
	Encrypted bool
	// Sparse reports whether the file is sparse. For sparse files [Reader.Read]
	// transparently reconstructs the logical content (holes are zero-filled)
	// and [Header.SparseExtents] holds the parsed sparse map. The STREAM_IS_SPARSE
	// bit is documented in MTF spec section 6.1.
	Sparse bool
	// StreamChecksum is the checksum field of the standard (STAN) data stream
	// header (zero unless the stream is checksummed).
	StreamChecksum uint16
	// DisplayableSize is the object size recorded in the common descriptor
	// block's Displayable Size field. For uncompressed files it equals Size;
	// for compressed or sparse objects it reflects the logical (expanded) size.
	DisplayableSize uint64

	// SecurityDescriptor holds the raw NTFS security descriptor (NACL stream)
	// associated with the entry, if any. It is a self-relative security
	// descriptor as produced by the Win32 BackupRead API. Present on both file
	// and directory entries.
	SecurityDescriptor []byte
	// ExtendedAttributes holds the raw NT extended-attribute data (NTEA stream)
	// associated with the entry, if any.
	ExtendedAttributes []byte
	// SparseExtents describes the sparse layout of a sparse file (one entry per
	// SPAR stream), or nil for a non-sparse entry. Each extent carries the
	// non-hole bytes located at [SparseExtent.Offset] in the logical file;
	// [Reader.Read] fills the gaps with zero bytes. See MTF spec section 6.2.1.7.
	SparseExtents []SparseExtent
	// Streams holds every data stream associated with the entry that is not
	// carried by a named field above (e.g. NTOI object ids, NTQU quota, CSUM
	// checksums, ADAT alternate data, or vendor-specific streams). Each element
	// preserves the stream's four-byte type code and raw bytes so no metadata
	// is lost. Streams both preceding and following the standard (STAN) data
	// stream are captured. It is nil in [Reader.HeaderOnly] mode, which skips
	// stream data, and for entries that have no extra streams.
	Streams []StreamData
}

Header describes a single entry (file, directory or volume) within an MTF archive. It is returned by Reader.Next via Block.Header. A single Header is reused across entries, so its fields are overwritten on the next call to Next; callers that retain an entry across iterations must copy the fields they need.

func (*Header) GroupSID 

func (h *Header) GroupSID() []byte

GroupSID returns the group SID from the security descriptor stored in Header.SecurityDescriptor, or nil if no descriptor is present or it is too short to contain a group.

func (*Header) OwnerSID 

func (h *Header) OwnerSID() []byte

OwnerSID returns the owner SID from the security descriptor stored in Header.SecurityDescriptor, or nil if no descriptor is present or it is too short to contain an owner. The SID is returned in raw binary form (self-relative SECURITY_DESCRIPTOR format).

To convert a raw SID to string form (S-1-5-21-...), use FormatSID.

func (*Header) UnixMode 

func (h *Header) UnixMode() uint32

UnixMode returns a best-effort Unix permission mode derived from the MTF descriptor-block attributes stored in Header.Attributes (Tables 13/14, which are always present and use the MTF bit layout). The mapping follows common conventions:

  • Directories get mode 0755 (readable/traversable by all, writable by owner).
  • Regular files get mode 0644 (readable by all, writable by owner), or 0755 if the modified ("archive") bit is set (a heuristic for executables on Unix-origin backups).
  • The read-only bit clears the owner-write bit.
  • The system bit is not mapped (Unix has no equivalent).

The MTF attributes are used because they are spec-defined for every archive regardless of OS version, whereas Header.WinAttributes (the OS-specific dwFileAttributes) is only populated for NT OS version 0/1. For precise permissions, parse Header.SecurityDescriptor and extract the DACL.

type MediaFamily 

type MediaFamily struct {
	// ID is the Media Family ID (MFMID) from the TAPE block. All cartridges in
	// the same family share this ID, making it the key for grouping scattered
	// tapes into a restore set.
	ID uint32
	// TotalTapes is the total number of cartridges in this family, derived from
	// the Set Map (the maximum MediaSeq across all data-set entries). It is 0 when
	// no Set Map was present — for example a data-only cartridge that is not the
	// last in the family won't carry a Set Map.
	TotalTapes int
	// TapeSequence is this cartridge's 1-based position in the family, matching
	// [TapeInfo.Sequence].
	TapeSequence int
	// TapeName is the name/label of this cartridge.
	TapeName string
	// SetMap is the parsed Set Map from this cartridge, or nil if none was
	// present. Each [SetMapEntry] carries a [SetMapEntry.MediaSeq] that identifies
	// which tape the data set starts on; scanning them reveals the full set of
	// tapes in the family.
	SetMap *SetMap
}

MediaFamily summarizes what is known about the media family from a single cartridge. It combines the TAPE descriptor and the Set Map (if present) to answer questions like "which media family is this?" and "how many tapes do I need for a full restore?"

Call Reader.Family to obtain one.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

func NewReader 

func NewReader(t Tape) *Reader

NewReader returns a new Reader reading from the given Tape. By default the Reader reads stream data sequentially when skipping past a file's content (SkipNever) — the spec-correct mode for full scans and bulk migration, since PBAs are sequential between filemarks (§3.4.1) and tape streams at native rate. Callers doing catalog-driven selective restore (§3.4.3) should seek the source to the target object directly rather than scanning.

func Open 

func Open(name string) (*Reader, error)

Open opens the named MTF/BKF file for reading.

func (*Reader) Catalog 

func (r *Reader) Catalog() *Catalog

Catalog returns the Media Based Catalog of the most recently completed data set, or nil if no MBC streams were captured. It is meaningful once Reader.Next has advanced past the data set's end (its first ESET block) or reached end-of-archive.

The catalog is parsed lazily on first call and cached.

func (*Reader) Census 

func (r *Reader) Census() (Census, error)

Census walks the cartridge, classifying it into a Census (returned by value, so it need not heap-allocate). File content is discarded rather than returned. It consumes the reader: after it returns the stream is exhausted (or stopped at the first read error). Use a fresh reader to extract content.

Census runs in header-only mode, so it performs zero per-entry allocations; the only allocations are the Reader itself and its block buffer.

Even when err is non-nil, the returned Census is populated with whatever was read before the error; this lets a caller classify partially readable cartridges.

func (*Reader) Checksum 

func (r *Reader) Checksum() (stored, computed uint16)

Checksum returns the MTF common-block header checksum field of the current block together with the value recomputed from the remaining header fields. Equal values indicate an intact header.

func (*Reader) Close 

func (r *Reader) Close() error

Close closes the underlying reader if it was opened by Open.

func (*Reader) CorruptObjects 

func (r *Reader) CorruptObjects() uint32

CorruptObjects returns the corrupt-object count reported by the most recent end-of-data-set (ESET) block, or zero if no set has ended yet.

func (*Reader) ESet 

func (r *Reader) ESet() *ESetInfo

ESet returns metadata from the most recent end-of-data-set (ESET) block, or nil if no data set has ended yet. The returned value is shared; callers must not retain it across subsequent calls to Reader.Next.

func (*Reader) Family 

func (r *Reader) Family() MediaFamily

Family returns what is known about the media family from the current cartridge. It combines the TAPE descriptor and the Set Map (if present) to answer questions like "which media family is this tape?" and "how many tapes do I need for a full restore?"

Call after the first KindMedia block has been processed (which fills TapeInfo) and, for the most complete family information, after the first KindSetEnd (which fills the Set Map).

The Set Map is cumulative — the one on the last cartridge of the family is the most complete. On a data-only cartridge (CatalogType 64) the Set Map may be nil; on a catalog cartridge (CatalogType 128) it is typically present.

func (*Reader) HeaderOnly 

func (r *Reader) HeaderOnly()

HeaderOnly puts the reader into header-only mode: metadata stream *data* (NTFS security descriptors, extended attributes, sparse maps) is skipped rather than read into Header fields, and standard (STAN) file content is never delivered. Entry *names* are also skipped (Header.Name is left empty) since string construction is the dominant per-entry allocation; block and stream *headers* are still parsed, so sizes and flags remain accurate. Combined with a seekable source this lets a caller walk a cartridge reading essentially only headers, with zero per-entry allocations. It is used by Reader.Census.

func (*Reader) MediaSequence 

func (r *Reader) MediaSequence() int

MediaSequence reports the 1-based index of the current physical medium. It is 1 for the initial medium and increments each time a continuation medium supplied via Reader.SetContinuation is switched to. It is always 1 for non-spanned archives.

func (*Reader) Next 

func (r *Reader) Next() (*Block, error)

Next advances through the MTF stream and returns the next structural block. The returned Block.Kind tells you what was encountered:

  • KindMedia: a medium (MTF_TAPE) started. Block.Tape holds its metadata.
  • KindSet: a data set (MTF_SSET) started. Block.Set holds its metadata.
  • KindEntry: an extractable object (MTF_VOLB/DIRB/FILE). Block.Header is fully materialized; call Reader.Read to stream its standard data.
  • KindSetEnd: a data set (MTF_ESET) ended. Block.ESet holds its metadata and Block.Catalog carries any Media Based Catalog (nil if none).

The medium's role is self-evident from the block sequence: a medium with KindEntry blocks but no trailing KindSetEnd is data-only (its data set continues on the next medium); one whose KindSetEnd carries a Catalog with no file-data entries is catalog-heavy; one with both is the normal case.

Media spanning is handled transparently: when a continuation is registered via Reader.SetContinuation, each consumed medium yields its own KindMedia. Next returns io.EOF when the archive is fully consumed.

func (*Reader) Position 

func (r *Reader) Position() int64

Position reports the byte offset already consumed from the underlying stream. It is intended for diagnostics and for building seek indexes for random access extraction.

func (*Reader) Read 

func (r *Reader) Read(p []byte) (int, error)

Read reads data from the current file entry into p. It returns the reconstructed file content: for a plain file this is the standard data (STAN) stream, transparently followed across continuation media; for a sparse file the holes are zero-filled according to the parsed sparse map.

Decompression and decryption are not performed: for a compressed or encrypted stream the raw stored bytes are returned.

Read returns io.EOF when the entry's content is exhausted. Calling Read on a non-file entry returns io.EOF immediately.

func (*Reader) Set 

func (r *Reader) Set() *SetInfo

Set returns metadata from the current (most recent) start-of-data-set block, or nil if none has been encountered yet.

func (*Reader) SetContinuation 

func (r *Reader) SetContinuation(next func(Continuation) (Tape, error))

SetContinuation registers a function that supplies the next physical medium when the current one ends (an MTF_EOTM block is reached).

This enables reading a single backup data set that spans multiple media (tapes or .bkf files). When the reader encounters an End Of Tape Marker (EOTM) — whether between entries or in the middle of a file's data stream — it calls next with a Continuation describing the medium that just ended, and resumes from the reader next returns.

The callback is the natural place to prompt an operator to load the next tape, e.g.: 

files := []string{"tape-1.bkf", "tape-2.bkf"}
r.SetContinuation(func(c mtf.Continuation) (io.Reader, error) {
    if c.Sequence >= len(files) {
        return nil, io.EOF // no more media
    }
    fmt.Printf("load %s (tape %d)\n", files[c.Sequence], c.Sequence+1)
    return os.Open(files[c.Sequence])
})

If next is nil (the default), an EOTM ends the archive like io.EOF. If next returns io.EOF or a nil reader, the archive ends.

func (*Reader) SetDecryptor 

func (r *Reader) SetDecryptor(d Decryptor)

SetDecryptor registers a callback used to decrypt encrypted data streams (§6.5). The MTF specification defines no data-encryption cipher, so the algorithm is vendor-specific; supply a Decryptor matching the writer. If a stream is encrypted and no decryptor is registered, [Read] returns ErrEncrypted.

func (*Reader) SetSkipFunc added in v0.19.0 

func (r *Reader) SetSkipFunc(f SkipFunc)

SetSkipFunc overrides the policy with a custom seek-vs-read predicate.

func (*Reader) SetSkipPolicy added in v0.19.0 

func (r *Reader) SetSkipPolicy(p SkipPolicy, threshold int64)

SetSkipPolicy sets how Reader skips stream data. The default is SkipNever (sequential read), which is correct for full scans and bulk migration. Selective-restore callers (catalog-driven, jumping to a specific object per §3.4.3) should keep SkipNever and instead seek the source directly to the target before constructing the reader. SkipIfOver with a drive-appropriate threshold is appropriate only for sources where LOCATE is cheaper than streaming the given range.

func (*Reader) Tape 

func (r *Reader) Tape() *TapeInfo

Tape returns metadata from the most recent TAPE descriptor block, or nil if none has been encountered yet.

func (*Reader) TruncatedByEOTM 

func (r *Reader) TruncatedByEOTM() bool

TruncatedByEOTM reports whether the archive ended prematurely because an End-Of-Tape-Media marker was reached without a continuation medium being registered via Reader.SetContinuation. When true, the data set spans further media that were not supplied, and the returned snapshot is incomplete. Callers should warn the operator.

func (*Reader) VerifyChecksum 

func (r *Reader) VerifyChecksum() bool

VerifyChecksum reports whether the common-block header (MTF_DB_HDR) checksum of the current block matches the recomputed word-wise XOR over the remaining header fields (MTF spec, "Header Checksum"). This may be used to detect media corruption. It should be called immediately after [Next] returns (before [Read] consumes the entry). It always returns true when the current block buffer is too short to contain a checksum (nothing to verify).

Note: some writers emit a zero checksum; such blocks verify as valid only when every other header word is also zero, so a false "invalid" is possible for such archives. Treat the result as advisory.

type SetInfo 

type SetInfo struct {
	Name   string // set name
	Label  string // set label
	Owner  string // set owner/user
	Number uint16 // data-set number
	// Password is the data-set password, if recorded.
	Password string
	// PBA is the physical block address of the SSET block, used for seek
	// indexing in conjunction with a Media Based Catalog.
	PBA uint64
	// SoftwareVendorID is the registered vendor ID of the writing software.
	SoftwareVendorID uint16
	// SoftwareVersion is the writer's software version number.
	SoftwareVersion uint16
	Attributes      uint32
	Compression     uint16
	Encryption      uint16
	MajorVersion    uint8
	MinorVersion    uint8
	TimeZone        int8
	CreateTime      time.Time
}

SetInfo holds metadata from the MTF start-of-data-set (SSET) block.

type SetMap 

type SetMap struct {
	// MediaFamilyID is the MFMID of the Media Family this Set Map describes; it
	// matches TapeInfo.MFMID.
	MediaFamilyID uint32
	// Entries holds one entry per data set in the Media Family.
	Entries []SetMapEntry
}

SetMap is the cumulative Media Family summary parsed from a 'TSMP' stream. It lists one SetMapEntry per data set in the family, in the order the data sets were written.

func ReadSetMap added in v0.15.0 

func ReadSetMap(tape Tape) (*SetMap, error)

type SetMapEntry 

type SetMapEntry struct {
	// Len is the entry's declared LENGTH (UINT16 at offset 0): the byte size of
	// the entry's fixed fields plus, in the spec Type 1 layout, its nested
	// volume records and appended strings. Populated by ParseSetMapEntryFixed.
	Len int
	// NumVolumes is the declared Number-Of-Volumes count for this entry.
	NumVolumes int
	// MediaSeq is the Media Sequence Number of the medium this data set begins
	// on, copied from the TAPE block.
	MediaSeq uint16
	// FDDMediaSeq is the FDD Media Sequence Number from the ESET block.
	FDDMediaSeq uint16
	// SetNumber is the data-set number copied from the SSET block.
	SetNumber uint16
	// BlockAttributes is the MTF_DB_HDR common block attributes word.
	BlockAttributes uint32
	// SSETAttributes is the SSET attributes word.
	SSETAttributes uint32
	// SSETPBA is the Physical Block Address of the SSET block.
	SSETPBA uint64
	// FDDPBA is the Physical Block Address of this data set's FDD stream.
	FDDPBA uint64
	// FLA is the Format Logical Address of the SSET block.
	FLA uint64
	// NumDirectories is the count of directories in the data set.
	NumDirectories uint32
	// NumFiles is the count of files in the data set.
	NumFiles uint32
	// NumCorrupt is the count of corrupt files in the data set.
	NumCorrupt uint32
	// Size is the cumulative displayable size of the data set.
	Size uint64
	// Name is the data-set name.
	Name string
	// Description is the data-set description.
	Description string
	// Owner is the user name associated with the data set.
	Owner string
	// WriteTime is the media write date copied from the SSET block.
	WriteTime time.Time
	// TimeZone is the SSET time-zone offset (signed quarter-hours).
	TimeZone int8
	// Volumes are the volume entries (MTF_FDD_VOLB) following this Set Map
	// Entry, one per VOLB in the data set. Each carries the source device,
	// volume label and machine name.
	Volumes []CatalogEntry
}

SetMapEntry summarizes one data set (one SSET, typically one host's backup) within a Media Family, and is followed by its volume entries.

func ParseSetMapEntryFixed added in v0.17.0 

func ParseSetMapEntryFixed(rec []byte) (entry SetMapEntry, length int, ok bool)

ParseSetMapEntryFixed decodes one Set Map Entry's fixed fields (everything up to and excluding its volume entries) and returns it with Len set to the entry's declared LENGTH. It does NOT parse volume entries; callers add those per their layout (spec: nested in LENGTH; Backup Exec SMP2: separate records). ok is false if rec is too short or the declared LENGTH is implausible.

type SetMapParser added in v0.17.0 

type SetMapParser interface {
	ParseSetMap(raw []byte) *SetMap
}

parseSetMap decodes a Type 1 'TSMP' stream payload into a Set Map. It returns nil for a payload too short or non-standard to contain a Set Map header. SetMapParser parses a proprietary Set Map stream payload (the bytes after the 22-byte stream header) into a *SetMap. The MTF spec (§7.3.3) defines the standard Type 1 'TSMP' and Type 2 'MAP2' layouts, which the main package parses directly. Vendors may use other stream IDs for their catalogs (the spec reserves the MBC fields for "application specific information", §5.1); those are implemented by registering a SetMapParser for the stream ID.

Implementations are read-only and should tolerate truncation.

type SetMapParserFunc added in v0.17.0 

type SetMapParserFunc func(raw []byte) *SetMap

SetMapParserFunc is a function adapter for SetMapParser.

func (SetMapParserFunc) ParseSetMap added in v0.17.0 

func (f SetMapParserFunc) ParseSetMap(raw []byte) *SetMap

ParseSetMap implements SetMapParser.

type SkipDecision added in v0.19.0 

type SkipDecision struct {
	Bytes    int64 // bytes to skip
	PhysSize int   // physical block size of the medium
}

SkipDecision is the per-skip input a SkipFunc uses to decide seek vs read.

type SkipFunc added in v0.19.0 

type SkipFunc func(d SkipDecision) bool

SkipFunc is a caller-supplied predicate returning true to SeekBlock past the skip, false to read it sequentially. nil is treated as SkipNever.

type SkipPolicy added in v0.19.0 

type SkipPolicy int

SkipPolicy decides how Reader skips a data stream's bytes when advancing past a file's data (during a header-only walk or when the caller isn't reading the file content).

Per the MTF spec, PBAs between filemarks are sequential (§3.4.1) and a tape streams at native read rate while a LOCATE-based SeekBlock costs a fixed start/stop repositioning penalty (~1-3 s on LTO). So seeking is only faster than reading when the skip is large enough that streaming it would take longer than repositioning. For a full sequential scan (listing, bulk migration) the correct choice is to never seek — just read the bytes in order at native rate. For catalog-driven selective restore the caller seeks once to the target object (§3.4.3 / §3.3.2.2). 

const (
	// SkipNever always reads stream data sequentially, never SeekBlock. This is
	// the spec-correct mode for full scans and bulk migration: PBAs are
	// sequential between filemarks (§3.4.1), so streaming at native tape rate
	// is optimal and avoids per-file LOCATE/reposition penalties. It is the
	// default.
	SkipNever SkipPolicy = iota
	// SkipAlways issues a §3.4.3 SSET-anchored SeekBlock for every skip. Use
	// only when the source seeks faster than it streams for the relevant sizes
	// (e.g. random-access file-backed sources, or jumping a few very large
	// gaps).
	SkipAlways
	// SkipIfOver seeks only when the skip exceeds Threshold bytes; otherwise it
	// reads sequentially. Threshold should be chosen so seek latency <
	// Threshold/nativeReadRate for the target drive.
	SkipIfOver
)

type SliceTape added in v0.12.0 

type SliceTape struct {
	// contains filtered or unexported fields
}

SliceTape is a Tape backed by an in-memory byte slice. It models a single recorded section (no filemarks): the whole slice is delivered as a sequence of fixed-size blocks, then io.EOF. Useful for tests and for archives built in memory.

func NewSliceTape added in v0.12.0 

func NewSliceTape(b []byte) *SliceTape

NewSliceTape returns a SliceTape over b. Reads deliver b in maxTapeBlock-sized chunks; SeekBlock addresses chunks.

func (*SliceTape) ReadBlock added in v0.12.0 

func (t *SliceTape) ReadBlock(dst []byte) (int, error)

ReadBlock implements Tape.

func (*SliceTape) SeekBlock added in v0.12.0 

func (t *SliceTape) SeekBlock(block int64) error

SeekBlock implements Tape.

func (*SliceTape) TellBlock added in v0.13.0 

func (t *SliceTape) TellBlock() (int64, error)

TellBlock implements Tape, reporting the synthetic block number of the next chunk (byte position / chunk size).

type SparseExtent 

type SparseExtent struct {
	// Offset is the logical byte offset within the file where Data begins.
	Offset int64
	// Data is the non-hole byte content located at Offset.
	Data []byte
}

SparseExtent describes one contiguous block of non-hole data within a sparse file, as parsed from a SPAR data stream. The logical file is reconstructed by placing each extent's Data at Offset and zero-filling the gaps between extents.

type StreamData 

type StreamData struct {
	Type uint32
	Data []byte
}

StreamData holds the raw bytes of a data stream not otherwise exposed by a named field on Header. Type is the four-byte stream type code (compare against the Stream* constants, e.g. StreamNTOI).

type Tape added in v0.12.0 

type Tape interface {
	// ReadBlock reads the next physical block into dst and returns its length.
	// The returned length is the block's recorded size (for a tape drive, the
	// physical block size). dst must be at least as large as the largest block
	// the medium can produce.
	//
	// A filemark between recorded sections is reported as [ErrFilemark] with a
	// length of zero; the caller skips it and reads again. End of recorded
	// data (two consecutive filemarks, or a genuine clean EOF) is reported as
	// io.EOF. A block followed by end of data may be returned as (n, io.EOF):
	// the n bytes are delivered first and the io.EOF surfaces on the next call.
	ReadBlock(dst []byte) (int, error)

	// SeekBlock positions the source at the start of the given physical block
	// (0-based from the beginning of the medium). After SeekBlock the next
	// ReadBlock returns that block. This is the fast-skip path the Reader uses
	// to jump over a file's data streams without reading them (MTF §3.4.3).
	SeekBlock(block int64) error

	// TellBlock returns the physical block address (PBA) of the block about to
	// be returned by the next ReadBlock — i.e. the current position expressed
	// as a 0-based block number from the start of the medium. This is the value
	// MTF stores in the MTF_SSET DBLK and uses as the anchor for the §3.4.3
	// FLA→PBA seek calculation. On a real drive it is MTIOCPOS; on the in-memory
	// and file wrappers it is the synthetic block number (byte offset / chunk).
	//
	// Implementations that cannot report position should return an error; the
	// Reader falls back to sequential reading when TellBlock is unavailable.
	TellBlock() (int64, error)
}

Tape is the block-oriented source a Reader reads from.

MTF on linear tape (LTO) is a sequence of fixed-size physical blocks organised into recorded sections separated by filemarks; logical blocks (the FLB-sized units the format itself deals in) are packed many-per-physical block. A byte-stream model (io.Reader) cannot represent that: a filemark reads as zero bytes, which io.Reader can only express as io.EOF, so the walk would end at the first filemark instead of continuing into the data set. Tape therefore exposes the medium as physical blocks plus an explicit filemark sentinel.

Implementations:

  • an LTO drive through the Linux st driver (e.g. via github.com/pbs-plus/go-tapedrive), where ReadBlock is one MTread and SeekBlock is MTSEEK;
  • SliceTape, an in-memory buffer (single section, no filemarks);
  • the file wrapper used by Open, which presents a .bkf file as a tape with a single section.

A Tape is not safe for concurrent use.

type TapeInfo 

type TapeInfo struct {
	Software   string // generator software string
	Name       string // tape name
	Label      string // tape label
	Password   string // media password, if recorded
	MFMID      uint32 // media family id
	Attributes uint32 // TAPE attributes
	Sequence   uint16 // media sequence number within the media family
	FLBSize    uint16 // logical block size used by the archive
	// PasswordAlgorithm is the registered ID of the password-encryption
	// algorithm used to protect the media password, or zero.
	PasswordAlgorithm uint16
	// SoftFilemarkBlockSize is the soft filemark (SFMB) block size in units of
	// 512 bytes; only meaningful when soft filemarks are used.
	SoftFilemarkBlockSize uint16
	// CatalogType is the Media Based Catalog format type recorded on the tape.
	CatalogType uint16
	// SoftwareVendorID is the registered vendor ID of the writing software.
	SoftwareVendorID uint16
	// MTFMajorVersion is the MTF major revision recorded in the TAPE block.
	MTFMajorVersion uint8
	CreateTime      time.Time
}

TapeInfo holds metadata from the MTF TAPE descriptor block.

Source Files

View all Source files

Directories

Path Synopsis
Package becatalog parses the Backup Exec catalog payload that some MTF writers place inside the standard File/Directory Detail (FDD) stream envelope.
 Package becatalog parses the Backup Exec catalog payload that some MTF writers place inside the standard File/Directory Detail (FDD) stream envelope.
Package besetmap parses the Backup Exec proprietary Set Map stream.
 Package besetmap parses the Backup Exec proprietary Set Map stream.
cmd
bkfcensus command
Command bkfcensus runs mtf.Census on a single .bkf file and prints a compact one-line classification.
 Command bkfcensus runs mtf.Census on a single .bkf file and prints a compact one-line classification.
bkfscan command
Command bkfscan classifies every .bkf file in one or more directories using mtf.Census, printing a one-line summary and tallying aggregate stats.
 Command bkfscan classifies every .bkf file in one or more directories using mtf.Census, printing a one-line summary and tallying aggregate stats.
mtfwalk command
Command mtfwalk opens an MTF archive (seekable) and prints each block the reader yields, plus the file size for entries.
 Command mtfwalk opens an MTF archive (seekable) and prints each block the reader yields, plus the file size for entries.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.