tree

Documentation

Index

• Variables
• type BubbleUpProps
• type DirJob
• type File
  • func (a *File) Equal(b File) bool
• type FileJob
• type FileTree
  • func ReadBinary(path string) (FileTree, error)
  • func WalkGenerateTreeRecursive(path string, depth int, isComprehensive bool, walkStats *stats.WalkStats) (tree *FileTree)
  • func WalkTreeIterativeDir(rootPath string, isComprehensive bool, walkStats *stats.WalkStats) (t *FileTree)
  • func WalkTreeIterativeFile(rootPath string, depth int, isComprehensive bool, walkStats *stats.WalkStats) (t *FileTree)
  • func (tree *FileTree) DeepCopy() FileTree
  • func (a *FileTree) Equal(b FileTree) error
  • func (tree *FileTree) WriteBinary(path string) error
• type ReadLocation

Variables

var (
	MEGABYTE = 1024 * 1024
)

Types

type BubbleUpProps

type BubbleUpProps struct {
	Size          int64
	NewestModtime time.Time
	NumFiles      int64
}

type DirJob

type DirJob struct {
	ThisIndexBuildQ int
	AllHashByte     *[]byte
	IsComprehensive bool
	Depth           int
	WalkStats       *stats.WalkStats
}

A "job" assigned to a thread's queue for it to perform a `readdir`, as well as any subsequent `stat`, "read" and "hash" operations required on files in the IMMEDIATE directory.

The 'path' for the `DirJob` to operate on is located in `buildQ[ThisIndexInBuildQ]`

type File

type File struct {
	Name         string
	Hash         utility.HashLocation
	Size         int64
	Err          string
	LastModified time.Time
}

Encapsulates size information for a file in the FS

func (*File) Equal

func (a *File) Equal(b File) bool

Not a complete equality check, doesn't check hashed bytes of both files

type FileJob

type FileJob struct {
	FullPath           string
	ParentIndexInQueue int
	ThisIndexInFiles   int
	Entry              os.DirEntry
	File               File
	ErrStrings         []string
	IsComprehensive    bool
	LastModified       time.Time
	AllHashByte        *[]byte

	WalkStats  *stats.WalkStats
	HashOffset int
	HashLength int
}

A "job" assigned to a thread's queue, for it to perform a `stat`, "read" and "hash" operations on a file, putting the result in:

The 'path' for the `FileJob` to operate on is located in `buildQ[ThisIndexInBuildQ]` (where it adds the file to it's `.Files` properties and updates some other props)

type FileTree

type FileTree struct {
	// Non-recursive data
	Comprehensive bool
	BasePath      string
	Files         []File
	ErrStrings    []string
	LastVisited   time.Time
	TimeTaken     time.Duration
	Depth         int

	LastModifiedDirect time.Time
	SizeDirect         int64
	NumFilesDirect     int64

	// Recursive data
	LastModifiedBelow time.Time
	SizeBelow         int64
	NumFilesBelow     int64
	SubTrees          []FileTree

	AllHash       []byte // Only populated at depth == 0
	AllHashOffset int64
}

A recursive structure, representing the full FileTree structure at a point in time

func ReadBinary

func ReadBinary(path string) (FileTree, error)

func WalkGenerateTreeRecursive

func WalkGenerateTreeRecursive(path string, depth int, isComprehensive bool, walkStats *stats.WalkStats) (tree *FileTree)

Currently the slowest, although still correct, algorithm. Currently singlethreaded.

TODO: Rework for a MT approach, we'd likely need information about the `n`, roughly, equally most expensive, subtrees ahead of time so we can split off from the main thread at those point. Then rejoin the subtrees from those threads with the main thread.

func WalkTreeIterativeDir

func WalkTreeIterativeDir(rootPath string, isComprehensive bool, walkStats *stats.WalkStats) (t *FileTree)

WARNING: This algorithm doesn't produce valid trees at the moment, tree order past the root is non-deterministic

A walk algorithm that runs `readdir` operations on immediate files (`stat`, "read" and "hash") for multiple dirs across multiple threads.

Fastest for "shallow" scans, but slower than `WalkTreeIterativeFile` for "comprehensive"

func WalkTreeIterativeFile

func WalkTreeIterativeFile(rootPath string, depth int, isComprehensive bool, walkStats *stats.WalkStats) (t *FileTree)

A walk algorithm that runs file operations (`stat`, "read" and "hash") for multiple files across multiple threads.

Fastest for "comprehensive" scans, but slower than `WalkTreeIterativeDir` for "shallow" scans

func (*FileTree) DeepCopy

func (tree *FileTree) DeepCopy() FileTree

Using `gob` to do a basic deep copy of a tree

(useful when you want to modify an existing tree by adding a `ScanDiff` to it)

func (*FileTree) Equal

func (a *FileTree) Equal(b FileTree) error

func (*FileTree) WriteBinary

func (tree *FileTree) WriteBinary(path string) error

type ReadLocation

type ReadLocation struct {
	WalkStats   *stats.WalkStats
	HashOffset  int
	HashLength  int
	FullPath    string
	Size        int64
	AllHashByte *[]byte
}