fastwalk

Documentation

Overview

Package fastwalk provides a faster version of filepath.Walk for file system scanning tools.

Index

• Variables
• func Walk(conf *Config, root string, walkFn WalkFunc) error
• type Config
• type DirEntry
• type EntryFilter
  • func NewEntryFilter() *EntryFilter
  • func (e *EntryFilter) Entry(_ string, de DirEntry) bool
• type WalkFunc
  • func FollowSymlinks(walkFn WalkFunc) WalkFunc
  • func IgnoreDuplicateFiles(walkFn WalkFunc) WalkFunc

Variables

var DefaultConfig = Config{
	Follow:     false,
	NumWorkers: max(runtime.NumCPU(), 4),
}

var ErrSkipFiles = errors.New("fastwalk: skip remaining files in directory")

ErrSkipFiles is a used as a return value from WalkFuncs to indicate that the callback should not be called for any other files in the current directory. Child directories will still be traversed.

var ErrTraverseLink = errors.New("fastwalk: traverse symlink, assuming target is a directory")

ErrTraverseLink is used as a return value from WalkFuncs to indicate that the symlink named in the call may be traversed.

Functions

func Walk

func Walk(conf *Config, root string, walkFn WalkFunc) error

Walk is a faster implementation of filepath.Walk.

filepath.Walk's design necessarily calls os.Lstat on each file, even if the caller needs less info. Many tools need only the type of each file. On some platforms, this information is provided directly by the readdir system call, avoiding the need to stat each file individually. fastwalk_unix.go contains a fork of the syscall routines.

See golang.org/issue/16399

Walk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root.

If walkFn returns filepath.SkipDir, the directory is skipped.

Unlike filepath.Walk:

• File stat calls must be done by the user and should be done via the DirEntry argument to walkFn since it caches the results of Stat and Lstat.
• Multiple goroutines stat the filesystem concurrently. The provided walkFn must be safe for concurrent use.
• Walk can follow symlinks if walkFn returns the TraverseLink sentinel error. It is the walkFn's responsibility to prevent Walk from going into symlink cycles.

Types

type Config

type Config struct {
	// Follow symbolic links ignoring duplicate directories.
	Follow bool

	// Number of parallel workers to use. If NumWorkers if ≤ 0 then
	// the greater of runtime.NumCPU() or 4 is used.
	NumWorkers int

	// TODO: do we want this?
	Error func(path string, err error) error
}

type DirEntry

type DirEntry interface {
	fs.DirEntry

	// Stat returns the FileInfo for the file or subdirectory described
	// by the entry. The returned FileInfo may be from the time of the
	// original directory read or from the time of the call to Stat.
	// If the entry denotes a symbolic link, Stat reports the information
	// about the target itself, not the link.
	Stat() (os.FileInfo, error)
}

A DirEntry extends the fs.DirEntry interface to add a Stat() method that returns the result of calling os.Stat() on the underlying file. The results of Info() and Stat() are cached.

type EntryFilter

type EntryFilter struct {
	// contains filtered or unexported fields
}

func NewEntryFilter

func NewEntryFilter() *EntryFilter

func (*EntryFilter) Entry

func (e *EntryFilter) Entry(_ string, de DirEntry) bool

type WalkFunc

type WalkFunc func(path string, ent DirEntry) error

WalkFunc is the type of the function called by Walk to visit each file or directory and must be safe for concurrent use.

func FollowSymlinks

func FollowSymlinks(walkFn WalkFunc) WalkFunc

TODO: consider using wrappers like this to ignore duplicate files and to traverse links

func IgnoreDuplicateFiles

func IgnoreDuplicateFiles(walkFn WalkFunc) WalkFunc

TODO: consider using wrappers like this to ignore duplicate files and to traverse links