README
¶
walk
-- import "github.com/wtsi-hgi/walk"
Package walk provides a fast, multi-threaded, memory efficient directectory walker.
Example
package main
import (
"fmt"
"os"
"strings"
"github.com/wtsi-hgi/walk"
)
func main() {
cwd, _ := os.Getwd()
cwd += "/"
if err := walk.New(func(de *walk.Dirent) error {
path := strings.TrimPrefix(string(de.Bytes()), cwd)
if !strings.HasPrefix(path, ".git/") {
fmt.Println(path)
}
return nil
}, true, false).Walk(cwd, func(path string, err error) {
fmt.Printf("%s: %s\n", path, err)
}); err != nil {
fmt.Println(err)
}
// Output:
// .github/
// .github/workflows/
// .github/workflows/go-checks.yml
// .golangci.yaml
// README.md
// dirent.go
// dirent_test.go
// example_test.go
// file_test.go
// go.mod
// go.sum
// stats.go
// walk.go
// walk_linux_amd64.go
// walk_linux_arm64.go
// walk_test.go
}
Documentation
Full API docs can be found at:
Documentation
¶
Overview ¶
******************************************************************************
- Copyright (c) 2025 Genome Research Ltd. *
- Author: Michael Woolnough <mw31@sanger.ac.uk> *
- Permission is hereby granted, free of charge, to any person obtaining
- a copy of this software and associated documentation files (the
- "Software"), to deal in the Software without restriction, including
- without limitation the rights to use, copy, modify, merge, publish,
- distribute, sublicense, and/or sell copies of the Software, and to
- permit persons to whom the Software is furnished to do so, subject to
- the following conditions: *
- The above copyright notice and this permission notice shall be included
- in all copies or substantial portions of the Software. *
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
- IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
- CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
- TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
- SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. *****************************************************************************
Example ¶
package main
import (
"fmt"
"path/filepath"
"runtime"
"strings"
"github.com/wtsi-hgi/walk"
)
func main() {
_, filename, _, _ := runtime.Caller(0)
cwd := filepath.Dir(filename) + "/"
if err := walk.New(func(de *walk.Dirent) error {
path := strings.TrimPrefix(string(de.Bytes()), cwd)
if !strings.HasPrefix(path, ".git/") {
fmt.Println(path)
}
return nil
}, true, false).Walk(cwd, func(path string, err error) {
fmt.Printf("%s: %s\n", path, err)
}); err != nil {
fmt.Println(err)
}
}
Output: .github/ .github/workflows/ .github/workflows/go-checks.yml .golangci.yaml README.md dirent.go dirent_test.go example_test.go file_test.go go.mod go.sum stats.go walk.go walk_linux_amd64.go walk_linux_arm64.go walk_test.go
Index ¶
- type Dirent
- type ErrorCallback
- type PathCallback
- type StatData
- type StatsOutput
- type Walker
- func (s *Walker) AddClose()
- func (s *Walker) AddOpen()
- func (s *Walker) AddRead(count int)
- func (s *Walker) AddStat()
- func (s *Walker) AddWrite(count int)
- func (w *Walker) EnableStats(interval time.Duration, output StatsOutput)
- func (s *Walker) LogStats(ctx context.Context) func()
- func (w *Walker) Walk(dir string, errCB ErrorCallback) error
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Dirent ¶
type Dirent struct {
// Inode is the file system inode number for this entry.
Inode uint64
// contains filtered or unexported fields
}
Dirent represents a file system directory entry (a file or a directory), providing information about the entry's path, type and inode.
type ErrorCallback ¶
ErrorCallback is a callback function you supply Walker.Walk(), and it will be provided problematic paths encountered during the walk.
type PathCallback ¶
PathCallback is a callback used by Walker.Walk() that receives a directory entry containing the path, inode and file type each time it's called. It should only return an error if you can no longer cope with receiving more paths, and wish to terminate the Walk.
type StatData ¶
type StatData struct {
Open, Read, Stat, Close, Bytes, Write, WriteBytes uint64
}
StatsData is a count of the various syscalls that have occurred and a count of the bytes read and written.
type StatsOutput ¶
RecordStatFunc is a function that will be periodically called given the current time and the number of syscalls that have occurred since the last time this function was called.
type Walker ¶
type Walker struct {
// contains filtered or unexported fields
}
Walker can be used to quickly walk a filesystem to just see what paths there are on it.
func New ¶
func New(cb PathCallback, includDirs, ignoreSymlinks bool) *Walker
New creates a new Walker that can Walk() a filesystem and send all the encountered paths to the given PathCallback.
Set includeDirs to true to have your PathCallback receive directory paths in addition to file paths.
Set ignoreSymlinks to true to have symlinks not be sent do your PathCallback.
func (*Walker) AddWrite ¶
func (s *Walker) AddWrite(count int)
AddWrite records the number of bytes given to the rolling written byte count, and increase the number of writes by one.
func (*Walker) EnableStats ¶
func (w *Walker) EnableStats(interval time.Duration, output StatsOutput)
func (*Walker) Walk ¶
func (w *Walker) Walk(dir string, errCB ErrorCallback) error
Walk will discover all the paths nested under the given dir, and send them to our PathCallback.
The given error callback will be called every time there's an error handling a file during the walk. Errors writing to an output file will result in the walk terminating early and this method returning the error; other kinds of errors will mean the path isn't output, but the walk will continue and this method won't return an error.
Source Files