nogo

README

NoGo

A .gitignore parser for Go.

Features

• parsing .gitignore files
• loading file trees with several .gitignore files
• fs.WalkDir and afero.Walk wrappers which ignore all ignored files
• full compatibility with git
• customizable ignore filename (instead of .gitignore)

As far as I could test it, it handles .gitignore files the same way as git.
If you find an inconsistency with git, create a new Issue.
The goal is to provide the exact same .gitignore handling.

Usage

By default, the working directory is used as root. You can modify that by adding nogo.WithFS(anyFSImplementation)

Preconfigured for standard '.gitignore' files:

n := nogo.NewGitignore()
match, err := n.MatchPath(toSearch)
if err != nil {
    panic(err)
}
fmt.Println(match)

Plain:

n := nogo.New([]string{".nogoignore"})
match, err := n.MatchPath(toSearch)
if err != nil {
    panic(err)
}
fmt.Println(match)

There are also some alternative Match* methods which return also the causing rule and/or accept a isDir boolean and therefore skip an extra Stat call which may be more performant.

Options

All functions (nogo.New, nogo.NewGitignore, nogo.WalkDir, nogo.WalkAfero) accept options as the last parameters, which can further modify NoGo. Please read the GoDoc for more information.

• WithFS (if not used, the working directory is used)
• WithRules (you can also pass new rules after creation by using any of the Add* methods)
• WithoutMatchParents (more performant, but may introduce false results if used in a wrong way)

There exists a predefined rule to ignore the .git folder automatically: (nogo.NewGitignore uses it by default)

n := nogo.New([]string{".nogoignore"}, WithRules(GitIgnoreRule...))

Walk

NoGo provides custom walk functions which ignore all files which are ignored by any ignore file. It exists in two flavours as I noticed that the fs.WalkDir function is not that easy to use with afero.NewIOFS especially on windows.
See walk and walkAfero for examples.

Dependencies

NoGo is built in pure Go with the exception of

• afero which is only needed by the AferoWalk function and to run the tests.
• testify which is only needed to run the tests.

Documentation

Overview

Package nogo implements gitignore parsing in pure go. It supports the official specification. https://git-scm.com/docs/gitignore/2.34.0

PATTERN FORMAT

  * A blank line matches no files, so it can serve as a separator for readability.

  * A line starting with # serves as a comment. Put a backslash ("\") in front of the first hash for patterns that begin with a hash.

  * Trailing spaces are ignored unless they are quoted with backslash ("\").

  * An optional prefix "!" which negates the pattern; any matching file excluded by a previous pattern will become included again. It is not possible to re-include a file if a parent directory of that file is excluded. Git doesn’t list excluded directories for performance reasons, so any patterns on contained files have no effect, no matter where they are defined. Put a backslash ("\") in front of the first "!" for patterns that begin with a literal "!", for example, "\!important!.txt".

  * The slash / is used as the directory separator. Separators may occur at the beginning, middle or end of the .gitignore search pattern.

  * If there is a separator at the beginning or middle (or both) of the pattern, then the pattern is relative to the directory level of the particular .gitignore file itself. Otherwise the pattern may also matches at any level below the .gitignore level.

  * If there is a separator at the end of the pattern then the pattern will only matches directories, otherwise the pattern can matches both files and directories.
    For example, a pattern doc/frotz/ matches doc/frotz directory, but not a/doc/frotz directory; however frotz/ matches frotz and a/frotz that is a directory (all paths are relative from the .gitignore file).

  * An asterisk "*" matches anything except a slash. The character "?" matches any one character except "/". The range notation, e.g. [a-zA-Z], can be used to matches one of the characters in a range. See fnmatch(3) and the FNM_PATHNAME flag for a more detailed description.

Two consecutive asterisks ("**") in patterns matched against full pathname may have special meaning:

  * A leading "**" followed by a slash means matches in all directories. For example, "**/foo" matches file or directory "foo" anywhere, the same as pattern "foo". "**/foo/bar" matches file or directory "bar" anywhere that is directly under directory "foo".

  * A trailing "/**" matches everything inside. For example, "abc/**" matches all files inside directory "abc", relative to the location of the .gitignore file, with infinite depth.

  * A slash followed by two consecutive asterisks then a slash matches zero or more directories. For example, "a/**/b" matches "a/b", "a/x/b", "a/x/y/b" and so on.

  * Other consecutive asterisks are considered regular asterisks and will matches according to the previous rules.

Index

• Variables
• func AferoWalk(ignoreFileNames []string, fsys afero.Fs, fn filepath.WalkFunc, ...) error
• func WalkDir(ignoreFileNames []string, fsys fs.FS, root string, fn fs.WalkDirFunc, ...) error
• type NoGo
  • func New(ignoreFileNames []string, options ...Option) *NoGo
  • func NewGitignore(options ...Option) *NoGo
  • func (n *NoGo) AddAll() error
  • func (n *NoGo) AddFile(path string) error
  • func (n *NoGo) AddRules(rules ...Rule)
  • func (n *NoGo) Apply(options ...Option)
  • func (n *NoGo) MatchPath(path string) (bool, error)
  • func (n *NoGo) MatchPathBecause(path string) (match bool, because Result, err error)
  • func (n *NoGo) MatchPathBecauseNoStat(path string, isDir bool) (match bool, because Result)
  • func (n *NoGo) MatchPathNoStat(path string, isDir bool) bool
• type Option
  • func WithFS(f fs.FS) Option
  • func WithRules(rules ...Rule) Option
  • func WithoutMatchParents() Option
• type Result
  • func (r Result) Resolve(isDir bool) bool
• type Rule
  • func Compile(prefix string, pattern string) (skip bool, rule Rule, err error)
  • func CompileAll(prefix string, data []byte) ([]Rule, error)
  • func MustCompileAll(prefix string, data []byte) []Rule
  • func (r Rule) MatchPath(path string) Result

Variables

var (
	GitIgnoreRule = MustCompileAll("", []byte(".git"))
)

Functions

func AferoWalk

func AferoWalk(ignoreFileNames []string, fsys afero.Fs, fn filepath.WalkFunc, options ...Option) error

AferoWalk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root. All errors that arise visiting files and directories are filtered by walkFn. The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient. Walk does not follow symbolic links.

This implementation skips all folders and files according to the ignore files found in the file-tree.

All options you pass, are applied to the internal NoGo instance.

func WalkDir

func WalkDir(ignoreFileNames []string, fsys fs.FS, root string, fn fs.WalkDirFunc, options ...Option) error

WalkDir walks the file tree rooted at root, calling fn for each file or directory in the tree, including root. This implementation skips all folders and files according to the ignore files found in the file-tree.

All options you pass, are applied to the internal NoGo instance.

All errors that arise visiting files and directories are filtered by fn: see the fs.WalkDirFunc documentation for details.

The files are walked in lexical order, which makes the output deterministic but requires WalkDir to read an entire directory into memory before proceeding to walk that directory.

WalkDir does not follow symbolic links found in directories, but if root itself is a symbolic link, its target will be walked.

Types

type NoGo

type NoGo struct {
	// contains filtered or unexported fields
}

func New

func New(ignoreFileNames []string, options ...Option) *NoGo

New creates a NoGo instance which works for the given ignoreFileNames. You can pass additional options if needed.

func NewGitignore

func NewGitignore(options ...Option) *NoGo

NewGitignore creates a new NoGo which is already preset for the use with .gitignore files, as this is very common.

It not only sets the ignoreFileName to ".gitignore" but also adds the GitIgnoreRule to avoid .git folders.

func (*NoGo) AddAll

func (n *NoGo) AddAll() error

AddAll ignore files which can be found. It only loads ignore files which are not ignored itself by another file.

func (*NoGo) AddFile

func (n *NoGo) AddFile(path string) error

AddFile reads the given file and tries to load the content as a ignore file. It does not check the filename. So you can add any file, independently of the configured ignoreFileNames.

The folder of the give filepath is used as Prefix for the rules.

Note that the order in which rules are added is very important. You should always first add the rules of parent folders and then of the children folders. TODO: in the future the rules could be re-sorted based on the prefix names.

func (*NoGo) AddRules

func (n *NoGo) AddRules(rules ...Rule)

AddRules to NoGo which are already compiled.

func (*NoGo) Apply

func (n *NoGo) Apply(options ...Option)

Apply options to NoGo. This is also possible when instantiating with New.

func (*NoGo) MatchPath

func (n *NoGo) MatchPath(path string) (bool, error)

MatchPath calculates if the path matches any rule.

It does the same as MatchPathBecauseNoStat, but it itself determines if path is a directory. If you already have that information, use MatchPathBecauseNoStat.

As it has to query the filesystem, it may return an error due to possible filesystem errors.

func (*NoGo) MatchPathBecause

func (n *NoGo) MatchPathBecause(path string) (match bool, because Result, err error)

MatchPathBecause calculates if the path matches any rule. It returns the match but also a result, where the match was calculated from. Use MatchPath or MatchPathNoStat if you do not need the cause.

It does the same as MatchPathBecauseNoStat, but it itself determines if path is a directory. If you already have that information, use MatchPathBecauseNoStat.

As it has to query the filesystem, it may return an error due to possible filesystem errors.

func (*NoGo) MatchPathBecauseNoStat

func (n *NoGo) MatchPathBecauseNoStat(path string, isDir bool) (match bool, because Result)

MatchPathBecauseNoStat calculates if the path matches any rule. It returns the match but also a result, where the match was calculated from. Use MatchPath or MatchPathNoStat if you do not need the cause.

It does the same as MatchPathBecause, but you have to pass yourself if the path is a directory. This can be useful to avoid unneeded Stat-calls. (e.g. in a Walk function where you already have that information)

func (*NoGo) MatchPathNoStat

func (n *NoGo) MatchPathNoStat(path string, isDir bool) bool

MatchPathNoStat calculates if the path matches any rule.

It does the same as MatchPath, but you have to pass yourself if the path is a directory. This can be useful to avoid unneeded Stat-calls. (e.g. in a Walk function where you already have that information)

type Option

type Option func(noGo *NoGo)

func WithFS

func WithFS(f fs.FS) Option

WithFS is an option which injects a filesystem. If this is not passed, the current working directory is used.

func WithRules

func WithRules(rules ...Rule) Option

WithRules can be used to add rules without an extra call to an Add method. This can be used to add predefined rules like the GitIgnoreRule. For example:

n := nogo.New(nogo.WithRules(nogo.GitIgnoreRule...))

func WithoutMatchParents

func WithoutMatchParents() Option

WithoutMatchParents disables the time-consuming check for all parents. You can use this if you know that no file gets checked without also all parents being checked before.

As the parent-check is time-consuming it is for example better to disable that check when using Walk function. (nogo.WalkDir and nogo.WalkAfero do this automatically).

Example:

Folder1
 - File1
.gitignore -> Rule: "/Folder1"

If the gitignore contains the rule "/Folder1" and you check the file `/Folder1/File1`, you will get a correct match.

But if you check the file WITH "WithoutMatchParents", the file will not match as it itself is not in any ignore-list and the parent folder does not get checked.

When doing file traversal with a Walk method, this doesn't matter as the Folder1 won't be read and therefore /Folder1/File1 won't be read either.

But when checking only the file /Folder1/File1 directly, you will NOT want "WithoutMatchParents".

type Result

type Result struct {
	Rule

	// Found is true if any matching rule was found.
	// Do not use it to check if the file is actually to be ignored!
	// For this use Resolve as it takes into account some special cases.
	Found bool

	// ParentMatch saves if the actual rule matched for a parent or not.
	// In case of a parent match the check for OnlyFolder has to be different.
	ParentMatch bool
}

func (Result) Resolve

func (r Result) Resolve(isDir bool) bool

Resolve the Result by taking into account OnlyFolder and if the matched path is a directory.

type Rule

type Rule struct {
	Regexp     *regexp.Regexp
	Prefix     string
	Pattern    string
	Negate     bool
	OnlyFolder bool
}

func Compile

func Compile(prefix string, pattern string) (skip bool, rule Rule, err error)

Compile the pattern into a single regexp. skip means that this pattern doesn't contain any rule (e.g. just a comment or empty line).zz

func CompileAll

func CompileAll(prefix string, data []byte) ([]Rule, error)

CompileAll rules in the given data line by line. The prefix is added to all rules.

func MustCompileAll

func MustCompileAll(prefix string, data []byte) []Rule

MustCompileAll does the same as CompileAll but panics on error.

func (Rule) MatchPath

func (r Rule) MatchPath(path string) Result