richglob

README

richglob

Brings Bash-style globbing to Go for filesystem matching like recursive **, extended patterns, case-insensitive searches, ignore rules, and more. All opt-in for flexibility.

Install

go get go.dw1.io/richglob

Quick start

package main

import (
	"fmt"
	"path/filepath"

	"go.dw1.io/richglob"
)

func main() {
	matched, err := richglob.Match(
		filepath.FromSlash("src/**/main.go"),
		filepath.FromSlash("src/cmd/tool/main.go"),
		richglob.WithGlobStar(),
	)
	if err != nil {
		panic(err)
	}

	println(matched)
}

matches, err := richglob.Glob("src/**/*.go", richglob.WithGlobStar())
if err != nil {
	panic(err)
}

for _, match := range matches {
	println(match)
}

for match, err := range richglob.GlobSeq2("src/**/*.go", richglob.WithGlobStar()) {
	if err != nil {
		panic(err)
	}

	println(match)
}

Pattern syntax

By default, richglob supports the usual glob operators:

• * matches any sequence of non-separator characters
• ? matches any single non-separator character
• [abc] matches a character class
• [^abc] matches any character outside a class
• \ escapes metacharacters on non-Windows platforms

Patterns are path-aware. * and ? do not cross path separators.

Options

• WithGlobStar() enables ** to match zero or more directory segments.
• WithExtGlob() enables extglob operators: ?(...), *(...), +(...), @(...), !(...).
• WithNoCaseGlob() makes matching case-insensitive.
• WithDotGlob() includes hidden entries when Bash pathname rules are active.
• WithGitIgnore() auto-discovers .gitignore files while walking and applies them as traversal-time filters.
• WithGlobSkipDots(true) skips . and .. during Bash-style expansion.
• WithGlobASCIIRanges() makes ranges such as [A-Z] use ASCII ordering.
• WithGlobIgnore(patterns...) filters Glob results through ignore patterns.
• WithSort(richglob.SortNone) keeps filesystem traversal order.
• WithNullGlob() returns an empty, non-nil slice when nothing matches.
• WithFailGlob() returns ErrNoMatch when nothing matches.

[!NOTE]

Behavior notes

• Match checks a single path or path segment string against a pattern.
• Glob walks the filesystem and returns matching paths.
• GlobSeq2 walks the filesystem lazily and yields (path, error) pairs.
• Glob sorts results lexicographically by default.
• GlobSeq2 yields matches in traversal order and does not globally sort before yielding.
• If Glob finds nothing, it returns nil, nil by default.
• GlobSeq2 yields a terminal error pair for malformed patterns, invalid ignore patterns, or WithFailGlob().
• WithFailGlob() takes precedence over WithNullGlob().
• Hidden files are included by default. When Bash-style pathname rules are enabled, hidden entries are excluded unless WithDotGlob() is also set. WithGlobIgnore(...) opts into Bash-style hidden-file handling on its own, while WithGitIgnore() keeps hidden entries filtered unless WithDotGlob() is also enabled.
• When WithGitIgnore() and WithGlobIgnore(...) are both set, .gitignore rules apply first during traversal and WithGlobIgnore(...) acts as an additional result filter.

Examples

Case-insensitive matching:

ok, err := richglob.Match("*.GO", "main.go", richglob.WithNoCaseGlob())

Recursive search with **:

matches, err := richglob.Glob("src/**/*.go", richglob.WithGlobStar())

Lazy recursive search:

for match, err := range richglob.GlobSeq2("src/**/*.go", richglob.WithGlobStar()) {
	if err != nil {
		panic(err)
	}

	println(match)
}

Extglob alternation:

ok, err := richglob.Match("@(main|util).go", "util.go", richglob.WithExtGlob())

Ignore generated files:

matches, err := richglob.Glob(
	"src/**/*.go",
	richglob.WithGlobStar(),
	richglob.WithGlobIgnore("src/**/*_generated.go"),
)

Respect nested .gitignore files while walking:

matches, err := richglob.Glob(
	"src/**/*.go",
	richglob.WithGlobStar(),
	richglob.WithGitIgnore(),
)

Benchmarks

Compared against the doublestar and the std library's filepath.{Match,Glob}.

benchstat

goos: linux
goarch: amd64
pkg: benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                  │  richglob   │              doublestar              │                  std                   │
                  │   sec/op    │    sec/op     vs base                │    sec/op     vs base                  │
Match-4             80.05n ± 3%   108.70n ± 0%  +35.80% (p=0.000 n=10)   106.65n ± 0%  +33.24% (p=0.000 n=10)
Match/recursive-4   129.4n ± 0%    105.7n ± 0%  -18.28% (p=0.000 n=10)
Glob-4              13.98µ ± 1%    13.74µ ± 1%   -1.70% (p=0.000 n=10)    17.55µ ± 1%  +25.54% (p=0.000 n=10)
Glob/recursive-4    90.50µ ± 1%   162.24µ ± 1%  +79.26% (p=0.000 n=10)
geomean             1.902µ         2.250µ       +18.25%                   1.368µ       +29.33%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                  │    richglob    │               doublestar                │                   std                    │
                  │      B/op      │     B/op       vs base                  │     B/op      vs base                    │
Match-4               0.000 ± 0%        0.000 ± 0%        ~ (p=1.000 n=10) ¹     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Match/recursive-4     0.000 ± 0%        0.000 ± 0%        ~ (p=1.000 n=10) ¹
Glob-4              1.495Ki ± 0%      1.872Ki ± 0%  +25.21% (p=0.000 n=10)     1.026Ki ± 0%  -31.35% (p=0.000 n=10)
Glob/recursive-4    8.115Ki ± 0%     11.178Ki ± 0%  +37.74% (p=0.000 n=10)
geomean                          ²                  +14.60%                ²                 -17.15%                ³ ²
¹ all samples are equal
² summaries must be >0 to compute geomean
³ benchmark set differs from baseline; geomeans may not be comparable

                  │   richglob   │              doublestar              │                  std                   │
                  │  allocs/op   │ allocs/op   vs base                  │ allocs/op   vs base                    │
Match-4             0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹   0.000 ± 0%        ~ (p=1.000 n=10) ¹
Match/recursive-4   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Glob-4              31.00 ± 0%     53.00 ± 0%  +70.97% (p=0.000 n=10)     24.00 ± 0%  -22.58% (p=0.000 n=10)
Glob/recursive-4    152.0 ± 0%     273.0 ± 0%  +79.61% (p=0.000 n=10)
geomean                        ²               +32.38%                ²               -12.01%                ³ ²
¹ all samples are equal
² summaries must be >0 to compute geomean
³ benchmark set differs from baseline; geomeans may not be comparable

Highlights:

• richglob outperforms doublestar by 35-36% in Match ops and is 25% faster than the standard library's Glob.
• For recursive globbing (**), richglob is 79% faster than doublestar.
• richglob uses fewer memory allocs (31 vs 53 for doublestar) and less memory in Glob ops.
• Overall geomean perf shows richglob is 18% faster than doublestar and 29% faster than std.

Run benchmarks yourself:

make -C benchmarks/

License

richglob is released with ♡ by @dwisiswant0 under the Apache 2.0 license. See LICENSE.

Documentation

Overview

Package richglob brings Bash-style globbing to your filesystem matching like globstar, extglob, case-insensitive matching, ignore patterns, and .gitignore-aware traversal.

• Match reports whether a single name matches a pattern.
• Glob walks the filesystem and returns matching paths.
• GlobSeq2 walks the filesystem lazily and yields matching paths together with a terminal error when one occurs.

By default, the package follows its own matching rules rather than Bash's pathname expansion rules. Bash-specific behaviors are enabled explicitly with options such as WithGlobStar, WithExtGlob, WithNoCaseGlob, WithDotGlob, and WithGitIgnore.

Index

• Constants
• Variables
• func Glob(pattern string, opts ...Option) ([]string, error)
• func GlobSeq2(pattern string, opts ...Option) iter.Seq2[string, error]
• func Match(pattern, name string, opts ...Option) (bool, error)
• type Option
  • func WithDotGlob() Option
  • func WithExtGlob() Option
  • func WithFailGlob() Option
  • func WithGitIgnore() Option
  • func WithGlobASCIIRanges() Option
  • func WithGlobIgnore(patterns ...string) Option
  • func WithGlobSkipDots(enabled bool) Option
  • func WithGlobStar() Option
  • func WithNoCaseGlob() Option
  • func WithNullGlob() Option
  • func WithSort(mode SortMode) Option
• type SortMode

Examples

• Glob
• GlobSeq2
• Match

Constants

const Separator = os.PathSeparator

Separator is the operating system-specific path separator used by patterns.

Variables

var ErrBadPattern = errors.New("syntax error in pattern")

ErrBadPattern indicates a pattern was malformed.

var ErrNoMatch = errors.New("no matches found")

ErrNoMatch reports that pathname expansion produced no matches.

Functions

func Glob

func Glob(pattern string, opts ...Option) ([]string, error)

Glob returns the filesystem paths that match pattern.

When nothing matches, Glob returns nil, nil by default. Use WithNullGlob to return an empty slice instead, or WithFailGlob to return ErrNoMatch.

Example

package main

import (
	"fmt"
	"os"
	"path/filepath"

	"go.dw1.io/richglob"
)

func main() {
	tmpDir, err := os.MkdirTemp("", "richglob-example-")
	if err != nil {
		fmt.Println(err)
		return
	}
	defer func() {
		if err := os.RemoveAll(tmpDir); err != nil {
			fmt.Println(err)
		}
	}()

	oldWD, err := os.Getwd()
	if err != nil {
		fmt.Println(err)
		return
	}
	defer func() {
		if err := os.Chdir(oldWD); err != nil {
			fmt.Println(err)
		}
	}()

	for _, path := range []string{
		filepath.Join(tmpDir, "src", "main.go"),
		filepath.Join(tmpDir, "src", "pkg", "util.go"),
		filepath.Join(tmpDir, "src", "pkg", "util.txt"),
	} {
		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
			fmt.Println(err)
			return
		}
		if err := os.WriteFile(path, []byte("package main\n"), 0o644); err != nil {
			fmt.Println(err)
			return
		}
	}

	if err := os.Chdir(tmpDir); err != nil {
		fmt.Println(err)
		return
	}

	matches, err := richglob.Glob(filepath.Join("src", "**", "*.go"), richglob.WithGlobStar())
	if err != nil {
		fmt.Println(err)
		return
	}

	for _, match := range matches {
		fmt.Println(filepath.ToSlash(match))
	}

}

Output:
src/main.go
src/pkg/util.go

func GlobSeq2

func GlobSeq2(pattern string, opts ...Option) iter.Seq2[string, error]

GlobSeq2 walks the filesystem lazily and yields matching paths.

The iterator yields `(path, nil)` pairs for matches. If pattern compilation, ignore-pattern compilation, or failglob processing encounters an error, the iterator yields a final `("", err)` pair and then stops. Matches are yielded in traversal order; unlike Glob, GlobSeq2 does not collect and globally sort results before emission.

Example

package main

import (
	"fmt"
	"os"
	"path/filepath"

	"go.dw1.io/richglob"
)

func main() {
	tmpDir, err := os.MkdirTemp("", "richglob-example-")
	if err != nil {
		fmt.Println(err)
		return
	}
	defer func() {
		if err := os.RemoveAll(tmpDir); err != nil {
			fmt.Println(err)
		}
	}()

	oldWD, err := os.Getwd()
	if err != nil {
		fmt.Println(err)
		return
	}
	defer func() {
		if err := os.Chdir(oldWD); err != nil {
			fmt.Println(err)
		}
	}()

	for _, path := range []string{
		filepath.Join(tmpDir, "src", "main.go"),
		filepath.Join(tmpDir, "src", "pkg", "util.go"),
		filepath.Join(tmpDir, "src", "pkg", "util.txt"),
	} {
		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
			fmt.Println(err)
			return
		}
		if err := os.WriteFile(path, []byte("package main\n"), 0o644); err != nil {
			fmt.Println(err)
			return
		}
	}

	if err := os.Chdir(tmpDir); err != nil {
		fmt.Println(err)
		return
	}

	for match, err := range richglob.GlobSeq2(filepath.Join("src", "**", "*.go"), richglob.WithGlobStar()) {
		if err != nil {
			fmt.Println(err)
			return
		}

		fmt.Println(filepath.ToSlash(match))
	}

}

Output:
src/main.go
src/pkg/util.go

func Match

func Match(pattern, name string, opts ...Option) (bool, error)

Match reports whether name matches pattern.

Pattern syntax is similar to filepath.Match, with optional extensions enabled through Option values.

Example

package main

import (
	"fmt"
	"path/filepath"

	"go.dw1.io/richglob"
)

func main() {
	matched, err := richglob.Match(
		filepath.FromSlash("src/**/main.go"),
		filepath.FromSlash("src/cmd/tool/main.go"),
		richglob.WithGlobStar(),
	)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Println(matched)
}

Output:
true

Types

type Option

type Option interface {
	// contains filtered or unexported methods
}

Option configures Match and Glob behavior.

func WithDotGlob

func WithDotGlob() Option

WithDotGlob includes hidden path entries when Bash pathname rules are active.

func WithExtGlob

func WithExtGlob() Option

WithExtGlob enables Bash extglob operators such as @(a|b) and !(tmp|cache).

func WithFailGlob

func WithFailGlob() Option

WithFailGlob makes Glob return ErrNoMatch when nothing matches.

func WithGitIgnore

func WithGitIgnore() Option

WithGitIgnore loads and applies .gitignore rules while walking directories.

func WithGlobASCIIRanges

func WithGlobASCIIRanges() Option

WithGlobASCIIRanges makes character ranges such as [A-Z] use ASCII ordering.

func WithGlobIgnore

func WithGlobIgnore(patterns ...string) Option

WithGlobIgnore removes matches that also match any of the supplied patterns.

func WithGlobSkipDots

func WithGlobSkipDots(enabled bool) Option

WithGlobSkipDots controls whether "." and ".." are skipped during Bash-style pathname expansion.

func WithGlobStar

func WithGlobStar() Option

WithGlobStar enables "**" to match zero or more directory segments.

func WithNoCaseGlob

func WithNoCaseGlob() Option

WithNoCaseGlob enables case-insensitive matching.

func WithNullGlob

func WithNullGlob() Option

WithNullGlob makes Glob return an empty, non-nil slice when nothing matches.

func WithSort

func WithSort(mode SortMode) Option

WithSort sets the ordering mode used by Glob.

type SortMode

type SortMode int

SortMode controls Glob result ordering.

const (
	// SortLexicographic sorts Glob results in ascending path order.
	SortLexicographic SortMode = iota
	// SortNone leaves Glob results in filesystem traversal order.
	SortNone
)