glob

README

glob.go

Go Globbing Library.

Install

    go get github.com/gobwas/glob

Example

package main

import "github.com/gobwas/glob"

func main() {
    var g *glob.Pattern

    // create simple glob
    g = glob.MustCompile("*.github.com")
    g.Match("api.github.com") // true

    // quote meta characters and then create simple glob
    g = glob.MustCompile(glob.QuoteMeta("*.github.com"))
    g.Match("*.github.com") // true

    // create new glob with set of delimiters as ["."]
    g = glob.MustCompile("api.*.com", '.')
    g.Match("api.github.com") // true
    g.Match("api.gi.hub.com") // false

    // create new glob with set of delimiters as ["."]
    // but now with super wildcard
    g = glob.MustCompile("api.**.com", '.')
    g.Match("api.github.com") // true
    g.Match("api.gi.hub.com") // true

    // create glob with single symbol wildcard
    g = glob.MustCompile("?at")
    g.Match("cat") // true
    g.Match("fat") // true
    g.Match("at") // false

    // create glob with single symbol wildcard and delimiters ['f']
    g = glob.MustCompile("?at", 'f')
    g.Match("cat") // true
    g.Match("fat") // false
    g.Match("at") // false

    // create glob with character-list matchers
    g = glob.MustCompile("[abc]at")
    g.Match("cat") // true
    g.Match("bat") // true
    g.Match("fat") // false
    g.Match("at") // false

    // create glob with character-list matchers
    g = glob.MustCompile("[!abc]at")
    g.Match("cat") // false
    g.Match("bat") // false
    g.Match("fat") // true
    g.Match("at") // false

    // create glob with character-range matchers
    g = glob.MustCompile("[a-c]at")
    g.Match("cat") // true
    g.Match("bat") // true
    g.Match("fat") // false
    g.Match("at") // false

    // create glob with character-range matchers
    g = glob.MustCompile("[!a-c]at")
    g.Match("cat") // false
    g.Match("bat") // false
    g.Match("fat") // true
    g.Match("at") // false

    // create glob with pattern-alternatives list
    g = glob.MustCompile("{cat,bat,[fr]at}")
    g.Match("cat") // true
    g.Match("bat") // true
    g.Match("fat") // true
    g.Match("rat") // true
    g.Match("at") // false
    g.Match("zat") // false
}

Compile reports malformed patterns with a *glob.SyntaxError carrying the byte offset and the reason:

_, err := glob.Compile("{a,b")
// err: glob: syntax error at 4: unclosed `{`

A compiled Pattern captures what it was compiled from, so it can be passed around instead of the raw arguments and inspected when needed (String() makes it a fmt.Stringer, like regexp.Regexp):

g := glob.MustCompile("*.github.com", '.')
g.String()     // "*.github.com"
g.Separators() // []rune{'.'}
g.Globstar()   // false

Compile covers the common case; the full set of options lives on Compiler, of which Compile is a shorthand (the way net.Dial is for net.Dialer.Dial):

c := glob.Compiler{
    Separators: []rune{'/'},
    Globstar:   true, // `**/` matches zero or more segments, as in the shells.
}
g, err := c.Compile("**/*.go")
g.Match("main.go")        // true
g.Match("cmd/app/main.go") // true

Syntax

Syntax is inspired by standard wildcards, with one addition: ** (the "super-asterisk"), which matches any sequence of characters including the separators, where * stops at them. Note that it is just that -- a * that crosses separators -- and not the **/ "globstar" of shells and file globbers: **/x requires the literal /, so it does not match x; use {**/,}x for that. The same applies to a ** between separators, e.g. a/**/b does not match a/b. See Globstar for the opt-in shell semantics.

pattern:
    { term }

term:
    `*`         matches any sequence of non-separator characters
    `**`        matches any sequence of characters
    `?`         matches any single non-separator character
    `[` [ `!` ] class `]`
                character class; `!` negates it
    `{` pattern-list `}`
                pattern alternatives
    c           matches character c (c != `*`, `**`, `?`, `\`, `[`, `{`, `}`)
    `\` c       matches character c

class:
    lo `-` hi   matches character c for lo <= c <= hi
    { c }       matches any of the listed characters (c != `\`, `]`;
                `\` c matches c, `-` is literal here); must be non-empty

pattern-list:
    pattern { `,` pattern }
                comma-separated (without spaces) patterns

Escaping

The backslash is the escape character: \* is a literal asterisk, and a backslash itself is \\. Mind the Go string literals: "foo\\bar" is the pattern foo\bar, which is the literal foobar, not foo\bar. To match a backslash (e.g. in the Windows paths) write "foo\\\\bar" or `foo\\bar`, or use QuoteMeta on the literal part.

Separators

The separators are not part of the pattern syntax -- they are configured once, at compilation time, as the extra arguments of Compile (or the Separators field of Compiler):

g := glob.MustCompile("api.*.com", '.', '/')

They only limit the wildcards: * and ? never match a separator, while ** matches across them; the literals and the character classes are not affected. With no separators given, * and ** are equivalent. A compiled *glob.Pattern keeps its separators for all matches -- to match the same pattern with different separators, compile it again.

Globstar

[!WARNING] By default ** is just a * that crosses the separators, not the **/ of shopt -s globstar, gitignore or most file globbers: **/x requires the / and does not match x, and a/**/b does not match a/b. If your patterns come from users who expect the shell behaviour, enable the option below.

The Globstar option of Compiler gives **/ the semantics of the shells and gitignore: a ** that is a whole segment and is followed by a separator matches zero or more segments.

c := glob.Compiler{Separators: []rune{'/'}, Globstar: true}
g, _ := c.Compile("a/**/b")
g.Match("a/b")     // true (false by default)
g.Match("a/x/y/b") // true

It is the same as writing {**/,} by hand, and only a ** that is a whole segment followed by a separator is affected: a**/b and a trailing a/** keep their meaning (a/** does not match a, as in gitignore). It needs the separators, which define the segments.

Performance

This library is created for compile-once patterns. This means, that compilation could take time, but strings matching is done faster, than in case when always parsing template.

If you will not use compiled *glob.Pattern object, and do g := glob.MustCompile(pattern); g.Match(...) every time, then your code will be much more slower.

Match performs zero allocations and is safe for concurrent use. Common pattern shapes (literals, prefixes, suffixes, substrings) are recognized at compile time and matched with plain string comparisons, and a trailing alternative group of such shapes (e.g. {*.png,*.jpg}) is tried as a plain loop over them; the backtracking engine behind the rest is differentially fuzzed against the regexp package (see FuzzMatchRegexp).

Run go test -bench=. from source root to see the benchmarks (the numbers below are from an Apple M4):

Pattern	Fixture	Match	Speed (ns/op)
[a-z][!a-x]*cat*[h][!b]*eyes*	my cat has very bright eyes	true	141
[a-z][!a-x]*cat*[h][!b]*eyes*	my dog has very bright eyes	false	46
https://*.google.*	https://account.google.com	true	16
https://*.google.*	https://google.com	false	13
{https://*.google.*,*yandex.*,*yahoo.*,*mail.ru}	http://yahoo.com	true	21
{https://*.google.*,*yandex.*,*yahoo.*,*mail.ru}	http://google.com	false	22
{https://*gobwas.com,http://exclude.gobwas.com}	https://safe.gobwas.com	true	7.7
{https://*gobwas.com,http://exclude.gobwas.com}	http://safe.gobwas.com	false	8.0
google.com	google.com	true	5.0
google.com	gobwas.com	false	3.9
abc*	abcdef	true	4.1
abc*	af	false	3.0
*def	abcdef	true	4.1
*def	af	false	2.9
ab*ef	abcdef	true	6.0
ab*ef	af	false	3.0

The same things with the regexp package -- not to pick on it (it is a general-purpose engine with much stronger guarantees), but as a reference for how the glob-shaped specialization pays off per pattern. The regular expressions are the exact equivalents: anchored, and with the s flag where there is a *, since a * matches a newline like any other character (see BenchmarkCompareGlobAndRegexp):

Pattern	Fixture	Match	Speed (ns/op)	glob is
(?s)^[a-z][^a-x].*cat.*[h][^b].*eyes.*$	my cat has very bright eyes	true	505	3.6x faster
(?s)^[a-z][^a-x].*cat.*[h][^b].*eyes.*$	my dog has very bright eyes	false	221	4.9x faster
(?s)^https://.*\.google\..*$	https://account.google.com	true	251	16x faster
(?s)^https://.*\.google\..*$	https://google.com	false	128	9.6x faster
(?s)^(https://.*\.google\..*|.*yandex\..*|.*yahoo\..*|.*mail\.ru)$	http://yahoo.com	true	396	19x faster
(?s)^(https://.*\.google\..*|.*yandex\..*|.*yahoo\..*|.*mail\.ru)$	http://google.com	false	558	25x faster
(?s)^(https://.*gobwas\.com|http://exclude\.gobwas\.com)$	https://safe.gobwas.com	true	210	27x faster
(?s)^(https://.*gobwas\.com|http://exclude\.gobwas\.com)$	http://safe.gobwas.com	false	46	5.7x faster
^google\.com$	google.com	true	25	5.0x faster
^google\.com$	gobwas.com	false	17	4.3x faster
(?s)^abc.*$	abcdef	true	43	10x faster
(?s)^abc.*$	af	false	1.5	2.0x slower
(?s)^.*def$	abcdef	true	73	18x faster
(?s)^.*def$	af	false	1.5	1.9x slower
(?s)^ab.*ef$	abcdef	true	77	13x faster
(?s)^ab.*ef$	af	false	1.5	2.0x slower

(The three slower rows are the tiny-mismatch cases. Both engines reject them with the same literal check; regexp just reaches it through less call overhead. In absolute terms it is 1.5ns vs 3ns -- negligible either way.)

Documentation

Index

• func QuoteMeta(s string) string
• type Compiler
  • func (c Compiler) Compile(str string) (*Pattern, error)
• type Pattern
  • func Compile(pattern string, separators ...rune) (*Pattern, error)
  • func MustCompile(pattern string, separators ...rune) *Pattern
  • func (p *Pattern) Globstar() bool
  • func (p *Pattern) Match(s string) bool
  • func (p *Pattern) Separators() []rune
  • func (p *Pattern) String() string
• type SyntaxError
  • func (s *SyntaxError) Error() string

Examples

• QuoteMeta

Functions

func QuoteMeta

func QuoteMeta(s string) string

QuoteMeta returns a string that escapes all glob meta characters inside s; the returned string is a glob pattern matching the literal text.

Characters that are special only in a particular context, such as `,` inside `{...}` or `-` and `!` inside `[...]`, are not escaped.

Example

s := "{foo*}"

Output:
\{foo\*\}

Types

type Compiler

type Compiler struct {
	// Separators are the characters `*` and `?` do not match; see [Compile].
	Separators []rune

	// Globstar makes `**/` match zero or more whole segments, as in bash with
	// `shopt -s globstar` or in gitignore: `**/x` matches `x` as well as
	// `a/b/x`, and `a/**/b` matches `a/b`.
	//
	// With no globstar, `**/x` requires the `/`, // and `{**/,}x` is the way
	// to spell this.
	//
	// Only a `**` that is a whole segment and is followed by a separator is
	// affected; `a**/b` and a trailing `a/**` keep their meaning.
	//
	// It has no effect without Separators, which define the segments.
	Globstar bool
}

Compiler holds the options a pattern is compiled with; see Compiler.Compile.

The zero value compiles patterns the way Compile does with no separators.

func (Compiler) Compile

func (c Compiler) Compile(str string) (*Pattern, error)

Compile compiles the glob pattern with the options of c; see Compile for the pattern syntax. A malformed pattern is reported with a *SyntaxError.

It parses the pattern into a matcher tree (see below), simplifies and specializes it, and computes the match-time hints and preconditions.

type Pattern

type Pattern struct {
	// contains filtered or unexported fields
}

Pattern represents a compiled glob pattern.

A pattern is compiled into a tree of matchers:

`a`      => "a"
`a*`     => ["a"·*]
`{a*,b}` => {["a"·*]|"b"}

Matching is a backtracking walk over that tree; see Pattern.Match.

func Compile

func Compile(pattern string, separators ...rune) (*Pattern, error)

Compile compiles the glob pattern. The separators, if given, are the characters `*` and `?` do not match (`**` does); they can not be changed after the compilation, see Pattern.Separators. A malformed pattern is reported with a *SyntaxError. Compile is a shorthand for Compiler.Compile with the Separators set and no other options, the way net.Dial is for net.Dialer.Dial.

The pattern syntax is:

pattern:
    { term }

term:
    `*`         matches any sequence of non-separator characters
    `**`        matches any sequence of characters
    `?`         matches any single non-separator character
    `[` [ `!` ] class `]`
                character class; `!` negates it
    `{` pattern-list `}`
                pattern alternatives
    c           matches character c (c != `*`, `**`, `?`, `\`, `[`, `{`, `}`)
    `\` c       matches character c

class:
    lo `-` hi   matches character c for lo <= c <= hi
    { c }       matches any of the listed characters (c != `\`, `]`;
                `\` c matches c, `-` is literal here); must be non-empty

pattern-list:
    pattern { `,` pattern }
                comma-separated (without spaces) patterns

func MustCompile

func MustCompile(pattern string, separators ...rune) *Pattern

MustCompile is the same as Compile, except that if Compile returns error, this will panic.

func (*Pattern) Globstar

func (p *Pattern) Globstar() bool

Globstar reports whether the pattern was compiled with Compiler.Globstar set. Like the separators, it is not part of String.

func (*Pattern) Match

func (p *Pattern) Match(s string) bool

Match reports whether s matches the pattern.

func (*Pattern) Separators

func (p *Pattern) Separators() []rune

Separators returns the separators the pattern was compiled with, in the order they were given to Compile; nil when there are none.

The returned slice is the very one given to Compile, sharing its backing array: it is not copied on the way in or out. Matching does not use it.

func (*Pattern) String

func (p *Pattern) String() string

String returns the source text used to compile the pattern, the same way regexp.Regexp.String does.

Note that separators are not part of String: they are given to Compile alongside the pattern text.

type SyntaxError

type SyntaxError struct {
	// Offset is a byte offset in the pattern.
	Offset int
	// Reason describes the error.
	Reason string
}

SyntaxError is returned by Compile when the given pattern can not be parsed. Offset points at the place in the pattern the error was detected at, so the tooling can do things like:

{a,b
----^ unclosed `{`

func (*SyntaxError) Error

func (s *SyntaxError) Error() string