Documentation
¶
Index ¶
- func File() string
- func Merge(dst, src1, src2 string)
- type Buffer
- func (b *Buffer) Align(n int)
- func (b *Buffer) Flush()
- func (b *Buffer) Offset() int
- func (b *Buffer) Write(x []byte)
- func (b *Buffer) WriteByte(x byte) error
- func (b *Buffer) WriteString(s string)
- func (b *Buffer) WriteTrigram(t uint32)
- func (b *Buffer) WriteUint(x int)
- func (b *Buffer) WriteVarint(x int)
- type CreateOption
- type Index
- func (ix *Index) Check() error
- func (ix *Index) Name(fileid int) Path
- func (ix *Index) Names(lo, hi int) iter.Seq[Path]
- func (ix *Index) NamesAt(min, max int) *PathReader
- func (ix *Index) PostingAnd(list []int, trigram uint32) []int
- func (ix *Index) PostingList(trigram uint32) []int
- func (ix *Index) PostingOr(list []int, trigram uint32) []int
- func (ix *Index) PostingQuery(q *Query) []int
- func (ix *Index) PrintStats()
- func (ix *Index) Roots() *PathReader
- type IndexWriter
- type Path
- type PathReader
- type PathWriter
- type Query
- type QueryOp
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Buffer ¶
type Buffer struct {
// contains filtered or unexported fields
}
A Buffer is a convenience wrapper: a closeable bufio.Writer.
func (*Buffer) WriteString ¶
func (*Buffer) WriteTrigram ¶
func (*Buffer) WriteVarint ¶
type CreateOption ¶ added in v1.1.0
type CreateOption func(*IndexWriter)
CreateOption is a functional option for configuring an IndexWriter.
func WithMaxFileLen ¶ added in v1.1.0
func WithMaxFileLen(n int64) CreateOption
WithMaxFileLen sets the maximum file length (in bytes) that will be indexed. Files longer than this are skipped. Default: 1<<30 (1 GB).
func WithMaxLineLen ¶ added in v1.1.0
func WithMaxLineLen(n int) CreateOption
WithMaxLineLen sets the maximum line length (in bytes) that will be indexed. Files containing lines longer than this are skipped. Default: 2000.
func WithMaxTextTrigrams ¶ added in v1.1.0
func WithMaxTextTrigrams(n int) CreateOption
WithMaxTextTrigrams sets the maximum number of distinct trigrams a file may contain to be indexed. Files exceeding this are assumed to be non-text and skipped. Default: 20000.
func WithZipExtensions ¶ added in v1.1.0
func WithZipExtensions(exts ...string) CreateOption
WithZipExtensions configures which file extensions should be treated as zip archives and have their contents indexed. Each extension must include the leading dot (e.g., ".zip", ".slx", ".jar"); it panics if any extension does not start with ".".
type Index ¶
type Index struct {
Verbose bool
// contains filtered or unexported fields
}
An Index implements read-only access to a trigram index.
func (*Index) NamesAt ¶
func (ix *Index) NamesAt(min, max int) *PathReader
NameAt returns a PathReader returning the names for fileids in the range [min, max).
func (*Index) PostingList ¶
func (*Index) PostingQuery ¶
func (*Index) PrintStats ¶
func (ix *Index) PrintStats()
type IndexWriter ¶
type IndexWriter struct {
LogSkip bool // log information about skipped files
Verbose bool // log status using package log
Zip bool // index content of zip files (shorthand for WithZipExtensions(".zip"))
// contains filtered or unexported fields
}
An IndexWriter creates an on-disk index corresponding to a set of files.
func Create ¶
func Create(file string, opts ...CreateOption) *IndexWriter
Create returns a new IndexWriter that will write the index to file. Options can be provided to override default tuning parameters.
func (*IndexWriter) Add ¶
func (ix *IndexWriter) Add(name string, f io.Reader) error
Add adds the file f to the index under the given name. It logs errors using package log.
func (*IndexWriter) AddFile ¶
func (ix *IndexWriter) AddFile(name string) error
AddFile adds the file with the given name (opened using os.Open) to the index. It logs errors using package log.
func (*IndexWriter) AddRoots ¶
func (ix *IndexWriter) AddRoots(roots []Path)
AddRoots adds the given roots to the index's list of roots.
func (*IndexWriter) Flush ¶
func (ix *IndexWriter) Flush()
Flush flushes the index entry to the target file.
type Path ¶
type Path struct {
// contains filtered or unexported fields
}
A Path is a Path stored in the index, either in the root list or the file list.
Paths stored in the index are ordered using the [Path.cmp] method.
func (Path) Compare ¶
Compare returns the comparison of p and q. It is analogous to strings.Compare(p, q) but x/y is ordered before x.foo by treating slashes as if they had byte value 0. On Windows, backslashes are treated as equal to slashes.
func (Path) HasPathPrefix ¶
type PathReader ¶
type PathReader struct {
// contains filtered or unexported fields
}
func NewPathReader ¶
func NewPathReader(version int, data []byte, limit int) *PathReader
func (*PathReader) Next ¶
func (r *PathReader) Next() bool
func (*PathReader) NumPaths ¶
func (r *PathReader) NumPaths() int
func (*PathReader) Path ¶
func (r *PathReader) Path() Path
func (*PathReader) Valid ¶
func (r *PathReader) Valid() bool
type PathWriter ¶
type PathWriter struct {
// contains filtered or unexported fields
}
func NewPathWriter ¶
func NewPathWriter(data, index *Buffer, version, group int) *PathWriter
func (*PathWriter) Collect ¶
func (w *PathWriter) Collect(paths iter.Seq[Path])
Collect iterates over paths and writes all the paths to w.
func (*PathWriter) Count ¶
func (w *PathWriter) Count() int
Count returns the number of paths written to w.
func (*PathWriter) Write ¶
func (w *PathWriter) Write(p Path)
type Query ¶
A Query is a matching machine, like a regular expression, that matches some text and not other text. When we compute a Query from a regexp, the Query is a conservative version of the regexp: it matches everything the regexp would match, and probably quite a bit more. We can then filter target files by whether they match the Query (using a trigram index) before running the comparatively more expensive regexp machinery.
func RegexpQuery ¶
RegexpQuery returns a Query for the given regexp.
Source Files