filter

package
v1.68.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 15, 2024 License: MIT Imports: 12 Imported by: 39

Documentation

Overview

Package filter controls the filtering of files

Index

Constants

This section is empty.

Variables

View Source 
var Opt = Options{
	MinAge:  fs.DurationOff,
	MaxAge:  fs.DurationOff,
	MinSize: fs.SizeSuffix(-1),
	MaxSize: fs.SizeSuffix(-1),
}

Opt is the default config for the filter

View Source 
var OptionsInfo = fs.Options{{
	Name:    "delete_excluded",
	Default: false,
	Help:    "Delete files on dest excluded from sync",
	Groups:  "Filter",
}, {
	Name:    "exclude_if_present",
	Default: []string{},
	Help:    "Exclude directories if filename is present",
	Groups:  "Filter",
}, {
	Name:    "files_from",
	Default: []string{},
	Help:    "Read list of source-file names from file (use - to read from stdin)",
	Groups:  "Filter",
}, {
	Name:    "files_from_raw",
	Default: []string{},
	Help:    "Read list of source-file names from file without any processing of lines (use - to read from stdin)",
	Groups:  "Filter",
}, {
	Name:    "min_age",
	Default: fs.DurationOff,
	Help:    "Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y",
	Groups:  "Filter",
}, {
	Name:    "max_age",
	Default: fs.DurationOff,
	Help:    "Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y",
	Groups:  "Filter",
}, {
	Name:    "min_size",
	Default: fs.SizeSuffix(-1),
	Help:    "Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P",
	Groups:  "Filter",
}, {
	Name:    "max_size",
	Default: fs.SizeSuffix(-1),
	Help:    "Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P",
	Groups:  "Filter",
}, {
	Name:    "ignore_case",
	Default: false,
	Help:    "Ignore case in filters (case insensitive)",
	Groups:  "Filter",
}, {
	Name:     "filter",
	Default:  []string{},
	ShortOpt: "f",
	Help:     "Add a file filtering rule",
	Groups:   "Filter",
}, {
	Name:    "filter_from",
	Default: []string{},
	Help:    "Read file filtering patterns from a file (use - to read from stdin)",
	Groups:  "Filter",
}, {
	Name:    "exclude",
	Default: []string{},
	Help:    "Exclude files matching pattern",
	Groups:  "Filter",
}, {
	Name:    "exclude_from",
	Default: []string{},
	Help:    "Read file exclude patterns from file (use - to read from stdin)",
	Groups:  "Filter",
}, {
	Name:    "include",
	Default: []string{},
	Help:    "Include files matching pattern",
	Groups:  "Filter",
}, {
	Name:    "include_from",
	Default: []string{},
	Help:    "Read file include patterns from file (use - to read from stdin)",
	Groups:  "Filter",
}, {
	Name:    "metadata_filter",
	Default: []string{},
	Help:    "Add a metadata filtering rule",
	Groups:  "Filter,Metadata",
}, {
	Name:    "metadata_filter_from",
	Default: []string{},
	Help:    "Read metadata filtering patterns from a file (use - to read from stdin)",
	Groups:  "Filter,Metadata",
}, {
	Name:    "metadata_exclude",
	Default: []string{},
	Help:    "Exclude metadatas matching pattern",
	Groups:  "Filter,Metadata",
}, {
	Name:    "metadata_exclude_from",
	Default: []string{},
	Help:    "Read metadata exclude patterns from file (use - to read from stdin)",
	Groups:  "Filter,Metadata",
}, {
	Name:    "metadata_include",
	Default: []string{},
	Help:    "Include metadatas matching pattern",
	Groups:  "Filter,Metadata",
}, {
	Name:    "metadata_include_from",
	Default: []string{},
	Help:    "Read metadata include patterns from file (use - to read from stdin)",
	Groups:  "Filter,Metadata",
}}

OptionsInfo describes the Options in use

Functions

func CopyConfig added in v1.55.0 

func CopyConfig(dstCtx, srcCtx context.Context) context.Context

CopyConfig copies the global config (if any) from srcCtx into dstCtx returning the new context.

func GetUseFilter added in v1.57.0 

func GetUseFilter(ctx context.Context) bool

GetUseFilter obtains the "use filter" flag from context The flag tells filter-aware backends (Drive) to constrain List using filter

func GlobPathToRegexp added in v1.68.0 

func GlobPathToRegexp(glob string, ignoreCase bool) (*regexp.Regexp, error)

GlobPathToRegexp converts an rsync style glob path to a regexp

func GlobStringToRegexp added in v1.68.0 

func GlobStringToRegexp(glob string, addAnchors bool, ignoreCase bool) (*regexp.Regexp, error)

GlobStringToRegexp converts an rsync style glob string to a regexp

Without adding of anchors but with ignoring of case, i.e. called `GlobStringToRegexp(glob, false, true)`, it takes a lenient approach where the glob "sum" would match "CheckSum", more similar to text search functions than strict glob filtering.

With adding of anchors and not ignoring case, i.e. called `GlobStringToRegexp(glob, true, false)`, it uses a strict glob interpretation where the previous example would have to be changed to "*Sum" to match "CheckSum".

func Reload added in v1.68.0 

func Reload(ctx context.Context) (err error)

Reload the filters from the flags

func ReplaceConfig added in v1.54.0 

func ReplaceConfig(ctx context.Context, f *Filter) context.Context

ReplaceConfig replaces the filter config in the ctx with the one passed in and returns a new context with that added to it.

func SetUseFilter added in v1.57.0 

func SetUseFilter(ctx context.Context, useFilter bool) context.Context

SetUseFilter returns a context having (re)set the "use filter" flag

Types

type FilesMap 

type FilesMap map[string]struct{}

FilesMap describes the map of files to transfer

type Filter 

type Filter struct {
	Opt         Options
	ModTimeFrom time.Time
	ModTimeTo   time.Time
	// contains filtered or unexported fields
}

Filter describes any filtering in operation

func AddConfig added in v1.54.0 

func AddConfig(ctx context.Context) (context.Context, *Filter)

AddConfig returns a mutable config structure based on a shallow copy of that found in ctx and returns a new context with that added to it.

func GetConfig added in v1.54.0 

func GetConfig(ctx context.Context) *Filter

GetConfig returns the global or context sensitive config

func NewFilter 

func NewFilter(opt *Options) (f *Filter, err error)

NewFilter parses the command line options and creates a Filter object. If opt is nil, then DefaultOpt will be used

func (*Filter) Add 

func (f *Filter) Add(Include bool, glob string) error

Add adds a filter rule with include or exclude status indicated

func (*Filter) AddFile 

func (f *Filter) AddFile(file string) error

AddFile adds a single file to the files from list

func (*Filter) AddRule 

func (f *Filter) AddRule(rule string) error

AddRule adds a filter rule with include/exclude indicated by the prefix

These are

  • glob
  • glob !

'+' includes the glob, '-' excludes it and '!' resets the filter list

Line comments may be introduced with '#' or ';'

func (*Filter) Clear 

func (f *Filter) Clear()

Clear clears all the filter rules

func (*Filter) DirContainsExcludeFile 

func (f *Filter) DirContainsExcludeFile(ctx context.Context, fremote fs.Fs, remote string) (bool, error)

DirContainsExcludeFile checks if exclude file is present in a directory. If fs is nil, it works properly if ExcludeFile is an empty string (for testing).

func (*Filter) DumpFilters 

func (f *Filter) DumpFilters() string

DumpFilters dumps the filters in textual form, 1 per line

func (*Filter) Files 

func (f *Filter) Files() FilesMap

Files returns all the files from the `--files-from` list

It may be nil if the list is empty

func (*Filter) HaveFilesFrom 

func (f *Filter) HaveFilesFrom() bool

HaveFilesFrom returns true if --files-from has been supplied

func (*Filter) InActive 

func (f *Filter) InActive() bool

InActive returns false if any filters are active

func (*Filter) Include 

func (f *Filter) Include(remote string, size int64, modTime time.Time, metadata fs.Metadata) bool

Include returns whether this object should be included into the sync or not and logs the reason for exclusion if not included

func (*Filter) IncludeDirectory 

func (f *Filter) IncludeDirectory(ctx context.Context, fs fs.Fs) func(string) (bool, error)

IncludeDirectory returns a function which checks whether this directory should be included in the sync or not.

func (*Filter) IncludeObject 

func (f *Filter) IncludeObject(ctx context.Context, o fs.Object) bool

IncludeObject returns whether this object should be included into the sync or not. This is a convenience function to avoid calling o.ModTime(), which is an expensive operation.

func (*Filter) IncludeRemote added in v1.56.0 

func (f *Filter) IncludeRemote(remote string) bool

IncludeRemote returns whether this remote passes the filter rules.

func (*Filter) ListContainsExcludeFile 

func (f *Filter) ListContainsExcludeFile(entries fs.DirEntries) bool

ListContainsExcludeFile checks if exclude file is present in the list.

func (*Filter) MakeListR 

func (f *Filter) MakeListR(ctx context.Context, NewObject func(ctx context.Context, remote string) (fs.Object, error)) fs.ListRFn

MakeListR makes function to return all the files set using --files-from

func (*Filter) UsesDirectoryFilters 

func (f *Filter) UsesDirectoryFilters() bool

UsesDirectoryFilters returns true if the filter uses directory filters and false if it doesn't.

This is used in deciding whether to walk directories or use ListR

type Options added in v1.68.0 

type Options struct {
	DeleteExcluded bool          `config:"delete_excluded"`
	RulesOpt                     // embedded so we don't change the JSON API
	ExcludeFile    []string      `config:"exclude_if_present"`
	FilesFrom      []string      `config:"files_from"`
	FilesFromRaw   []string      `config:"files_from_raw"`
	MetaRules      RulesOpt      `config:"metadata"`
	MinAge         fs.Duration   `config:"min_age"`
	MaxAge         fs.Duration   `config:"max_age"`
	MinSize        fs.SizeSuffix `config:"min_size"`
	MaxSize        fs.SizeSuffix `config:"max_size"`
	IgnoreCase     bool          `config:"ignore_case"`
}

Options configures the filter

type RulesOpt added in v1.61.0 

type RulesOpt struct {
	FilterRule  []string `config:"filter"`
	FilterFrom  []string `config:"filter_from"`
	ExcludeRule []string `config:"exclude"`
	ExcludeFrom []string `config:"exclude_from"`
	IncludeRule []string `config:"include"`
	IncludeFrom []string `config:"include_from"`
}

RulesOpt is configuration for a rule set

Source Files

View all Source files

Directories

Path Synopsis
Package filterflags implements command line flags to set up a filter
 Package filterflags implements command line flags to set up a filter

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.