rulelist

Documentation

Overview

Package rulelist contains the implementation of the standard rule-list filter that wraps an urlfilter filtering-engine.

TODO(a.garipov): Add a new update worker.

Index

• Constants
• type Engine
  • func NewEngine(c *EngineConfig) (e *Engine)
  • func (e *Engine) Close() (err error)
  • func (e *Engine) FilterRequest(req *urlfilter.DNSRequest) (res *urlfilter.DNSResult, hasMatched bool)
  • func (e *Engine) Refresh(ctx context.Context, parseBuf []byte, cli *http.Client, cacheDir string, ...) (err error)
• type EngineConfig
• type Filter
  • func NewFilter(c *FilterConfig) (f *Filter, err error)
  • func (f *Filter) Close() (err error)
  • func (f *Filter) Refresh(ctx context.Context, parseBuf []byte, cli *http.Client, cacheDir string, ...) (parseRes *ParseResult, err error)
• type FilterConfig
• type ParseResult
• type Parser
  • func NewParser() (p *Parser)
  • func (p *Parser) Parse(dst io.Writer, src io.Reader, buf []byte) (r *ParseResult, err error)
• type TextEngine
  • func NewTextEngine(c *TextEngineConfig) (e *TextEngine, err error)
  • func (e *TextEngine) Close() (err error)
  • func (e *TextEngine) FilterRequest(req *urlfilter.DNSRequest) (res *urlfilter.DNSResult, hasMatched bool)
• type TextEngineConfig
• type UID
  • func MustNewUID() (uid UID)
  • func NewUID() (uid UID, err error)
  • func (id UID) String() (s string)
• type URLFilterID

Constants

const DefaultMaxRuleListSize = 64 * datasize.MB

DefaultMaxRuleListSize is the default maximum filtering-rule list size.

const DefaultRuleBufSize = 1024

DefaultRuleBufSize is the default length of a buffer used to read a line with a filtering rule, in bytes.

TODO(a.garipov): Consider using datasize.ByteSize. It is currently only used as an int.

const ErrHTML errors.Error = "data is HTML, not plain text"

ErrHTML is returned by Parser.Parse if the data is likely to be HTML.

TODO(a.garipov): This error is currently returned to the UI. Stop that and make it all-lowercase.

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is a single DNS filter based on one or more rule lists. This structure contains the filtering engine combining several rule lists.

TODO(a.garipov): Merge with TextEngine in some way?

func NewEngine

func NewEngine(c *EngineConfig) (e *Engine)

NewEngine returns a new rule-list filtering engine. The engine is not refreshed, so a refresh should be performed before use.

func (*Engine) Close

func (e *Engine) Close() (err error)

Close closes the underlying rule-list engine as well as the rule lists.

func (*Engine) FilterRequest

func (e *Engine) FilterRequest(
	req *urlfilter.DNSRequest,
) (res *urlfilter.DNSResult, hasMatched bool)

FilterRequest returns the result of filtering req using the DNS filtering engine.

func (*Engine) Refresh

func (e *Engine) Refresh(
	ctx context.Context,
	parseBuf []byte,
	cli *http.Client,
	cacheDir string,
	maxSize datasize.ByteSize,
) (err error)

Refresh updates all rule lists in e. ctx is used for cancellation. parseBuf, cli, cacheDir, and maxSize are used for updates of rule-list filters; see Filter.Refresh.

TODO(a.garipov): Unexport and test in an internal test or through enigne tests.

type EngineConfig

type EngineConfig struct {
	// Name is the human-readable name of this engine, like "allowed",
	// "blocked", or "custom".
	Name string

	// Filters is the data about rule lists in this engine.  There must be no
	// other references to the elements of this slice.
	Filters []*Filter
}

EngineConfig is the configuration for rule-list filtering engines created by combining refreshable filters.

type Filter

type Filter struct {
	// contains filtered or unexported fields
}

Filter contains information about a single rule-list filter.

TODO(a.garipov): Use.

func NewFilter

func NewFilter(c *FilterConfig) (f *Filter, err error)

NewFilter creates a new rule-list filter. The filter is not refreshed, so a refresh should be performed before use.

func (*Filter) Close

func (f *Filter) Close() (err error)

Close closes the underlying rule list.

func (*Filter) Refresh

func (f *Filter) Refresh(
	ctx context.Context,
	parseBuf []byte,
	cli *http.Client,
	cacheDir string,
	maxSize datasize.ByteSize,
) (parseRes *ParseResult, err error)

Refresh updates the data in the rule-list filter. parseBuf is the initial buffer used to parse information from the data. cli and maxSize are only used when f is a URL-based list.

TODO(a.garipov): Unexport and test in an internal test or through enigne tests.

TODO(a.garipov): Consider not returning parseRes.

type FilterConfig

type FilterConfig struct {
	// URL is the URL of this rule-list filter.  Supported schemes are:
	//   - http
	//   - https
	//   - file
	URL *url.URL

	// Name is the human-readable name of this rule-list filter.  If not set, it
	// is either taken from the rule-list data or generated synthetically from
	// the UID.
	Name string

	// UID is the unique ID of this rule-list filter.
	UID UID

	// URLFilterID is used for working with package urlfilter.
	URLFilterID URLFilterID

	// Enabled, if true, means that this rule-list filter is used for filtering.
	Enabled bool
}

FilterConfig contains the configuration for a Filter.

type ParseResult

type ParseResult struct {
	// Title is the title contained within the filtering-rule list, if any.
	Title string

	// RulesCount is the number of rules in the list.  It excludes empty lines
	// and comments.
	RulesCount int

	// BytesWritten is the number of bytes written to dst.
	BytesWritten int

	// Checksum is the CRC-32 checksum of the rules content.  That is, excluding
	// empty lines and comments.
	Checksum uint32
}

ParseResult contains information about the results of parsing a filtering-rule list by Parser.Parse.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser is a filtering-rule parser that collects data, such as the checksum and the title, as well as counts rules and removes comments.

func NewParser

func NewParser() (p *Parser)

NewParser returns a new filtering-rule parser.

func (*Parser) Parse

func (p *Parser) Parse(dst io.Writer, src io.Reader, buf []byte) (r *ParseResult, err error)

Parse parses data from src into dst using buf during parsing. r is never nil.

type TextEngine

type TextEngine struct {
	// contains filtered or unexported fields
}

TextEngine is a single DNS filter based on a list of rules in text form.

func NewTextEngine

func NewTextEngine(c *TextEngineConfig) (e *TextEngine, err error)

NewTextEngine returns a new rule-list filtering engine that uses rules directly. The engine is ready to use and should not be refreshed.

func (*TextEngine) Close

func (e *TextEngine) Close() (err error)

Close closes the underlying rule list engine as well as the rule lists.

func (*TextEngine) FilterRequest

func (e *TextEngine) FilterRequest(
	req *urlfilter.DNSRequest,
) (res *urlfilter.DNSResult, hasMatched bool)

FilterRequest returns the result of filtering req using the DNS filtering engine.

type TextEngineConfig

type TextEngineConfig struct {
	// Name is the human-readable name of this engine, like "allowed",
	// "blocked", or "custom".
	Name string

	// Rules is the text of the filtering rules for this engine.
	Rules []string

	// ID is the ID to use inside a URL-filter engine.
	ID URLFilterID
}

TextEngineConfig is the configuration for a rule-list filtering engine created from a filtering rule text.

type UID

type UID uuid.UUID

UID is the type for the unique IDs of filtering-rule lists.

func MustNewUID

func MustNewUID() (uid UID)

MustNewUID is a wrapper around NewUID that panics if there is an error.

func NewUID

func NewUID() (uid UID, err error)

NewUID returns a new filtering-rule list UID. Any error returned is an error from the cryptographic randomness reader.

func (UID) String

func (id UID) String() (s string)

String implements the fmt.Stringer interface for UID.

type URLFilterID

type URLFilterID = int

URLFilterID is a semantic type-alias for IDs used for working with package urlfilter.

const (
	URLFilterIDCustom          URLFilterID = 0
	URLFilterIDEtcHosts        URLFilterID = -1
	URLFilterIDBlockedService  URLFilterID = -2
	URLFilterIDParentalControl URLFilterID = -3
	URLFilterIDSafeBrowsing    URLFilterID = -4
	URLFilterIDSafeSearch      URLFilterID = -5
)

The IDs of built-in filter lists.

NOTE: Do not change without the need for it and keep in sync with client/src/helpers/constants.js.

TODO(a.garipov): Add type URLFilterID once it is used consistently in package filtering.

TODO(d.kolyshev): Add URLFilterIDLegacyRewrite here and to the UI.