Documentation
¶
Overview ¶
Package filewatcher provides a go implementation of baseplate's FileWatcher: https://baseplate.readthedocs.io/en/stable/api/baseplate/lib/file_watcher.html
Example ¶
This example demonstrates how to use filewatcher.
package main
import (
"context"
"encoding/json"
"io"
"time"
"github.com/reddit/baseplate.go/filewatcher"
"github.com/reddit/baseplate.go/log"
)
func main() {
const (
// The path to the file.
path = "/opt/data.json"
// Timeout on the initial read.
timeout = time.Second * 30
)
// The type of the parsed data
type dataType map[string]interface{}
// Wrap a json decoder as parser
parser := func(f io.Reader) (interface{}, error) {
var data dataType
err := json.NewDecoder(f).Decode(&data)
return data, err
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
data, err := filewatcher.New(
ctx,
filewatcher.Config{
Path: path,
Parser: parser,
},
)
if err != nil {
log.Fatal(err)
}
getData := func() dataType {
return data.Get().(dataType)
}
// Whenever you need to use the parsed data, just call getData():
_ = getData()
}
Output:
Index ¶
Examples ¶
Constants ¶
const ( DefaultMaxFileSize = 10 << 20 HardLimitMultiplier = 10 )
DefaultMaxFileSize is the default MaxFileSize used when it's <= 0.
It's 10 MiB, with hard limit multiplier of 10.
const DefaultFSEventsDelay = 1 * time.Second
DefaultFSEventsDelay is the default FSEventsDelay used when creating a new FileWatcher.
const DefaultPollingInterval = 30 * time.Second
DefaultPollingInterval is the default PollingInterval used when creating a new FileWatcher.
Variables ¶
var InitialReadInterval = time.Second / 2
InitialReadInterval is the interval to keep retrying to open the file when creating a new FileWatcher, when the file was not initially available.
It's intentionally defined as a variable instead of constant, so that the caller can tweak its value when needed.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// The path to the file to be watched, required.
Path string `yaml:"path"`
// The parser to parse the data load, required.
Parser Parser
// Optional. When non-nil, it will be used to log errors,
// either returned by parser or by the underlying file system watcher.
// Please note that this does not include errors returned by the first parser
// call, which will be returned directly.
Logger log.Wrapper `yaml:"logger"`
// Optional. When <=0 DefaultMaxFileSize will be used instead.
//
// This is the soft limit,
// we will also auto add a hard limit which is 10x (see HardLimitMultiplier)
// of soft limit.
//
// If the soft limit is violated,
// the violation will be reported via log.DefaultWrapper and prometheus
// counter of limitopen_softlimit_violation_total,
// but it does not stop the normal parsing process.
//
// If the hard limit is violated,
// The loading of the file will fail immediately.
MaxFileSize int64 `yaml:"maxFileSize"`
// Optional, the interval to check file changes proactively.
//
// Default to DefaultPollingInterval.
// To disable polling completely, set it to a negative value.
//
// Without polling filewatcher relies solely on fs events from the parent
// directory. This works for most cases but will not work in the cases that
// the parent directory will be remount upon change
// (for example, k8s ConfigMap).
PollingInterval time.Duration `yaml:"pollingInterval"`
// Optional, the delay between receiving the fs events and actually reading
// and parsing the changes.
//
// It's used to avoid short bursts of fs events (for example, when watching a
// directory) causing reading and parsing repetively.
//
// Defaut to DefaultFSEventsDelay.
FSEventsDelay time.Duration `yaml:"fsEventsDelay"`
}
Config defines the config to be used in New function.
Can be deserialized from YAML.
type DirParser ¶ added in v0.9.12
A DirParser is a callback function that will be called when the watched directory has its content changed or is read for the first time.
Please note that a DirParser must return a consistent type. Inconsistent types will cause a panic, as does returning nil data and nil error.
Use WrapDirParser to wrap it into a Parser to be used with FileWatcher.
type FileWatcher ¶ added in v0.2.1
type FileWatcher interface {
// Get returns the latest, parsed data from the FileWatcher.
Get() any
// Stop stops the FileWatcher.
//
// After Stop is called you won't get any updates on the file content,
// but you can still call Get to get the last content before stopping.
//
// It's OK to call Stop multiple times.
// Calls after the first one are essentially no-op.
Stop()
}
FileWatcher loads and parses data from a file or directory, and watches for changes in order to refresh its stored data.
type MockDirWatcher ¶ added in v0.9.12
type MockDirWatcher struct {
// contains filtered or unexported fields
}
MockDirWatcher is an implementation of FileWatcher for testing with watching directories.
func NewMockDirWatcher ¶ added in v0.9.12
func NewMockDirWatcher(dir fs.FS, parser DirParser) (*MockDirWatcher, error)
NewMockDirWatcher creates a MockDirWatcher with the initial data and the given DirParser.
It provides Update function to update the data after it's been created.
func (*MockDirWatcher) Get ¶ added in v0.9.12
func (dw *MockDirWatcher) Get() any
Get implements FileWatcher by returning the last updated data.
type MockFileWatcher ¶ added in v0.2.1
type MockFileWatcher struct {
// contains filtered or unexported fields
}
MockFileWatcher is an implementation of FileWatcher that does not actually read from a file, it simply returns the data given to it when it was initialized with NewMockFilewatcher. It provides an additional Update method that allows you to update this data after it has been created.
func NewMockFilewatcher ¶ added in v0.2.1
func NewMockFilewatcher(r io.Reader, parser Parser) (*MockFileWatcher, error)
NewMockFilewatcher returns a pointer to a new MockFileWatcher object initialized with the given io.Reader and Parser.
func (*MockFileWatcher) Get ¶ added in v0.2.1
func (fw *MockFileWatcher) Get() any
Get returns the parsed data.
type Parser ¶
A Parser is a callback function to be called when a watched file has its content changed, or is read for the first time.
Please note that Parser should always return the consistent type. Inconsistent type will cause panic, as does returning nil data and nil error.
func WrapDirParser ¶ added in v0.9.12
WrapDirParser wraps a DirParser for a directory to a Parser.
When using FileWatcher to watch a directory instead of a single file, you MUST use WrapDirParser instead of any other Parser implementations.
type Result ¶
type Result struct {
// contains filtered or unexported fields
}
Result is the return type of New. Use Get function to get the actual data.
func New ¶
New creates a new FileWatcher.
If the path is not available at the time of calling, it blocks until the file becomes available, or context is cancelled, whichever comes first.
When logger is non-nil, it will be used to log errors, either returned by parser or by the underlying file system watcher. Please note that this does not include errors returned by the first parser call, which will be returned directly.
Source Files