fsnotify

package module
v1.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 25, 2023 License: BSD-3-Clause Imports: 9 Imported by: 0

README

fsnotify is a Go library to provide cross-platform filesystem notifications on Windows, Linux, macOS, BSD, and illumos.

Go 1.16 or newer is required; the full documentation is at https://pkg.go.dev/github.com/fsnotify/fsnotify


Platform support:

Backend OS Status
inotify Linux Supported
kqueue BSD, macOS Supported
ReadDirectoryChangesW Windows Supported
FEN illumos Supported in main branch
fanotify Linux 5.9+ Not yet
AHAFS AIX aix branch; experimental due to lack of maintainer and test environment
FSEvents macOS Needs support in x/sys/unix
USN Journals Windows Needs support in x/sys/windows
Polling All Not yet

Linux and illumos should include Android and Solaris, but these are currently untested.

Usage

A basic example:

package main

import (
    "log"

    "github.com/fsnotify/fsnotify"
)

func main() {
    // Create new watcher.
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        log.Fatal(err)
    }
    defer watcher.Close()

    // Start listening for events.
    go func() {
        for {
            select {
            case event, ok := <-watcher.Events:
                if !ok {
                    return
                }
                log.Println("event:", event)
                if event.Has(fsnotify.Write) {
                    log.Println("modified file:", event.Name)
                }
            case err, ok := <-watcher.Errors:
                if !ok {
                    return
                }
                log.Println("error:", err)
            }
        }
    }()

    // Add a path.
    err = watcher.Add("/tmp")
    if err != nil {
        log.Fatal(err)
    }

    // Block main goroutine forever.
    <-make(chan struct{})
}

Some more examples can be found in cmd/fsnotify, which can be run with:

% go run ./cmd/fsnotify

Further detailed documentation can be found in godoc: https://pkg.go.dev/github.com/fsnotify/fsnotify

FAQ

Will a file still be watched when it's moved to another directory?

No, not unless you are watching the location it was moved to.

Are subdirectories watched too?

No, you must add watches for any directory you want to watch (a recursive watcher is on the roadmap: #18).

Do I have to watch the Error and Event channels in a goroutine?

As of now, yes (you can read both channels in the same goroutine using select, you don't need a separate goroutine for both channels; see the example).

Why don't notifications work with NFS, SMB, FUSE, /proc, or /sys?

fsnotify requires support from underlying OS to work. The current NFS and SMB protocols does not provide network level support for file notifications, and neither do the /proc and /sys virtual filesystems.

This could be fixed with a polling watcher (#9), but it's not yet implemented.

Why do I get many Chmod events?

Some programs may generate a lot of attribute changes; for example Spotlight on macOS, anti-virus programs, backup applications, and some others are known to do this. As a rule, it's typically best to ignore Chmod events. They're often not useful, and tend to cause problems.

Spotlight indexing on macOS can result in multiple events (see #15). A temporary workaround is to add your folder(s) to the Spotlight Privacy settings until we have a native FSEvents implementation (see #11).

Platform-specific notes

Linux

When a file is removed a REMOVE event won't be emitted until all file descriptors are closed; it will emit a CHMOD instead:

fp := os.Open("file")
os.Remove("file")        // CHMOD
fp.Close()               // REMOVE

This is the event that inotify sends, so not much can be changed about this.

The fs.inotify.max_user_watches sysctl variable specifies the upper limit for the number of watches per user, and fs.inotify.max_user_instances specifies the maximum number of inotify instances per user. Every Watcher you create is an "instance", and every path you add is a "watch".

These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and /proc/sys/fs/inotify/max_user_instances

To increase them you can use sysctl or write the value to proc file:

# The default values on Linux 5.18
sysctl fs.inotify.max_user_watches=124983
sysctl fs.inotify.max_user_instances=128

To make the changes persist on reboot edit /etc/sysctl.conf or /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check your distro's documentation):

fs.inotify.max_user_watches=124983
fs.inotify.max_user_instances=128

Reaching the limit will result in a "no space left on device" or "too many open files" error.

kqueue (macOS, all BSD systems)

kqueue requires opening a file descriptor for every file that's being watched; so if you're watching a directory with five files then that's six file descriptors. You will run in to your system's "max open files" limit faster on these platforms.

The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to control the maximum number of open files.

Documentation

Overview

Package fsnotify provides a cross-platform interface for file system notifications.

Currently supported systems:

Linux 2.6.32+    via inotify
BSD, macOS       via kqueue
Windows          via ReadDirectoryChangesW
illumos          via FEN

Index

Constants

View Source
const (
	Create Op = 1 << iota
	// IN_ACCESS
	// IN_CREATE
	// MovedTO
	// IN_MOVED_TO
	// IN_DELETE_SELF
	// IN_DELETE
	// IN_MODIFY
	// The pathname was written to; this does *not* mean the write has finished,
	// and a write can be followed by more writes.
	Write

	// The path was removed; any watches on it will be removed. Some "remove"
	// operations may trigger a Rename if the file is actually moved (for
	// example "remove to trash" is often a rename).
	Remove

	// The path was renamed to something else; any watched on it will be
	// removed.
	Rename

	// File attributes were changed.
	//
	// It's generally not recommended to take action on this event, as it may
	// get triggered very frequently by some software. For example, Spotlight
	// indexing on macOS, anti-virus software, backup software, etc.
	Chmod
	IN_ACCESS        = 0x1
	IN_ALL_EVENTS    = 0xfff
	IN_ATTRIB        = 0x4
	IN_CLASSA_HOST   = 0xffffff
	IN_CLASSA_MAX    = 0x80
	IN_CLASSA_NET    = 0xff000000
	IN_CLASSA_NSHIFT = 0x18
	IN_CLASSB_HOST   = 0xffff
	IN_CLASSB_MAX    = 0x10000
	IN_CLASSB_NET    = 0xffff0000
	IN_CLASSB_NSHIFT = 0x10
	IN_CLASSC_HOST   = 0xff
	IN_CLASSC_NET    = 0xffffff00
	IN_CLASSC_NSHIFT = 0x8
	IN_CLOSE         = 0x18
	IN_CLOSE_NOWRITE = 0x10
	IN_CLOSE_WRITE   = 0x8
	IN_CREATE        = 0x100
	IN_DELETE        = 0x200
	IN_DELETE_SELF   = 0x400
	IN_DONT_FOLLOW   = 0x2000000
	IN_EXCL_UNLINK   = 0x4000000
	IN_IGNORED       = 0x8000
	IN_ISDIR         = 0x40000000
	IN_LOOPBACKNET   = 0x7f
	IN_MASK_ADD      = 0x20000000
	IN_MASK_CREATE   = 0x10000000
	IN_MODIFY        = 0x2
	IN_MOVE          = 0xc0
	IN_MOVED_FROM    = 0x40
	IN_MOVED_TO      = 0x80
	IN_MOVE_SELF     = 0x800
	IN_ONESHOT       = 0x80000000
	IN_ONLYDIR       = 0x1000000
	IN_OPEN          = 0x20
	IN_Q_OVERFLOW    = 0x4000
)

The operations fsnotify can trigger; see the documentation on Watcher for a full description, and check them with Event.Has.

Variables

View Source
var (
	ErrNonExistentWatch = errors.New("fsnotify: can't remove non-existent watcher")
	ErrEventOverflow    = errors.New("fsnotify: queue or buffer overflow")
	ErrClosed           = errors.New("fsnotify: watcher already closed")
)

Common errors that can be reported.

Functions

func GetDirNames

func GetDirNames(names []string) ([]string, error)

func WithBufferSize

func WithBufferSize(bytes int) addOpt

WithBufferSize sets the buffer size for the Windows backend. This is a no-op for other backends.

The default value is 64K (65536 bytes) which is the highest value that works on all filesystems and should be enough for most applications, but if you have a large burst of events it may not be enough. You can increase it if you're hitting "queue or buffer overflow" errors (ErrEventOverflow).

Types

type Event

type Event struct {
	// Path to the file or directory.
	//
	// Paths are relative to the input; for example with Add("dir") the Name
	// will be set to "dir/file" if you create that file, but if you use
	// Add("/path/to/dir") it will be "/path/to/dir/file".
	Name string

	// File operation that triggered the event.
	//
	// This is a bitmask and some systems may send multiple operations at once.
	// Use the Event.Has() method instead of comparing with ==.
	Op Op
}

Event represents a file system notification.

func (Event) Has

func (e Event) Has(op Op) bool

Has reports if this event has the given operation.

func (Event) String

func (e Event) String() string

String returns a string representation of the event with their path.

type Op

type Op uint32

Op describes a set of file operations.

func (Op) Has

func (o Op) Has(h Op) bool

Has reports if this operation has the given operation.

func (Op) String

func (o Op) String() string

type Watcher

type Watcher struct {
	// Events sends the filesystem change events.
	//
	// fsnotify can send the following events; a "path" here can refer to a
	// file, directory, symbolic link, or special file like a FIFO.
	//
	//   fsnotify.Create    A new path was created; this may be followed by one
	//                      or more Write events if data also gets written to a
	//                      file.
	//
	//   fsnotify.Remove    A path was removed.
	//
	//   fsnotify.Rename    A path was renamed. A rename is always sent with the
	//                      old path as Event.Name, and a Create event will be
	//                      sent with the new name. Renames are only sent for
	//                      paths that are currently watched; e.g. moving an
	//                      unmonitored file into a monitored directory will
	//                      show up as just a Create. Similarly, renaming a file
	//                      to outside a monitored directory will show up as
	//                      only a Rename.
	//
	//   fsnotify.Write     A file or named pipe was written to. A Truncate will
	//                      also trigger a Write. A single "write action"
	//                      initiated by the user may show up as one or multiple
	//                      writes, depending on when the system syncs things to
	//                      disk. For example when compiling a large Go program
	//                      you may get hundreds of Write events, so you
	//                      probably want to wait until you've stopped receiving
	//                      them (see the dedup example in cmd/fsnotify).
	//                      Some systems may send Write event for directories
	//                      when the directory content changes.
	//
	//   fsnotify.Chmod     Attributes were changed. On Linux this is also sent
	//                      when a file is removed (or more accurately, when a
	//                      link to an inode is removed). On kqueue it's sent
	//                      and on kqueue when a file is truncated. On Windows
	//                      it's never sent.
	Events chan Event

	// Errors sends any errors.
	//
	// [ErrEventOverflow] is used to indicate there are too many events:
	//
	//  - inotify: there are too many queued events (fs.inotify.max_queued_events sysctl)
	//  - windows: The buffer size is too small; [WithBufferSize] can be used to increase it.
	//  - kqueue, fen: not used.
	Errors chan error
	// contains filtered or unexported fields
}

Watcher watches a set of paths, delivering events on a channel.

A watcher should not be copied (e.g. pass it by pointer, rather than by value).

Linux notes

When a file is removed a Remove event won't be emitted until all file descriptors are closed, and deletes will always emit a Chmod. For example:

fp := os.Open("file")
os.Remove("file")        // Triggers Chmod
fp.Close()               // Triggers Remove

This is the event that inotify sends, so not much can be changed about this.

The fs.inotify.max_user_watches sysctl variable specifies the upper limit for the number of watches per user, and fs.inotify.max_user_instances specifies the maximum number of inotify instances per user. Every Watcher you create is an "instance", and every path you add is a "watch".

These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and /proc/sys/fs/inotify/max_user_instances

To increase them you can use sysctl or write the value to the /proc file:

# Default values on Linux 5.18
sysctl fs.inotify.max_user_watches=124983
sysctl fs.inotify.max_user_instances=128

To make the changes persist on reboot edit /etc/sysctl.conf or /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check your distro's documentation):

fs.inotify.max_user_watches=124983
fs.inotify.max_user_instances=128

Reaching the limit will result in a "no space left on device" or "too many open files" error.

kqueue notes (macOS, BSD)

kqueue requires opening a file descriptor for every file that's being watched; so if you're watching a directory with five files then that's six file descriptors. You will run in to your system's "max open files" limit faster on these platforms.

The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to control the maximum number of open files, as well as /etc/login.conf on BSD systems.

Windows notes

Paths can be added as "C:\path\to\dir", but forward slashes ("C:/path/to/dir") will also work.

The default buffer size is 64K, which is the largest value that is guaranteed to work with SMB filesystems. If you have many events in quick succession this may not be enough, and you will have to use WithBufferSize to increase the value.

func NewWatcher

func NewWatcher() (*Watcher, error)

NewWatcher creates a new Watcher.

func WatcherRecursivelyWithExclude

func WatcherRecursivelyWithExclude() (*Watcher, error)

func (*Watcher) Add

func (w *Watcher) Add(name string) error

Add starts monitoring the path for changes.

A path can only be watched once; watching it more than once is a no-op and will not return an error. Paths that do not yet exist on the filesystem cannot watched.

A watch will be automatically removed if the watched path is deleted or renamed. The exception is the Windows backend, which doesn't remove the watcher on renames.

Notifications on network filesystems (NFS, SMB, FUSE, etc.) or special filesystems (/proc, /sys, etc.) generally don't work.

Returns ErrClosed if Watcher.Close was called.

See [AddWith] for a version that allows adding options.

Watching directories

All files in a directory are monitored, including new files that are created after the watcher is started. By default subdirectories are not watched (i.e. it's non-recursive), but if the path ends with "/..." all files and subdirectories are watched too.

Watching files

Watching individual files (rather than directories) is generally not recommended as many tools update files atomically. Instead of "just" writing to the file a temporary file will be written to first, and if successful the temporary file is moved to to destination removing the original, or some variant thereof. The watcher on the original file is now lost, as it no longer exists.

Instead, watch the parent directory and use Event.Name to filter out files you're not interested in. There is an example of this in cmd/fsnotify/file.go.

func (*Watcher) AddWith

func (w *Watcher) AddWith(name string, opts ...addOpt) error

AddWith is like [Add], but allows adding options. When using Add() the defaults described below are used.

Possible options are:

  • WithBufferSize sets the buffer size for the Windows backend; no-op on other platforms. The default is 64K (65536 bytes).

func (*Watcher) Close

func (w *Watcher) Close() error

Close removes all watches and closes the events channel.

func (*Watcher) Remove

func (w *Watcher) Remove(name string) error

Remove stops monitoring the path for changes.

If the path was added as a recursive watch (e.g. as "/tmp/dir/...") then the entire recursive watch will be removed. You can use either "/tmp/dir" or "/tmp/dir/..." (they behave identically).

You cannot remove individual files or subdirectories from recursive watches; e.g. Add("/tmp/path/...") and then Remove("/tmp/path/sub") will fail.

For other watches directories are removed non-recursively. For example, if you added "/tmp/dir" and "/tmp/dir/subdir" then you will need to remove both.

Removing a path that has not yet been added returns ErrNonExistentWatch.

Returns nil if Watcher.Close was called.

func (*Watcher) WatchList

func (w *Watcher) WatchList() []string

WatchList returns all paths added with [Add] (and are not yet removed).

Returns nil if Watcher.Close was called.

Directories

Path Synopsis
cmd
fsnotify
Command fsnotify provides example usage of the fsnotify library.
Command fsnotify provides example usage of the fsnotify library.
Package internal contains some helpers.
Package internal contains some helpers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL