fsdedupe

README

fsdedupe

Utils to deduplicate local filesystem

go install github.com/mxmCherry/fsdedupe/cmd/fsdedupe@latest

find <SOMEDIR> -type f -not -path '*/.*' | fsdedupe symlink

Documentation

Index

• func DedupeSymlink(ctx context.Context, filenames Iterator) error
• type Iterator
  • func Lines(r io.Reader) Iterator

Functions

func DedupeSymlink

func DedupeSymlink(ctx context.Context, filenames Iterator) error

DedupeSymlink deduplicates input filenames by symlinking files to the first-seen file by SHA512 content hash.

Types

type Iterator

type Iterator interface {
	Next() (string, error)
}

Iterator defines a string (filename) iterator. It is expected to return io.EOF on no more entries.

func Lines

func Lines(r io.Reader) Iterator

Lines is an Iterator-adapter for an io.Reader (os.Stdin etc). It strips leading/trailing whitespaces and skips empty lines.