README
¶
rabin 
The rabin package implements Rabin hashing (aka fingerprinting) and content-defined chunking based on Rabin hashing.
Rabin hashing has the unusual property that it can efficiently compute a "rolling hash" of a stream of data, where the hash value reflects only the most recent w bytes of the stream, for some window size w. This property makes it ideal for "content-defined chunking", which sub-divides sequential data on boundaries that are robust to insertions and deletions.
The details of Rabin fingerprinting are described in Rabin, Michael (1981). "Fingerprinting by Random Polynomials." Center for Research in Computing Technology, Harvard University. Tech Report TR-CSE-03-01.
Installation
To download go-rabin, run
go get -d -u github.com/aclements/go-rabin/rabin
You can then import this package into your projects with
import "github.com/aclements/go-rabin/rabin"
Demos
There is a small program in cmd/cdsplit that divides an input file
into content-defined chunks. It's only intended as a demo, but can be
installed using
go get github.com/aclements/go-rabin/cmd/cdsplit
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
cdsplit
Command cdsplit divides a file into variable-sized, content-defined chunks that are robust to insertions, deletions, and changes to the input file.
|
Command cdsplit divides a file into variable-sized, content-defined chunks that are robust to insertions, deletions, and changes to the input file. |
|
Package rabin implements Rabin hashing (fingerprinting).
|
Package rabin implements Rabin hashing (fingerprinting). |