Version: v0.8.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2022 License: MIT Imports: 10 Imported by: 3



This subdirectory contains an older version of the c4 id package this is deprecated in favor of The newer package is much simpler and faster then this one, which is only kept for now to provide backward compatability.



## C4 ID

The c4 id package implements the C4 ID standard. C4 IDs are base 58 encoded SHA-512 hashes. C4 IDs represent a universally unique and consistent identification system for all observers world wide without the need for a central registry. The standardized encoding makes the IDs easer to use in common contexts like filenames, or keys in an object store.

A single C4 IDs may represent one or more files (or bocks of data), by hashing the contents of the file(s). This makes identifying collections of files much easer. C4 IDs are not effect by filename, location, modification time or other metadata.

### Types

This implementation exports the basic types ID and Digest, and Slice collections for both. The Slice types support an Insert method to insure proper ordering for identification of a collection.

### Single File Identification

To create a C4 ID for a file (or any block of contiguous data) there are two interfaces.

#### Encoder

Encoder implements io.Writer and is best for identifying streaming data without having to hold all the data in ram at once.

e := c4.NewEncoder()
io.Copy(e, src)
id := e.ID()

Every time ID() is called the ID will be the data written to the encoder so for. The Encoder can be cleared and re-used for different data by calling Reset().

#### Identify

The second interface for identifying contiguous data is the Identify method. It takes an io.Reader and returns a C4 ID, if there is an error other than io.EOF from the Reader Identify will return nil.

### Multiple File Identification

For multiple files or other types of non-contiguous data first using the process above generate the C4 ID of each file in the collection and use the Insert method to add them to a Slice, or DigestSlice. Insert insures the proper order for identification.

For a Slice the ID() method returns the C4 ID of the Slice. For DigestSlice the Digest method returns the C4 Digest.

    var digests c4.DigestSlice
    for digest := range inputDigests {
     collectionDigest := digests.Digest()
     collectionID := collectionDigest.ID()

### Parsing IDs

To parse an ID string, use the Parse function.




This section is empty.


This section is empty.


This section is empty.


type Digest

type Digest []byte

Digest represents a 64 byte "C4 Digest", which is the SHA-512 hash. Amongst other things Digest enforces padding to insure alignment with the original 64 byte hash.

A digest is simply a slice of bytes and can be use wherever the raw SHA hash might be needed.

func NewDigest

func NewDigest(b []byte) Digest

NewDigest creates a Digest and initializes it with the argument, enforcing byte alignment by padding with 0 (zero) if needed. NewDigest will return nil if the argument is larger then 64 bytes. For performance NewDigest may not copy the data to a new slice, so copies must be made explicitly when needed.

func (Digest) ID

func (d Digest) ID() *ID

ID returns the C4 ID representation of the digest by directly translating the byte slice to the standard C4 ID format (the bytes are not (re)hashed).

func (Digest) Read

func (d Digest) Read(p []byte) (n int, err error)

Read implements the io.Reader interface. It reads exactly 64 bytes into p. Read only returns an error if p is less than 64 bytes, in which case it returns io.EOF, without reading any bytes.

func (Digest) Sum

func (l Digest) Sum(r Digest) Digest

Sum returns the digest of the receiver and argument combined. Insuring proper order. C4 Digests of C4 Digests are always identified by concatenating the bytes of the larger digest after the bytes of the lesser digest to form a block of 128 bytes which are then IDed.

func (Digest) Write

func (d Digest) Write(p []byte) (n int, err error)

Write implements the io.Writer interface. It writes exactly 64 bytes replacing the value of the digest. The bytes must be a valid c4 Digest (i.e. sha-512 hash), any other value and the behavior of Write is undefined. Write only return an error if less than 64 bytes of input are available, in which case it returns io.EOF, without writing any bytes.

type DigestSlice

type DigestSlice []Digest

A DigestSlice represents a sorted list of unique Digests, and can be used to compute a C4 Digest for any set of non-contiguous data such as files in a folder.

func (*DigestSlice) Digest

func (ds *DigestSlice) Digest() Digest

Digest returns the Digest of the slice, or nil if the slice is empty. The Digest is computed by identifying successive pairs of Digests from the slice and iterating across each new list of digest repeating the process until only a single ID remains which is the ID returned as the C4 ID of the items in the slice.

func (*DigestSlice) Index

func (ds *DigestSlice) Index(x Digest) int

Index returns the location of x in the DigestSlice, or the index at which x would be inserted into the slice if it is not in the set.

func (*DigestSlice) Insert

func (ds *DigestSlice) Insert(d Digest) int

Insert adds a Digest to the slice in sorted order. Insert has no effect if the argument is nil, or is already a member of the slice. When successful Insert returns the insertion index. If d is nil then Insert will return -1. If d is already in the slice then Insert will return the index as a negative minus 1.

For example if d is already item 5 of the slice Insert will return -6. A return value of -1 means d was inserted at position 0 if d is not nil.

func (*DigestSlice) Read

func (ds *DigestSlice) Read(p []byte) (int, error)

Read implements the io.Reader interface to output the bytes of the DigestSlice. Read returns an error if p is not large enough to hold the entire DigestSlice (64 * it's length). The output of Read is the most compact form of the DigestSlice and cannot be compressed further.

func (*DigestSlice) Write

func (s *DigestSlice) Write(p []byte) (int, error)

Write implements the io.Writer interface to input the bytes of a serialized DigestSlice. Write returns an error and does not write any data if p is not a multiple of 64 in length.

type Encoder

type Encoder struct {
	// contains filtered or unexported fields

Encoder generates an ID for a contiguous bock of data.

func NewEncoder

func NewEncoder() *Encoder

NewIDEncoder makes a new Encoder.

func (*Encoder) Digest

func (e *Encoder) Digest() Digest

Digest get the Digest for the bytes written so far.

func (*Encoder) ID

func (e *Encoder) ID() *ID

ID returns the ID for the bytes written so far.

func (*Encoder) Reset

func (e *Encoder) Reset()

Reset the encoder so it can identify a new block of data.

func (*Encoder) Write

func (e *Encoder) Write(b []byte) (int, error)

Write writes bytes to the hash that makes up the ID.

type ID

type ID big.Int

ID represents a C4 ID.

var (
	// Id of nil (i.e. "")

	// Id with all bytes set to 0.


func Identify

func Identify(src io.Reader) *ID

Generate an id from an io.Reader

func Parse

func Parse(src string) (*ID, error)

Parse parses a C4 ID string into an ID.

func (*ID) Cmp

func (l *ID) Cmp(r *ID) int

* Cmp compares two IDs. * There are 3 possible return values. * -1 : Argument id is less than calling id. * 0: Argument id and calling id are identical. * +1: Argument id is greater than calling id. * Comparison is done on the actual numerical value of the ids. * Not the string representation.

func (*ID) Digest

func (id *ID) Digest() Digest

Digest returns the C4 Digest of the ID.

func (*ID) GobDecode added in v0.8.0

func (id *ID) GobDecode(buf []byte) error

GobEncode implements the gob.GobDecoder interface.

func (*ID) GobEncode added in v0.8.0

func (id *ID) GobEncode() ([]byte, error)

GobEncode implements the gob.GobEncoder interface.

func (*ID) Less

func (id *ID) Less(idArg *ID) bool

Returns true if B less than A in: A.Less(B)

func (*ID) MarshalBinary

func (id *ID) MarshalBinary() (data []byte, err error)

func (*ID) MarshalJSON

func (id *ID) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.MarshalJSON interface.

func (*ID) MarshalText

func (id *ID) MarshalText() (text []byte, err error)

func (*ID) String

func (id *ID) String() (s string)

String returns the standard string representation of a C4 id.

func (*ID) UnmarshalBinary

func (id *ID) UnmarshalBinary(data []byte) error

func (*ID) UnmarshalJSON

func (id *ID) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.UnmarshalJSON interface.

func (*ID) UnmarshalText

func (id *ID) UnmarshalText(text []byte) error

type Identifiable

type Identifiable interface {
	ID() *ID

Identifiable is an interface that requires an ID() method that returns the c4 ID of the of the object.

type Node

type Node struct {
	// contains filtered or unexported fields

A node represents a specific ID triplet within a tree. Nodes have three IDs: the Label, Left, and Right.

func (Node) Label

func (n Node) Label() Digest

func (Node) Left

func (n Node) Left() Digest

func (Node) Parent

func (n Node) Parent() Node

func (Node) Right

func (n Node) Right() Digest

type ReadCloser

type ReadCloser interface {

ReadCloser is an interface that matches io.ReadCloser and adds Identifiable.

type Reader

type Reader interface {

Reader is an interface that matches io.Reader and adds Identifiable.

type Slice

type Slice []*ID

func (Slice) ID

func (s Slice) ID() *ID

The ID method returns the ID of a sorted slice of IDs. For performance the ID() method assumes the slice is already sorted, and will return an incorrect ID() if that is not the case. If an error is encountered nil is returned. Possible errors are, the slice is empty, or the slice has nil entries

func (Slice) Index

func (s Slice) Index(id *ID) int

Index returns the array index where the ID is, or would be inserted if not in the slice already. The slice must be sorted in ascending order.

func (*Slice) Insert

func (s *Slice) Insert(id *ID)

Insert adds an ID to the slice in sorted order.

func (*Slice) String

func (s *Slice) String() string

String returns the slice of IDs concatenated together without spaces or newlines.

type Tree

type Tree struct {
	// contains filtered or unexported fields

func NewTree

func NewTree(s DigestSlice) *Tree

NewTree creates a new Tree from a DigestSlice, and copies the digests into the tree. However, it does not compute the tree.

func (*Tree) At

func (t *Tree) At(row, index int) Digest

At returns the Digest located at the given row, and index

func (*Tree) Compute

func (t *Tree) Compute() Digest

Compute resolves all Digests in the tree, and returns the root Digest

func (*Tree) Count

func (t *Tree) Count() int

Count returns the number of items in the list this tree represents.

func (*Tree) Digest

func (t *Tree) Digest() Digest

func (*Tree) ID

func (t *Tree) ID() *ID

func (*Tree) IDcount

func (t *Tree) IDcount() int

func (*Tree) Length

func (t *Tree) Length() int

Length returns the number of IDs in the entire tree

func (*Tree) MarshalBinary

func (t *Tree) MarshalBinary() (data []byte, err error)

func (*Tree) MarshalText

func (t *Tree) MarshalText() (text []byte, err error)

func (*Tree) Node

func (t *Tree) Node(i uint64) Node

func (*Tree) NodeCount

func (t *Tree) NodeCount() int

func (*Tree) Row

func (t *Tree) Row(i int) []Digest

func (*Tree) RowCount

func (t *Tree) RowCount() int

func (*Tree) Size

func (t *Tree) Size() int

Size returns the number of bytes required to serialize the tree (in binary format).

func (*Tree) String

func (t *Tree) String() string

func (*Tree) UnmarshalBinary

func (t *Tree) UnmarshalBinary(data []byte) error

type WriteCloser

type WriteCloser interface {

WriteCloser is an interface that matches io.WriteCloser and adds Identifiable.

type Writer

type Writer interface {

Writer is an interface that matches io.Writer and adds Identifiable.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL