checksum

Documentation

Overview

Package checksum implements the client-visible checksums S3 exposes as x-amz-checksum-*.

This is a *second* checksum, distinct from the content MD5 behind ETag and from the integrity checksum the scrubber uses. It has its own algorithms, its own negotiation, and its own composition rule for multipart objects, and conflating it with either of the others is the mistake this package exists to make hard: the ETag is a property of how the object was stored, while this is a digest the *client* chose, sent, and expects back unchanged.

Index

• func CompositeOf(a Algorithm, partDigests [][]byte) (string, error)
• func Encode(sum []byte) string
• func SplitComposite(v string) (digest string, parts int, ok bool)
• type Algorithm
  • func Parse(s string) (Algorithm, error)
  • func (a Algorithm) Decode(s string) ([]byte, error)
  • func (a Algorithm) DefaultType() Type
  • func (a Algorithm) Header() string
  • func (a Algorithm) New() (hash.Hash, error)
  • func (a Algorithm) Size() (int, error)
  • func (a Algorithm) SupportsFullObject() bool
• type Type

Functions

func CompositeOf

func CompositeOf(a Algorithm, partDigests [][]byte) (string, error)

CompositeOf computes a multipart object's COMPOSITE checksum: the digest of the concatenated *raw part digests*, with the part count appended after a dash.

The "-N" suffix is not decoration. It is the only thing in the value that says "this is not a digest of the object's bytes" — a client that fed the object back through the same algorithm would get something else entirely, and the suffix is what stops that being a mystery.

Parts must be in ascending part-number order, which is the order they are concatenated in and therefore the order the digest depends on.

func Encode

func Encode(sum []byte) string

Encode renders a digest the way S3 does: standard base64.

func SplitComposite

func SplitComposite(v string) (digest string, parts int, ok bool)

SplitComposite separates a composite value into its digest and part count. ok is false when the value carries no suffix, which is what a FULL_OBJECT checksum looks like.

Types

type Algorithm

type Algorithm string

Algorithm is one of the checksum algorithms S3 defines. The zero value means no checksum was requested.

const (
	CRC32     Algorithm = "CRC32"
	CRC32C    Algorithm = "CRC32C"
	CRC64NVME Algorithm = "CRC64NVME"
	SHA1      Algorithm = "SHA1"
	SHA256    Algorithm = "SHA256"
)

The algorithms S3 accepts, spelled as they appear on the wire.

func Parse

func Parse(s string) (Algorithm, error)

Parse maps a wire value to an Algorithm, case-insensitively as S3 does.

func (Algorithm) Decode

func (a Algorithm) Decode(s string) ([]byte, error)

Decode reads a digest a client sent, and reports whether it is well-formed for the algorithm.

A digest of the wrong length is rejected here rather than compared and found unequal, so "you sent nonsense" and "your bytes did not match" stay distinguishable.

func (Algorithm) DefaultType

func (a Algorithm) DefaultType() Type

DefaultType is the checksum type S3 uses when a multipart upload names an algorithm but no type.

func (Algorithm) Header

func (a Algorithm) Header() string

Header is the request/response header carrying this algorithm's digest.

func (Algorithm) New

func (a Algorithm) New() (hash.Hash, error)

New returns a running hash for the algorithm.

func (Algorithm) Size

func (a Algorithm) Size() (int, error)

Size is the digest length in bytes.

func (Algorithm) SupportsFullObject

func (a Algorithm) SupportsFullObject() bool

SupportsFullObject reports whether the algorithm can produce a FULL_OBJECT multipart checksum. Only the CRCs can: combining two of them yields the CRC of the concatenation, which is what makes a whole-object digest computable from parts without re-reading the object.

type Type

type Type string

Type is how a multipart object's checksum relates to its parts.

const (
	// Composite is a digest *of the part digests*, which is why its value
	// carries a "-N" suffix naming the part count: it is not a digest of the
	// object's bytes and must not be mistaken for one.
	Composite Type = "COMPOSITE"
	// FullObject is a digest of the whole body, as if it had been written by a
	// single PUT. Only the CRCs support it, because only they compose: two
	// CRCs can be combined into the CRC of the concatenation, and two SHAs
	// cannot.
	FullObject Type = "FULL_OBJECT"
)