zar

README

ZAR (Encrypted Archive Format)

ZAR is encrypted file archive format using modern cryptography and compression. Utilising AES 256, SipHash, Argon2, HKDF, SHA512/SHA256, Brotil.

Built to maximise security and compression ratio, all the while being streamable and fast. Utilising AES_256_CTR + SipHash to encrypt and authenticate the archive without requiring the whole file being extracted/read.

WARNING: PROJECT UNDER DEVELOPMENT & IS NOT STABLE

Archive Format

Headers

Encrypted Body
    - []CompressedBlock
        - []FileContents
        - SipHash
    - Almanac
        - []FileMetaData
        - SipHash
    - Almanac Offset
    - MAC

Almanac Offset

Compression Block

Compression block is a collection of file contents and a MAC. A compression block is used to improve compression ratios for small files by combining them together into a bigger block. Compression block size varies and can get quite large depending on what files it contains.

The Almanac/Index

The almanac is a array of file metadata. Name/path, modified date, size, block offset. All this information can be used to locate the; first cipher text block, compression block offset form start of cipher block, offset from start of compression block to file & file length.

The almanac is separate from file contents which allows it to be read quickly and not require the full ciphertext from being decrypted. This section is authenticated with SipHash and the "master mac", the mac used on the full ciphertext.

Documentation

Index

• Constants
• Variables
• type Almanac
• type Block
  • func (b Block) MAC() []byte
• type Decoder
  • func NewDecoder(r io.ReaderAt, key []byte, size int64) (*Decoder, error)
  • func (d *Decoder) Extract(output string) error
• type Encoder
  • func New(w io.Writer, key []byte) (*Encoder, error)
  • func (e *Encoder) Add(name string, modified uint64, r io.Reader) (int64, error)
  • func (e *Encoder) Close() error
• type File
  • func (f *File) BlockSize(index []File, id int) uint64
  • func (f *File) CipherBlock() uint64
  • func (f *File) CipherBlockOffset() uint64
  • func (f *File) End(start uint64) uint64
  • func (f *File) Start(index []File, id int) uint64
• type Header

Constants

const BlockMacSize = 8

BlockMacSize is the size of the message authentication code for a block

Variables

var (
	// ErrShotRead is returned when read didn't fill buffer
	ErrShotRead = errors.New("short read")
	// ErrFileName is returned when the file name is invalid
	ErrFileName = errors.New("file name invalid")
	// ErrFilesTooMany is returned when a compression block has more files
	// than the maximum
	ErrFilesTooMany = errors.New("too many files in compression block")
)

var (
	// ErrIntegrityFailed is returned when a computed and actual MAC does not
	// match
	ErrIntegrityFailed = errors.New("message authentication code failed")
)

Types

type Almanac

type Almanac struct {
	// Note is a encrypted message included in the archive
	Note []byte
	// Files is a list of meta data pointing to the location of each file
	Files []File
	// MAC is SipHash used to authenticate this section has not
	// been modified without having to authenticate the full archive
	MAC []byte
}

Almanac stores the metadata for each file

type Block

type Block []byte

Block is a collection of files, or one large file, combined into one buffer which is compressed together and authenticated with SipHash.

This block represents a NON compressed buffer i.e. decompression has already been executed.

func (Block) MAC

func (b Block) MAC() []byte

MAC returns the MAC for the block.

If len(block) < 8 the method will panic

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder will take a reader of the archive file

func NewDecoder

func NewDecoder(r io.ReaderAt, key []byte, size int64) (*Decoder, error)

NewDecoder creates a new zar archive decoder

func (*Decoder) Extract

func (d *Decoder) Extract(output string) error

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes the archive

func New

func New(w io.Writer, key []byte) (*Encoder, error)

New creates a new ZAR encoder

func (*Encoder) Add

func (e *Encoder) Add(name string, modified uint64, r io.Reader) (int64, error)

Add will read a file and add it to the archive

func (*Encoder) Close

func (e *Encoder) Close() error

Close must be called to finalise the archive

type File

type File struct {
	// Name refers to the relative path name without the "/" prefix
	Name string
	// Modified is the file modified date
	Modified uint64
	// Size refers to the size of the compressed file and mac
	Size uint64
	// Offset is a offset from the start of the encrypted body
	Offset uint64
}

File holds metadata on a file and the parameters used to locate it

func (*File) BlockSize

func (f *File) BlockSize(index []File, id int) uint64

BlockSize returns the compressed length of the block

func (*File) CipherBlock

func (f *File) CipherBlock() uint64

CipherBlock returns the ciphertext block ID of the compression block

func (*File) CipherBlockOffset

func (f *File) CipherBlockOffset() uint64

CipherBlockOffset returns the distance between the start of the cipher's block and the compression block's start

func (*File) End

func (f *File) End(start uint64) uint64

End returns the relative end index of the file

func (*File) Start

func (f *File) Start(index []File, id int) uint64

Start returns the relative offset within the block to the start of the file.

type Header

type Header struct {
	MagicNumber [3]byte
	Mode        uint8
	CipherSuite uint8
	Mac         uint8
	Compression uint8
}

Header is the first 7 bytes of a file and contains metadata on how to open it