appleoracle

README

appleoracle

macOS's own libcompression, exposed so a test can ask the platform what the bytes should be instead of asserting against bytes the implementation under test produced.

Nothing here is a codec.

enc, ok := appleoracle.Encode(appleoracle.LZFSE, src)
got, ok := appleoracle.Decode(appleoracle.LZFSE, enc, len(src))

Why

One organisation answered that question three different ways:

repository	how it asks Apple
go-compressions/lzfse	links libcompression and compares both directions
go-compressions/adc	commits files hdiutil produced
go-compressions/lz4	builds Apple-format frames by hand — no oracle at all

The first is what caught the short-literal-payload defect in lzfse#12: 48 of 64 Apple-encoded runs the decoder refused. The third is the one that worries: a fixture the implementation's author wrote can be wrong in exactly the way the implementation is wrong, and agree with it.

A committed witness is real, and it goes stale without saying so. A live oracle cannot.

Availability

Everything is behind darwin && cgo. Elsewhere the package is a stub: Available() reports false and both entry points refuse rather than returning a plausible-looking empty slice, so a consumer can import it unconditionally from a _test.go and skip.

Failure is loud

compression_encode_buffer reports failure by writing zero bytes — which is also what it writes for an input it declines to compress, and for an empty input. Those are not the same thing, so:

• Encode returns ok == false for "the platform produced no stream". It cannot say why, and the doc comment says so. LZVN declines a 1-byte input, measured. A caller reads that as the oracle has nothing to say here.
• an empty input is (nil, true), because nothing to compress is not a refusal.
• Decode decodes into want+1 bytes, not want. compression_decode_buffer fills the buffer it is given, so decoding into exactly want returns want for a stream holding more and the truncation is invisible. Measured: asking for len(src)-1 succeeded and handed back a prefix.

Scope

LZFSE, LZVN and LZ4 — what a repository here compares against today. LZ4 is Apple's framed variant (bv41/bv4-/bv4$), not a bare block, and a test pins that; picking COMPRESSION_LZ4_RAW instead would give an oracle that disagrees with a correct implementation on every input.

The list stops there. An entry nobody calls is an entry nobody has run, and a wrong constant produces an oracle that confidently disagrees with everything. ADC is absent because libcompression does not implement it — which is why go-compressions/adc uses committed hdiutil output and will keep doing so.

Documentation

Overview

Package appleoracle exposes macOS's own libcompression so a test can ask the platform what the bytes should be, instead of asserting against bytes the implementation under test produced.

It exists because one fleet answered that question three different ways:

• go-compressions/lzfse links libcompression directly and compares both directions against it. That is what caught the short-literal-payload defect in #12 -- 48 of 64 Apple-encoded runs the decoder refused.
• go-compressions/adc commits files hdiutil produced. A committed witness is real, and it goes stale without saying so.
• go-compressions/lz4 builds Apple-format frames BY HAND in its tests, with no oracle at all -- so those tests can be wrong in exactly the way the implementation is wrong, and agree with it.

Every function here is a thin call into compression_encode_buffer / compression_decode_buffer. Nothing in this package is a codec.

Availability

The whole package is behind `darwin && cgo`. On every other build it is empty, and Available reports false, so a consumer can import it unconditionally from a _test.go file and skip.

Failure is loud

libcompression reports failure by returning zero bytes written, which is indistinguishable from "the empty input compressed to nothing". Every function here returns an explicit ok, and a caller that ignores it has written a test that passes when the oracle is not working -- which is the failure mode this package exists to remove, not to reproduce.

Index

• func Available() bool
• func Decode(a Algorithm, src []byte, want int) (out []byte, ok bool)
• func Encode(a Algorithm, src []byte) (out []byte, ok bool)
• type Algorithm
  • func (a Algorithm) String() string

Functions

func Available

func Available() bool

Available reports whether this build can ask the platform. True here, false in the stub, so a _test.go can import this package unconditionally.

func Decode

func Decode(a Algorithm, src []byte, want int) (out []byte, ok bool)

Decode decompresses src with the platform's implementation of a, expecting exactly want bytes.

ok is false when the platform refused the stream OR produced a different length than want -- in either direction. Those are one answer on purpose: a result that is not exactly what the caller knows the input held is not a decode, and a test that accepted it would be asserting on a prefix.

⛔ The destination is want+1 bytes, not want. compression_decode_buffer fills the buffer it is given and reports how much it wrote, so decoding into exactly want bytes returns want for a stream that holds MORE than want and the truncation is invisible. Measured: asking for len(src)-1 succeeded and handed back a prefix. One spare byte makes an over-long stream report want+1, which is not want, which is a refusal.

func Encode

func Encode(a Algorithm, src []byte) (out []byte, ok bool)

Encode compresses src with the platform's implementation of a.

ok is false when the platform produced no stream. compression_encode_buffer signals that with a zero return and does not say why, so this cannot tell an unsupported algorithm from an input the codec declined to compress -- and it DOES decline: LZVN returns nothing for a 1-byte input, measured. A caller therefore reads ok == false as "the oracle has nothing to say about this input", skips it, and must not read it as "the oracle is broken".

An empty input is the exception: nothing to compress is not a refusal, so that case is (nil, true).

Types

type Algorithm

type Algorithm int

Algorithm is one of the codecs libcompression implements.

The list stops at what a repository in this organisation actually compares against today. An entry nobody calls is an entry nobody has run, and a wrong constant here produces an oracle that confidently disagrees with everything.

const (
	// LZFSE is Apple's LZFSE (COMPRESSION_LZFSE), the raw bvx stream.
	// Caller: go-compressions/lzfse.
	LZFSE Algorithm = iota
	// LZVN is the LZVN stream Apple uses for small payloads
	// (COMPRESSION_LZVN, 0x900). Caller: go-compressions/lzfse.
	LZVN
	// LZ4 is Apple's FRAMED LZ4 (COMPRESSION_LZ4): the "bv41"/"bv4-"/"bv4$"
	// block sequence, not a bare LZ4 block. Caller: go-compressions/lz4,
	// whose apple.go implements exactly that framing.
	LZ4
)

func (Algorithm) String

func (a Algorithm) String() string