conformance

Documentation

Overview

Package conformance implements test vector runners for Evidence Pack spec conformance testing. This package is intended for this repository's test harness and does not provide a stable public API contract for external consumers.

Index

• func FixturePath(category, fixturePath string) string
• func ListVectorFiles(category string) ([]string, error)
• func LoadVector[T any](category, filename string) (*T, error)
• func LoadVectorRaw(category, filename string) (json.RawMessage, error)
• func SkipIfNoVectors(t *testing.T)
• func VectorPath(category, filename string) string
• func VectorsAvailable() bool
• type AccessPolicyCase
• type AccessPolicyInvalidCase
• type AccessPolicyVector
• type ArtifactFieldTest
• type ArtifactFieldsVector
• type ArtifactInput
• type AttestationVector
• type Computed
• type DigestFormatInvalidCase
• type DigestFormatTest
• type DigestFormatValidCase
• type DigestFormatVector
• type DigestFormatVectorActual
• type IntegerCase
• type IntegerVector
• type JCSTest
• type JCSVector
• type LimitsTest
• type LimitsVector
• type ManifestDigestVector
• type ManifestRequiredFieldsTest
• type ManifestRequiredFieldsVector
• type ManifestTest
• type ManifestVector
• type PackDigestVector
• type PathValidationTest
• type PathValidationVector
• type ProvenanceInvalidCase
• type ProvenanceValidCase
• type ProvenanceVector
• type ResultDetail
• type StructureTest
• type StructureVector
• type TimestampCase
• type TimestampVector
• type VectorError
• type VectorResult
• type ZipSafetyTest
• type ZipSafetyVector

Functions

func FixturePath

func FixturePath(category, fixturePath string) string

FixturePath returns the full path to a fixture file referenced by a vector.

func ListVectorFiles

func ListVectorFiles(category string) ([]string, error)

ListVectorFiles returns all JSON files in a vector category directory.

func LoadVector

func LoadVector[T any](category, filename string) (*T, error)

LoadVector loads and unmarshals a JSON test vector file.

func LoadVectorRaw

func LoadVectorRaw(category, filename string) (json.RawMessage, error)

LoadVectorRaw loads a test vector file as raw JSON.

func SkipIfNoVectors

func SkipIfNoVectors(t *testing.T)

SkipIfNoVectors skips the test if the test vectors directory is not available.

func VectorPath

func VectorPath(category, filename string) string

VectorPath returns the full path to a vector file.

func VectorsAvailable

func VectorsAvailable() bool

VectorsAvailable returns true if the test vectors directory exists.

Types

type AccessPolicyCase

type AccessPolicyCase struct {
	Policy      string `json:"policy"`
	Description string `json:"description"`
}

AccessPolicyCase is a valid policy test case.

type AccessPolicyInvalidCase

type AccessPolicyInvalidCase struct {
	Policy      string `json:"policy"`
	Valid       bool   `json:"valid"`
	Reason      string `json:"reason"`
	Description string `json:"description"`
}

AccessPolicyInvalidCase is an invalid policy test case.

type AccessPolicyVector

type AccessPolicyVector struct {
	Name            string                    `json:"name"`
	Description     string                    `json:"description"`
	ValidPolicies   []AccessPolicyCase        `json:"valid_policies"`
	InvalidPolicies []AccessPolicyInvalidCase `json:"invalid_policies"`
	Notes           string                    `json:"notes,omitempty"`
}

AccessPolicyVector represents test vectors for access policy validation.

type ArtifactFieldTest

type ArtifactFieldTest struct {
	Name          string          `json:"name"`
	Input         json.RawMessage `json:"input"`
	Valid         bool            `json:"valid"`
	ExpectedError string          `json:"expected_error,omitempty"`
	Requirements  []string        `json:"requirements,omitempty"`
	Description   string          `json:"description"`
}

ArtifactFieldTest is a single artifact field validation test case.

type ArtifactFieldsVector

type ArtifactFieldsVector struct {
	Name                    string              `json:"name"`
	Description             string              `json:"description"`
	EmbeddedArtifactTests   []ArtifactFieldTest `json:"embedded_artifact_tests"`
	ReferencedArtifactTests []ArtifactFieldTest `json:"referenced_artifact_tests"`
	Notes                   string              `json:"notes,omitempty"`
}

ArtifactFieldsVector represents test vectors for artifact field validation.

type ArtifactInput

type ArtifactInput struct {
	Type   string `json:"type"`
	Path   string `json:"path"`
	Digest string `json:"digest"`
	Size   int64  `json:"size"`
}

ArtifactInput represents an artifact in test vector input.

type AttestationVector

type AttestationVector struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Input       struct {
		Bundle json.RawMessage `json:"bundle"`
	} `json:"input"`
	Valid         bool   `json:"valid"`
	ExpectedError string `json:"expected_error,omitempty"`
	Notes         string `json:"notes,omitempty"`
}

AttestationVector represents a test vector for attestation validation.

type Computed

type Computed struct {
	PackDigest     string `json:"pack_digest,omitempty"`
	CanonicalInput string `json:"canonical_input,omitempty"`
}

Computed contains computed values for comparison with expected values.

type DigestFormatInvalidCase

type DigestFormatInvalidCase struct {
	Value       string `json:"value"`
	Valid       bool   `json:"valid"`
	Reason      string `json:"reason"`
	Description string `json:"description"`
}

DigestFormatInvalidCase is an invalid digest test case.

type DigestFormatTest

type DigestFormatTest struct {
	Digest        string `json:"digest"`
	Valid         bool   `json:"valid"`
	ExpectedError string `json:"expected_error,omitempty"`
	Description   string `json:"description"`
}

DigestFormatTest is a single digest format test case.

type DigestFormatValidCase

type DigestFormatValidCase struct {
	Value       string `json:"value"`
	Description string `json:"description"`
}

DigestFormatValidCase is a valid digest test case.

type DigestFormatVector

type DigestFormatVector struct {
	Name        string             `json:"name"`
	Description string             `json:"description"`
	Tests       []DigestFormatTest `json:"tests"`
	Notes       string             `json:"notes,omitempty"`
}

DigestFormatVector represents test vectors for digest format validation.

type DigestFormatVectorActual

type DigestFormatVectorActual struct {
	Name           string                    `json:"name"`
	Description    string                    `json:"description"`
	ValidDigests   []DigestFormatValidCase   `json:"valid_digests"`
	InvalidDigests []DigestFormatInvalidCase `json:"invalid_digests"`
	Notes          string                    `json:"notes,omitempty"`
}

DigestFormatVectorActual matches the actual test vector format with separate arrays.

type IntegerCase

type IntegerCase struct {
	Value       int64  `json:"value,omitempty"`
	JSON        string `json:"json"`
	Reason      string `json:"reason,omitempty"`
	Description string `json:"description"`
}

IntegerCase is a single integer validation test case.

type IntegerVector

type IntegerVector struct {
	Name            string        `json:"name"`
	Description     string        `json:"description"`
	ValidIntegers   []IntegerCase `json:"valid_integers"`
	InvalidIntegers []IntegerCase `json:"invalid_integers"`
	Notes           string        `json:"notes,omitempty"`
}

IntegerVector represents a test vector file for JSON integer validation.

type JCSTest

type JCSTest struct {
	Name              string          `json:"name"`
	Input             json.RawMessage `json:"input"`
	ExpectedCanonical string          `json:"expected_canonical"`
	Valid             bool            `json:"valid"`
	ExpectedError     string          `json:"expected_error,omitempty"`
	Description       string          `json:"description"`
}

JCSTest is a single JCS test case.

type JCSVector

type JCSVector struct {
	Name        string    `json:"name"`
	Description string    `json:"description"`
	Tests       []JCSTest `json:"tests"`
	Notes       string    `json:"notes,omitempty"`
}

JCSVector represents a test vector for JCS canonicalization.

type LimitsTest

type LimitsTest struct {
	Config      map[string]int64 `json:"config"`
	Expected    string           `json:"expected"` // "reject_config" or "accept_config"
	Reason      string           `json:"reason,omitempty"`
	Description string           `json:"description"`
}

LimitsTest is a single limit enforcement test case.

type LimitsVector

type LimitsVector struct {
	Name                  string `json:"name"`
	Description           string `json:"description"`
	MinimumRequiredLimits struct {
		MaxArtifactSize struct {
			Minimum      int64  `json:"minimum"`
			MinimumHuman string `json:"minimum_human"`
			Description  string `json:"description"`
		} `json:"max_artifact_size"`
		MaxPackSize struct {
			Minimum      int64  `json:"minimum"`
			MinimumHuman string `json:"minimum_human"`
			Description  string `json:"description"`
		} `json:"max_pack_size"`
		MaxArtifactCount struct {
			Minimum     int    `json:"minimum"`
			Description string `json:"description"`
		} `json:"max_artifact_count"`
	} `json:"minimum_required_limits"`
	Tests []LimitsTest `json:"tests"`
	Notes string       `json:"notes,omitempty"`
}

LimitsVector represents a test vector file for limit enforcement.

type ManifestDigestVector

type ManifestDigestVector struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Input       struct {
		Manifest json.RawMessage `json:"manifest"`
	} `json:"input"`
	Expected struct {
		JCSCanonical string `json:"jcs_canonical"`
		SHA256       string `json:"sha256"`
	} `json:"expected"`
	Valid bool   `json:"valid"`
	Notes string `json:"notes,omitempty"`
}

ManifestDigestVector represents a test vector for manifest_digest computation.

type ManifestRequiredFieldsTest

type ManifestRequiredFieldsTest struct {
	Name          string          `json:"name"`
	Input         json.RawMessage `json:"input"`
	Valid         bool            `json:"valid"`
	ExpectedError string          `json:"expected_error,omitempty"`
	Description   string          `json:"description"`
}

ManifestRequiredFieldsTest is a single required fields test case.

type ManifestRequiredFieldsVector

type ManifestRequiredFieldsVector struct {
	Name        string                       `json:"name"`
	Description string                       `json:"description"`
	Tests       []ManifestRequiredFieldsTest `json:"tests"`
	Notes       string                       `json:"notes,omitempty"`
}

ManifestRequiredFieldsVector represents test vectors for manifest required fields.

type ManifestTest

type ManifestTest struct {
	Name          string          `json:"name"`
	Input         json.RawMessage `json:"input"`
	Valid         bool            `json:"valid"`
	ExpectedError string          `json:"expected_error,omitempty"`
	Requirements  []string        `json:"requirements,omitempty"`
	Description   string          `json:"description"`
}

ManifestTest is a single manifest validation test case.

type ManifestVector

type ManifestVector struct {
	Name        string         `json:"name"`
	Description string         `json:"description"`
	Tests       []ManifestTest `json:"tests,omitempty"`
	Notes       string         `json:"notes,omitempty"`
}

ManifestVector represents a test vector file for manifest validation.

type PackDigestVector

type PackDigestVector struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Input       struct {
		Artifacts []ArtifactInput `json:"artifacts"`
	} `json:"input"`
	Expected struct {
		PackDigest     string   `json:"pack_digest"`
		CanonicalInput string   `json:"canonical_input,omitempty"`
		SortedPaths    []string `json:"sorted_paths,omitempty"`
	} `json:"expected"`
	Valid bool   `json:"valid"`
	Notes string `json:"notes,omitempty"`
}

PackDigestVector represents a test vector for pack_digest computation.

type PathValidationTest

type PathValidationTest struct {
	Path        string   `json:"path,omitempty"`
	Paths       []string `json:"paths,omitempty"`
	Valid       bool     `json:"valid"`
	Reason      string   `json:"reason,omitempty"`
	Description string   `json:"description,omitempty"`
}

PathValidationTest is a single path validation test case.

type PathValidationVector

type PathValidationVector struct {
	Name        string               `json:"name"`
	Description string               `json:"description"`
	Tests       []PathValidationTest `json:"tests"`
}

PathValidationVector represents a test vector file for path validation.

type ProvenanceInvalidCase

type ProvenanceInvalidCase struct {
	Description string          `json:"description"`
	Provenance  json.RawMessage `json:"provenance"`
	Valid       bool            `json:"valid"`
	Reason      string          `json:"reason"`
}

ProvenanceInvalidCase is an invalid provenance test case.

type ProvenanceValidCase

type ProvenanceValidCase struct {
	Description string          `json:"description"`
	Provenance  json.RawMessage `json:"provenance"`
	Valid       bool            `json:"valid"`
}

ProvenanceValidCase is a valid provenance test case.

type ProvenanceVector

type ProvenanceVector struct {
	Name              string                  `json:"name"`
	Description       string                  `json:"description"`
	ValidProvenance   []ProvenanceValidCase   `json:"valid_provenance"`
	InvalidProvenance []ProvenanceInvalidCase `json:"invalid_provenance"`
	Notes             string                  `json:"notes,omitempty"`
}

ProvenanceVector represents test vectors for provenance validation.

type ResultDetail

type ResultDetail struct {
	OK       bool          `json:"ok"`
	Errors   []VectorError `json:"errors,omitempty"`
	Computed *Computed     `json:"computed,omitempty"`
}

ResultDetail contains the actual test execution results.

type StructureTest

type StructureTest struct {
	Name          string   `json:"name"`
	Structure     []string `json:"structure,omitempty"`
	Filename      string   `json:"filename,omitempty"`
	Content       string   `json:"content,omitempty"`
	Valid         bool     `json:"valid"`
	ExpectedError string   `json:"expected_error,omitempty"`
	Requirements  []string `json:"requirements,omitempty"`
	Description   string   `json:"description"`
}

StructureTest is a single structure validation test case.

type StructureVector

type StructureVector struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Tests       []StructureTest `json:"tests,omitempty"`
	// Additional metadata sections
	ZIP64Requirements json.RawMessage `json:"zip64_requirements,omitempty"`
	MetadataHandling  json.RawMessage `json:"metadata_handling,omitempty"`
	Notes             string          `json:"notes,omitempty"`
}

StructureVector represents a test vector file for pack structure validation.

type TimestampCase

type TimestampCase struct {
	Value       string `json:"value"`
	Valid       bool   `json:"valid"`
	Reason      string `json:"reason,omitempty"`
	Description string `json:"description"`
}

TimestampCase is a single timestamp validation test case.

type TimestampVector

type TimestampVector struct {
	Name              string          `json:"name"`
	Description       string          `json:"description"`
	ValidTimestamps   []TimestampCase `json:"valid_timestamps"`
	InvalidTimestamps []TimestampCase `json:"invalid_timestamps"`
	Notes             string          `json:"notes,omitempty"`
}

TimestampVector represents a test vector file for timestamp validation.

type VectorError

type VectorError struct {
	Code     string `json:"code"`
	Message  string `json:"message,omitempty"`
	Location string `json:"location,omitempty"`
}

VectorError describes an error encountered during vector execution.

type VectorResult

type VectorResult struct {
	Vector string       `json:"vector"`
	Valid  bool         `json:"valid"`
	Result ResultDetail `json:"result"`
}

VectorResult represents the output of running a single test vector, per runner-spec.md.

type ZipSafetyTest

type ZipSafetyTest struct {
	EntryPath   string `json:"entry_path,omitempty"`
	Path        string `json:"path,omitempty"`
	Expected    string `json:"expected"` // "reject" or "accept"
	Reason      string `json:"reason"`
	Description string `json:"description,omitempty"`
}

ZipSafetyTest is a single ZIP safety test case.

type ZipSafetyVector

type ZipSafetyVector struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	// For single-test vectors with ZIP fixtures
	Fixture string `json:"fixture,omitempty"`
	Valid   bool   `json:"valid"`
	Reason  string `json:"reason,omitempty"`
	Notes   string `json:"notes,omitempty"`
	// For multi-test vectors
	Tests []ZipSafetyTest `json:"tests,omitempty"`
}

ZipSafetyVector represents a test vector file for ZIP safety checks. Some vectors have a single test (with fixture), others have multiple tests.