modelwrap

package module
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: MIT Imports: 10 Imported by: 0

README

Modelwrap

Builds reproducible dm-verity EROFS images of Hugging Face models. Learn more about how this works on the Tinfoil blog.

For the artifact format, trust assumptions, and EMWP cryptographic parameters, see SPEC.md.

This repository is also the Go module github.com/tinfoilsh/modelwrap. The root package defines the protocol surface shared by both sides (format constants, artifact reference parsing, EMWP key derivation), wrap implements the packer, and unwrap implements the consumer side (used by cvmimage to mount model packs at boot).

Usage

The modelwrap CLI is a launcher: it runs the packing inside a digest-pinned container image (requires docker), so artifact bytes are always produced by the pinned toolchain. Release binaries embed the matching image digest.

modelwrap mistralai/Ministral-3-3B-Instruct-2512@cfcb068fa7c44114cf77a462357c6cdcd2c304b4

Delete the generated artifacts and downloaded cache for one pinned revision:

modelwrap --delete mistralai/Ministral-3-3B-Instruct-2512@cfcb068fa7c44114cf77a462357c6cdcd2c304b4

To pack and encrypt a local/private model directory:

PRIVATE_MODEL_KEY_B64="${PRIVATE_MODEL_KEY_B64}" modelwrap --model-dir /path/to/model --encrypt

Arguments

  • model: Hugging Face model ID, preferably with @revision. If omitted with --model-dir, modelwrap derives basename@contentHash.
  • --model-dir <path>: pack a local/private model directory instead of downloading from Hugging Face. If model is provided without @revision, modelwrap uses the directory content hash as the revision.
  • --encrypt: emit encrypted modelwrap output (.emwp). Requires a master key via --key-file or PRIVATE_MODEL_KEY_B64.
  • --key-file <path>: file containing the base64-encoded 64-byte EMWP master key.
  • --verify: optional. Runs veritysetup verify for MWP and decrypts then verifies EMWP, which is useful for cached artifacts or release checks.
  • --output <path> / --cache <path>: output and download cache directories (default ./output, ./cache).
  • --image <ref>: override the packer container image the launcher runs (defaults to the release-pinned digest; also MODELWRAP_IMAGE).
  • --local: run the packer directly on the current machine instead of in a container. Artifact bytes then depend on locally installed tool versions.

Set HF_TOKEN when accessing gated or private Hugging Face models; the launcher passes it into the container without exposing the value on the docker command line.

MWP mode emits:

  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.mpk: dm-verity EROFS image
  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.info: metadata file in the format ROOTHASH_OFFSET_VERITYUUID

EMWP mode additionally emits:

  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.emwp: disk image with one encrypted payload partition
  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.emwp.info: metadata file in the format ROOTHASH_OFFSET_PARTUUID

Supply Chain Pins

The published image is built in two digest-pinned stages: a Go builder that compiles the modelwrap binary (CGO_ENABLED=0, -trimpath, hash-locked Go dependencies via go.sum), and a runtime image based on the official Python image on Debian Trixie that installs erofs-utils, cryptsetup, and gdisk from a dated snapshot.debian.org archive plus a hash-checked requirements.txt (only huggingface_hub, which provides the hf CLI used for downloads).

The packer currently pins:

  • erofs-utils=1.8.6-1
  • cryptsetup=2:2.7.5-2
  • gdisk=1.0.10-2

The packer passes the dm-verity hash algorithm, format, and block sizes explicitly so tool default changes do not silently alter the dm-verity format.

To update Python dependencies, edit requirements.in and regenerate the lockfile:

python3 -m piptools compile --generate-hashes --output-file requirements.txt requirements.in

Documentation

Overview

Package modelwrap defines the Modelwrap (MWP) and Encrypted Modelwrap (EMWP) artifact format: the cryptographic constants, the artifact reference grammar, the deterministic identity derivations, and the key derivation used by both the packer and the consumer. See SPEC.md for the full format specification.

Index

Constants

View Source
const (
	EMWPCipher         = "aes-xts-plain64"
	EMWPKeySizeBits    = 512
	EMWPKeyBytes       = EMWPKeySizeBits / 8
	EMWPMasterKeyBytes = 64
	EMWPSectorSize     = 4096
	EMWPKeyDeriveInfo  = "tinfoil/emwp/dm-crypt-key/v1"
)

EMWP dm-crypt parameters.

View Source
const (
	GPTSectorSize            = 512
	EMWPPartitionStartSector = 2048
	EMWPGPTTrailingSectors   = 40
)

EMWP GPT disk image geometry. The encrypted payload partition starts at a fixed sector so ciphertext placement is deterministic.

View Source
const (
	VerityFormat        = 1
	VerityHashAlgorithm = "sha256"
	VerityDataBlockSize = 4096
	VerityHashBlockSize = 4096
	VeritySaltBytes     = 32 // a SHA-256 digest

)

dm-verity format parameters. These are passed explicitly to veritysetup so tool default changes never silently alter the artifact format, and consumers never read them from the artifact.

Variables

This section is empty.

Functions

func DeriveKey

func DeriveKey(masterKey []byte, ref *ArtifactRef) ([]byte, error)

DeriveKey derives the per-artifact dm-crypt key from the EMWP master key using HKDF-SHA256 with the artifact ID as salt.

func HashDir

func HashDir(dir string) (string, error)

HashDir computes a deterministic content hash over a model directory tree, used as the revision for local model directories.

func ParseMasterKey

func ParseMasterKey(encoded string) ([]byte, error)

ParseMasterKey decodes and validates a base64-encoded EMWP master key.

func UUIDv5URL

func UUIDv5URL(name string) string

UUIDv5URL computes the deterministic RFC 4122 version 5 UUID of name in the URL namespace. All artifact UUIDs (dm-verity UUID, GPT disk GUID, GPT PARTUUID) are derived from the model identity this way.

func VeritySalt added in v0.2.1

func VeritySalt(model string) []byte

VeritySalt returns the deterministic dm-verity salt for a model identity string (name@revision): the SHA-256 of the identity. The packer uses this when formatting; the consumer re-derives it from the attested model identity, so no on-disk metadata is ever consulted.

Types

type ArtifactRef

type ArtifactRef struct {
	RootHash   string
	HashOffset string
	UUID       string
}

ArtifactRef is a parsed artifact reference of the form rootHash_hashOffset_uuid. For MWP artifacts the UUID is the dm-verity superblock UUID; for EMWP artifacts it is the GPT PARTUUID of the encrypted payload partition.

func ParseRef

func ParseRef(ref string) (*ArtifactRef, error)

ParseRef parses and validates a rootHash_hashOffset_uuid reference.

func (*ArtifactRef) ArtifactID

func (r *ArtifactRef) ArtifactID() string

ArtifactID returns the rootHash_uuid identity used as the HKDF salt for EMWP key derivation, binding the derived key to one specific artifact.

func (*ArtifactRef) HashOffsetBytes added in v0.2.1

func (r *ArtifactRef) HashOffsetBytes() (uint64, error)

HashOffsetBytes parses the reference's hash offset field.

func (*ArtifactRef) String

func (r *ArtifactRef) String() string

String returns the canonical rootHash_hashOffset_uuid form.

type VerityParams added in v0.2.1

type VerityParams struct {
	Salt           []byte
	DataBlocks     uint64
	HashTreeOffset uint64 // byte offset of the first hash tree block
}

VerityParams are the fully explicit parameters for opening an MWP hash tree with `veritysetup --no-superblock`, so that no unverified on-disk metadata is ever parsed by the dm-verity stack.

func VerityParamsForArtifact added in v0.2.1

func VerityParamsForArtifact(hashOffset uint64, salt []byte) (*VerityParams, error)

VerityParamsForArtifact derives explicit dm-verity parameters from the attested hash offset and the salt. In the MWP layout the data area is exactly the hashOffset bytes preceding the hash area, so the data size is attested rather than read from the artifact, and the hash tree begins one hash block past hashOffset (the slot occupied by the untrusted, ignored superblock the packer writes there).

func (*VerityParams) VeritysetupArgs added in v0.2.1

func (p *VerityParams) VeritysetupArgs() []string

VeritysetupArgs returns the explicit veritysetup arguments pinning every format parameter for a --no-superblock open or verify.

Directories

Path Synopsis
cmd
modelwrap command
Command modelwrap packs model weights into MWP/EMWP artifacts.
Command modelwrap packs model weights into MWP/EMWP artifacts.
Package crypt implements the EMWP dm-crypt encryption (aes-xts-plain64, 512-bit key, 4096-byte sectors) in pure Go.
Package crypt implements the EMWP dm-crypt encryption (aes-xts-plain64, 512-bit key, 4096-byte sectors) in pure Go.
Package unwrap implements the consumer side of the Modelwrap protocol: opening dm-crypt and dm-verity mappings over MWP/EMWP artifacts and mounting the verified filesystem read-only.
Package unwrap implements the consumer side of the Modelwrap protocol: opening dm-crypt and dm-verity mappings over MWP/EMWP artifacts and mounting the verified filesystem read-only.
Package wrap implements the Modelwrap packer: it builds deterministic EROFS images of model directories, wraps them with dm-verity (MWP), and optionally encrypts them into a GPT disk image with dm-crypt (EMWP).
Package wrap implements the Modelwrap packer: it builds deterministic EROFS images of model directories, wraps them with dm-verity (MWP), and optionally encrypts them into a GPT disk image with dm-crypt (EMWP).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL