modelwrap

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2026 License: MIT Imports: 9 Imported by: 0

README

Modelwrap

Builds reproducible dm-verity EROFS images of Hugging Face models. Learn more about how this works on the Tinfoil blog.

For the artifact format, trust assumptions, and EMWP cryptographic parameters, see SPEC.md.

This repository is also the Go module github.com/tinfoilsh/modelwrap. The root package defines the protocol surface shared by both sides (format constants, artifact reference parsing, EMWP key derivation), wrap implements the packer, and unwrap implements the consumer side (used by cvmimage to mount model packs at boot).

Usage

The modelwrap CLI is a launcher: it runs the packing inside a digest-pinned container image (requires docker), so artifact bytes are always produced by the pinned toolchain. Release binaries embed the matching image digest.

modelwrap mistralai/Ministral-3-3B-Instruct-2512@cfcb068fa7c44114cf77a462357c6cdcd2c304b4

To pack and encrypt a local/private model directory:

PRIVATE_MODEL_KEY_B64="${PRIVATE_MODEL_KEY_B64}" modelwrap --model-dir /path/to/model --encrypt

Arguments

  • model: Hugging Face model ID, preferably with @revision. If omitted with --model-dir, modelwrap derives basename@contentHash.
  • --model-dir <path>: pack a local/private model directory instead of downloading from Hugging Face. If model is provided without @revision, modelwrap uses the directory content hash as the revision.
  • --encrypt: emit encrypted modelwrap output (.emwp). Requires a master key via --key-file or PRIVATE_MODEL_KEY_B64.
  • --key-file <path>: file containing the base64-encoded 64-byte EMWP master key.
  • --verify: optional. Runs veritysetup verify for MWP and decrypts then verifies EMWP, which is useful for cached artifacts or release checks.
  • --output <path> / --cache <path>: output and download cache directories (default ./output, ./cache).
  • --image <ref>: override the packer container image the launcher runs (defaults to the release-pinned digest; also MODELWRAP_IMAGE).
  • --local: run the packer directly on the current machine instead of in a container. Artifact bytes then depend on locally installed tool versions.

Set HF_TOKEN when accessing gated or private Hugging Face models; the launcher passes it into the container without exposing the value on the docker command line.

MWP mode emits:

  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.mpk: dm-verity EROFS image
  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.info: metadata file in the format ROOTHASH_OFFSET_VERITYUUID

EMWP mode additionally emits:

  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.emwp: disk image with one encrypted payload partition
  • output/mistralai/Ministral-3-3B-Instruct-2512/cfcb068fa7c44114cf77a462357c6cdcd2c304b4.emwp.info: metadata file in the format ROOTHASH_OFFSET_PARTUUID

Supply Chain Pins

The published image is built in two digest-pinned stages: a Go builder that compiles the modelwrap binary (CGO_ENABLED=0, -trimpath, hash-locked Go dependencies via go.sum), and a runtime image based on the official Python image on Debian Trixie that installs erofs-utils, cryptsetup, and gdisk from a dated snapshot.debian.org archive plus a hash-checked requirements.txt (only huggingface_hub, which provides the hf CLI used for downloads).

The packer currently pins:

  • erofs-utils=1.8.6-1
  • cryptsetup=2:2.7.5-2
  • gdisk=1.0.10-2

The packer passes the dm-verity hash algorithm, format, and block sizes explicitly so tool default changes do not silently alter the dm-verity format.

To update Python dependencies, edit requirements.in and regenerate the lockfile:

python3 -m piptools compile --generate-hashes --output-file requirements.txt requirements.in

Documentation

Overview

Package modelwrap defines the Modelwrap (MWP) and Encrypted Modelwrap (EMWP) artifact format: the cryptographic constants, the artifact reference grammar, the deterministic identity derivations, and the key derivation used by both the packer and the consumer. See SPEC.md for the full format specification.

Index

Constants

View Source
const (
	VerityFormat        = 1
	VerityHashAlgorithm = "sha256"
	VerityDataBlockSize = 4096
	VerityHashBlockSize = 4096
)

dm-verity format parameters. These are passed explicitly to veritysetup so tool default changes never silently alter the artifact format.

View Source
const (
	EMWPCipher         = "aes-xts-plain64"
	EMWPKeySizeBits    = 512
	EMWPKeyBytes       = EMWPKeySizeBits / 8
	EMWPMasterKeyBytes = 64
	EMWPSectorSize     = 4096
	EMWPKeyDeriveInfo  = "tinfoil/emwp/dm-crypt-key/v1"
)

EMWP dm-crypt parameters.

View Source
const (
	GPTSectorSize            = 512
	EMWPPartitionStartSector = 2048
	EMWPGPTTrailingSectors   = 40
)

EMWP GPT disk image geometry. The encrypted payload partition starts at a fixed sector so ciphertext placement is deterministic.

Variables

This section is empty.

Functions

func DeriveKey

func DeriveKey(masterKey []byte, ref *ArtifactRef) ([]byte, error)

DeriveKey derives the per-artifact dm-crypt key from the EMWP master key using HKDF-SHA256 with the artifact ID as salt.

func HashDir

func HashDir(dir string) (string, error)

HashDir computes a deterministic content hash over a model directory tree, used as the revision for local model directories.

func ParseMasterKey

func ParseMasterKey(encoded string) ([]byte, error)

ParseMasterKey decodes and validates a base64-encoded EMWP master key.

func UUIDv5URL

func UUIDv5URL(name string) string

UUIDv5URL computes the deterministic RFC 4122 version 5 UUID of name in the URL namespace. All artifact UUIDs (dm-verity UUID, GPT disk GUID, GPT PARTUUID) are derived from the model identity this way.

Types

type ArtifactRef

type ArtifactRef struct {
	RootHash   string
	HashOffset string
	UUID       string
}

ArtifactRef is a parsed artifact reference of the form rootHash_hashOffset_uuid. For MWP artifacts the UUID is the dm-verity superblock UUID; for EMWP artifacts it is the GPT PARTUUID of the encrypted payload partition.

func ParseRef

func ParseRef(ref string) (*ArtifactRef, error)

ParseRef parses and validates a rootHash_hashOffset_uuid reference.

func (*ArtifactRef) ArtifactID

func (r *ArtifactRef) ArtifactID() string

ArtifactID returns the rootHash_uuid identity used as the HKDF salt for EMWP key derivation, binding the derived key to one specific artifact.

func (*ArtifactRef) String

func (r *ArtifactRef) String() string

String returns the canonical rootHash_hashOffset_uuid form.

Directories

Path Synopsis
cmd
modelwrap command
Command modelwrap packs model weights into MWP/EMWP artifacts.
Command modelwrap packs model weights into MWP/EMWP artifacts.
Package unwrap implements the consumer side of the Modelwrap protocol: opening dm-crypt and dm-verity mappings over MWP/EMWP artifacts and mounting the verified filesystem read-only.
Package unwrap implements the consumer side of the Modelwrap protocol: opening dm-crypt and dm-verity mappings over MWP/EMWP artifacts and mounting the verified filesystem read-only.
Package wrap implements the Modelwrap packer: it builds deterministic EROFS images of model directories, wraps them with dm-verity (MWP), and optionally encrypts them into a GPT disk image with dm-crypt (EMWP).
Package wrap implements the Modelwrap packer: it builds deterministic EROFS images of model directories, wraps them with dm-verity (MWP), and optionally encrypts them into a GPT disk image with dm-crypt (EMWP).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL