modelarchive

Documentation

Overview

Package modelarchive builds uncompressed tar archives of model directories for upload to Baseten.

Archive layout: files are stored at the archive root with paths relative to the input directory, symlinks are not followed, and only regular files are included.

Ignore handling is driven by a caller-supplied IgnoreFileFunc. If a .truss_ignore file is present at the root of the input directory, callers must supply an IgnoreFileProcessor to parse it; otherwise DefaultIgnoreFile (or a caller-provided default) is applied. Note the underscore in .truss_ignore.

This package does not parse config.yaml. Callers that need to inline external package directories or substitute a different config.yaml into the archive must extract those values themselves and pass them via BuildModelArchiveOptions.

Index

• func BuildModelArchive(ctx context.Context, opts BuildModelArchiveOptions) (io.ReadCloser, error)
• func DefaultIgnoreFile(_ context.Context, opts IgnoreFileOptions) (bool, error)
• type BuildModelArchiveOptions
• type IgnoreFileFunc
• type IgnoreFileOptions
• type IgnoreFileProcessorOptions

Functions

func BuildModelArchive

func BuildModelArchive(ctx context.Context, opts BuildModelArchiveOptions) (io.ReadCloser, error)

BuildModelArchive returns a io.ReadCloser that streams an uncompressed tar archive of the model directory described by opts. The archive is produced lazily as the reader is consumed; callers must Close it to release the underlying walk goroutine.

Errors encountered during the walk surface from the next Read call. Cancelling ctx also aborts the build.

func DefaultIgnoreFile

func DefaultIgnoreFile(_ context.Context, opts IgnoreFileOptions) (bool, error)

DefaultIgnoreFile reports whether a path should be excluded using the default ignore rules, applied when no .truss_ignore file is present. It excludes the usual Python build, cache, and environment cruft (__pycache__, build/dist directories, virtualenvs, *.pyc, .DS_Store, .git, and so on).

A directory named by a directory-only rule (such as __pycache__) is itself kept while its contents are excluded: the bare directory still appears in a model's signature even when everything inside it is ignored. So directory rules ([isDefaultIgnoredDirName] and *.egg-info) match only an ancestor component, while bare-name rules ([isDefaultIgnoredName]) match the entry itself or any ancestor.

Types

type BuildModelArchiveOptions

type BuildModelArchiveOptions struct {
	// Dir is the absolute or relative path to the model directory to
	// archive. Required.
	Dir string

	// ConfigYAMLOverride, if non-nil, replaces the contents of the root
	// config.yaml entry in the archive. If nil, any config.yaml on disk
	// at Dir is archived verbatim.
	ConfigYAMLOverride []byte

	// ExternalPackageDirs are extra directories whose contents are inlined
	// under BundledPackagesDir in the archive. Paths may be absolute or
	// relative to Dir. The basename of each entry is not preserved; its
	// children land directly under BundledPackagesDir.
	//
	// Read from the `external_package_dirs` field of the model's config.yaml.
	ExternalPackageDirs []string

	// BundledPackagesDir is the directory inside the archive that receives
	// inlined ExternalPackageDirs contents. Required when ExternalPackageDirs
	// is non-empty.
	//
	// Read from the `bundled_packages_dir` field of the model's config.yaml
	// (the canonical default is "packages").
	BundledPackagesDir string

	// IgnoreFileProcessor parses the contents of a .truss_ignore file
	// found at the root of Dir into an [IgnoreFileFunc]. Required if a
	// .truss_ignore file is present; otherwise [BuildModelArchive] returns
	// an error. When nil and no .truss_ignore exists, DefaultIgnoreFile
	// is used.
	IgnoreFileProcessor func(context.Context, IgnoreFileProcessorOptions) (IgnoreFileFunc, error)

	// DefaultIgnoreFile is applied when no .truss_ignore is present in
	// Dir. If nil, the package-level [DefaultIgnoreFile] function is used.
	// Pass a no-op function to disable default ignoring entirely.
	DefaultIgnoreFile IgnoreFileFunc
}

BuildModelArchiveOptions configures BuildModelArchive.

type IgnoreFileFunc

type IgnoreFileFunc func(context.Context, IgnoreFileOptions) (ignore bool, err error)

IgnoreFileFunc reports whether a given path should be excluded from the archive. Returning an error aborts the archive build immediately and propagates the error to the reader.

When the function returns true for a directory, the walker prunes the entire subtree.

type IgnoreFileOptions

type IgnoreFileOptions struct {
	// RelPath is the path relative to the archive root, using forward
	// slashes on all platforms.
	RelPath string
	// Entry is the directory entry as returned by [filepath.WalkDir].
	Entry fs.DirEntry
}

IgnoreFileOptions is passed to an IgnoreFileFunc for each candidate path encountered during the walk.

type IgnoreFileProcessorOptions

type IgnoreFileProcessorOptions struct {
	// Path is the absolute path to the .truss_ignore file.
	Path string
	// Contents is the raw bytes of the .truss_ignore file.
	Contents []byte
}

IgnoreFileProcessorOptions is passed to an IgnoreFileProcessor when a .truss_ignore file is found at the root of the input directory.