client

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 7, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package client provides Go clients for the Baseten management, inference, and sandbox APIs.

Use NewManagementClient to interact with the management API (models, deployments, secrets, etc.), NewInferenceClient to call deployed models and chains, and NewSandboxClient to call a single sandbox.

All clients expose the underlying generated API client via their API() method for direct access to all low-level endpoints.

Index

Constants

This section is empty.

Variables

View Source 
var ErrMalformedVolumeEndpoint = errors.New("the token exchange returned an empty volume endpoint")

ErrMalformedVolumeEndpoint reports a token-exchange response whose volume endpoint is present but unusable — an empty string where a URL belongs. It is a service-side defect, deliberately distinct from ErrNoVolumeAPI's null, which is the deployment saying it has no volume API at all. It stays a sentinel here rather than becoming a reason on the volume error type: reasons report the volume service refusing an operation, and this response comes from the management API's exchange before that service is ever reached — a reason would report a refusal from a service that never spoke.

View Source 
var ErrNoVolumeAPI = errors.New("this environment does not expose a volume API")

ErrNoVolumeAPI reports that the environment has no volume service to talk to. It is a deployment fact rather than a failure, and distinct from a transport error, so a caller can say so plainly instead of surfacing something that reads like an outage.

Functions

func ApplyUserAgentHeader 

func ApplyUserAgentHeader(h http.Header)

ApplyUserAgentHeader builds and sets a Baseten User-Agent on h if one is not already present. The value looks like "baseten-go/0.1.0 (Go/1.25.0; linux)". This is a no-op if the client name has been disabled via SetClientName("").

func ClientName 

func ClientName() string

ClientName returns the current client name used in the Baseten User-Agent header.

func InferenceClientDefaultBaseURL 

func InferenceClientDefaultBaseURL(modelID, chainID, environment string) (string, error)

InferenceClientDefaultBaseURL computes the default inference API base URL from the given options. Exactly one of modelID or chainID must be non-empty.

func InferenceClientHost 

func InferenceClientHost(modelID, chainID, environment, rootApiHost string) (string, error)

InferenceClientHost computes the inference API host (no scheme) from the given options. Exactly one of modelID or chainID must be non-empty. environment is the optional regional environment slug (e.g. "prod-us") that selects a regional deployment; leave empty for the default region. See https://docs.baseten.co/deployment/environments#regional-environments. rootApiHost defaults to "api.baseten.co" when empty.

func SetClientName 

func SetClientName(name string)

SetClientName overrides the product token used in the Baseten User-Agent header (e.g. "baseten-cli"). Defaults to "baseten-go". Set to "" to disable User-Agent injection.

Must be called before constructing any client. The User-Agent value is captured into the client's headers at construction time; later changes do not propagate to existing clients.

Types

type FetchVolumeManifestOptions added in v0.3.0 

type FetchVolumeManifestOptions struct {
	// Ref names the version to read. A bare volume reads the version head
	// points at; a tag or a digest names one point.
	//
	// A path on the ref narrows the read to the entries at or under it,
	// matched on slash boundaries, and a path naming nothing yields no entries
	// rather than an error. [VolumeRef.Path] of "/" narrows nothing, being the
	// whole tree.
	//
	// Use [ParseVolumeRef] to read one from a string.
	Ref VolumeRef

	// EntryFilter keeps the entries it returns true for and drops the rest.
	// Nil keeps everything, which is the whole version and can be very large:
	// a manifest carries one entry per file, directory, and symlink.
	//
	// It narrows what is HELD, never what is read: a version's digest covers
	// the whole manifest, so the document is downloaded and verified either
	// way, and a filter buys memory rather than time. Entries arrive in the
	// order the manifest carried them rather than the sorted order they are
	// returned in, so a filter must decide on the entry in front of it without
	// depending on entries it has not seen.
	//
	// A path on Ref narrows as well, and the two compose: an entry is kept
	// when it is under the path and this returns true for it.
	//
	// ctx is the one driving the read, so it carries the caller's values and
	// is cancelled with it.
	EntryFilter func(ctx context.Context, entry VolumeEntry) bool

	// Hasher returns an unkeyed BLAKE3 hash with a 32-byte digest. Required:
	// the manifest is verified against the digest the version resolved to, and
	// nothing about the tree is reported before that check passes. See
	// [PullVolumeOptions.Hasher] for what to pass.
	Hasher func() hash.Hash

	// Store reads the volume's objects. Required: a manifest is an object like
	// any other, and the service compresses what it stores at its own
	// discretion, so a reader has to be able to download and decompress
	// whatever comes back.
	Store VolumeObjectStore
}

FetchVolumeManifestOptions describes a manifest read.

type InferenceClient 

type InferenceClient struct {
	// contains filtered or unexported fields
}

InferenceClient provides access to the Baseten inference API for a specific model or chain.

func NewInferenceClient 

func NewInferenceClient(opts InferenceClientOptions) (*InferenceClient, error)

NewInferenceClient creates a new InferenceClient.

func (*InferenceClient) API 

func (c *InferenceClient) API() *inferenceapi.Client

API returns the underlying generated inference API client. The generated API surface is not covered by Go compatibility guarantees and may change between versions.

type InferenceClientOptions 

type InferenceClientOptions struct {
	// APIKey is the Baseten API key used for authentication. Required
	// unless DeferAuth is true.
	APIKey string

	// ModelID is the model to target. Mutually exclusive with ChainID.
	ModelID string

	// ChainID is the chain to target. Mutually exclusive with ModelID.
	ChainID string

	// Environment is the optional regional environment slug (e.g. "prod-us")
	// that selects a regional deployment. Leave empty for the default region.
	// See https://docs.baseten.co/deployment/environments#regional-environments.
	Environment string

	// BaseURL overrides the computed inference API base URL. Mutually exclusive
	// with ModelID, ChainID, and Environment.
	BaseURL string

	// HTTPClient overrides the default HTTP client. If nil, http.DefaultClient
	// is used.
	HTTPClient interface {
		Do(*http.Request) (*http.Response, error)
	}

	// DeferAuth, when true, skips the APIKey requirement and does not set
	// any Authorization header. The caller is expected to provide an
	// HTTPClient that injects the appropriate auth header.
	DeferAuth bool

	// Headers are added to every request. The map is cloned at construction,
	// so later mutations by the caller do not affect the live client.
	Headers http.Header
}

InferenceClientOptions configures an InferenceClient.

type ManagementClient 

type ManagementClient struct {
	// contains filtered or unexported fields
}

ManagementClient provides access to the Baseten management API.

func NewManagementClient 

func NewManagementClient(opts ManagementClientOptions) (*ManagementClient, error)

NewManagementClient creates a new ManagementClient.

func (*ManagementClient) API 

func (c *ManagementClient) API() *managementapi.Client

API returns the underlying generated management API client. The generated API surface is not covered by Go compatibility guarantees and may change between versions.

func (*ManagementClient) FetchVolumeManifest added in v0.3.0 

func (c *ManagementClient) FetchVolumeManifest(
	ctx context.Context, opts FetchVolumeManifestOptions,
) (*VolumeManifest, error)

FetchVolumeManifest reads the list of entries in a version of a volume. FetchVolumeManifestOptions configures it.

This is the metadata read behind listing a volume's contents and describing one entry: what the version holds, at what size and mode, and nothing about the bytes. Reading the bytes is ManagementClient.PullVolume, whose EntryHandler streams them without writing a directory.

The document is verified against the digest the version resolved to before any of it is reported, and it is read in one pass rather than held whole, so FetchVolumeManifestOptions.EntryFilter narrowing to one subtree of a large volume holds only that subtree.

Each call exchanges its own capability token and resolves the ref again, sharing nothing with any other call, so this is not currently optimized for many calls against the same volume.

func (*ManagementClient) PullVolume added in v0.3.0 

func (c *ManagementClient) PullVolume(ctx context.Context, opts PullVolumeOptions) (*PullVolumeResult, error)

PullVolume downloads a version of a volume into a directory. PullVolumeOptions configures it.

A volume is a versioned directory tree stored by content, so pushing a tree that mostly matches one already stored transfers only what differs, and downloading the same version twice transfers nothing the second time.

Every chunk is checked against its recorded digest before it is written, so a corrupted or truncated read fails the download rather than producing a file that merely looks complete. An interrupted download can be run again and picks up where it stopped.

Each call exchanges its own capability token and resolves the ref again, sharing nothing with any other call, so this is not currently optimized for many calls against the same volume.

func (*ManagementClient) PushModel added in v0.3.0 

func (c *ManagementClient) PushModel(ctx context.Context, opts PushModelOptions) (*PushModelResult, error)

PushModel deploys a model from a local directory: it validates the push and obtains upload credentials via `POST /v1/prepare_model_upload`, builds and uploads the archive through opts.ModelUploader, then creates the model (`POST /v1/models`) or adds a deployment to an existing one (`POST /v1/models/{model_id}/deployments`).

The returned deployment is not yet live. Poll it to wait for a terminal status.

Set opts.DryRun to validate a push without creating anything.

func (*ManagementClient) PushVolume added in v0.3.0 

func (c *ManagementClient) PushVolume(ctx context.Context, opts PushVolumeOptions) (*PushVolumeResult, error)

PushVolume uploads a directory and publishes it as a new version of a volume. PushVolumeOptions configures it.

A volume is a versioned directory tree stored by content, so pushing a tree that mostly matches one already stored transfers only what differs, and downloading the same version twice transfers nothing the second time.

Nothing is visible until the whole tree has been uploaded, so an interrupted push publishes nothing. What it uploaded is not wasted: the next push finds those objects already stored and skips them.

Each call exchanges its own capability token and shares nothing with any other, so this is not currently optimized for many calls against the same volume.

type ManagementClientOptions 

type ManagementClientOptions struct {
	// APIKey is the Baseten API key used for authentication. Required
	// unless DeferAuth is true.
	APIKey string

	// BaseURL overrides the default management API base URL. If empty,
	// "https://api.baseten.co" is used.
	BaseURL string

	// HTTPClient overrides the default HTTP client. If nil, http.DefaultClient
	// is used.
	//
	// Volume transfers use this client too, and lean on http.Client's
	// contract that Do closes the request body even on error: an
	// implementation that never closes it hangs an upload rather than
	// leaking the buffer behind it. Any real transport already complies.
	HTTPClient interface {
		Do(*http.Request) (*http.Response, error)
	}

	// DeferAuth, when true, skips the APIKey requirement and does not set
	// any Authorization header. The caller is expected to provide an
	// HTTPClient that injects the appropriate auth header (e.g. via a
	// custom RoundTripper). This is intended for CLI use where auth may
	// come from OAuth tokens or other credential sources.
	DeferAuth bool

	// Headers are added to every request. The map is cloned at construction,
	// so later mutations by the caller do not affect the live client.
	Headers http.Header
}

ManagementClientOptions configures a ManagementClient.

type ModelUpload added in v0.3.0 

type ModelUpload struct {
	// Bucket is the destination S3 bucket.
	Bucket string
	// Key is the destination S3 key.
	Key string
	// Region is the AWS region the bucket resides in.
	Region string

	// AccessKeyID, SecretAccessKey, and SessionToken are short-lived STS
	// credentials scoped to Bucket and Key.
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string

	// Body streams the tar archive. It is produced lazily as it is read, so
	// read errors may surface from the walk of the model directory.
	Body io.Reader
}

ModelUpload is the archive and destination handed to a ModelUploader.

type ModelUploader added in v0.3.0 

type ModelUploader func(context.Context, ModelUpload) error

ModelUploader uploads a model archive to the location the server issued credentials for.

type PullVolumeOptions added in v0.3.0 

type PullVolumeOptions struct {
	// Ref names what to download. A bare volume takes the version head points
	// at; a tag or a digest names one point. Whatever it names is resolved once
	// and pinned, so a tag that moves partway through cannot produce a tree
	// assembled from two versions.
	//
	// A path on the ref narrows the download to that path, and does so by
	// being treated exactly as though it had been added to Include: it is
	// volume-root-relative, and giving both a path and Include entries selects
	// the union of them rather than looking for Include entries beneath the
	// path. [VolumeRef.Path] of "/" narrows nothing, being the whole tree.
	//
	// Narrowing does not move anything: an entry lands at its own path within
	// the volume, under DestDir, so a ref of "bdn:weights/llama/config/x.json"
	// writes "<DestDir>/config/x.json" and creates the directories above it.
	// A partial pull therefore lays out exactly as the whole volume would,
	// which is what lets two of them into one directory compose. Directories
	// above the narrowing keep their recorded modes where the volume records
	// them and take 0755 where it does not.
	//
	// Use [ParseVolumeRef] to read one from a string.
	Ref VolumeRef

	// DestDir is where the tree ends up. Unless Overwrite is set it must not
	// exist or must be empty, and the tree is assembled beside it and moved
	// into place only once it is complete.
	//
	// Exactly one of DestDir and EntryHandler is set.
	DestDir string

	// EntryHandler receives every selected entry instead of anything being
	// written to disk, which is what reproducing a version somewhere that is
	// not a filesystem looks like: an archive, a stream, another store.
	//
	// Entries arrive one at a time, in path order, so a directory always
	// precedes what is under it and a subtree is contiguous. Directories and
	// symlinks are included; see [VolumePulledEntry.Reader] for what they
	// read as. Returning an error stops the pull and is what it returns.
	//
	// Because there is no destination directory, Overwrite and Restart do not
	// apply and are refused. A file's chunks are prefetched concurrently within
	// the operation and byte limits, verified, and delivered in file order.
	//
	// ctx is the one driving the pull, so it carries the caller's values and
	// is cancelled with it.
	EntryHandler func(ctx context.Context, entry VolumePulledEntry) error

	// Overwrite writes into an existing DestDir in place. Files already
	// there that the volume does not describe are left alone; a failed
	// download leaves a partly written directory rather than an untouched
	// one.
	Overwrite bool

	// Include narrows the download to named entries. Each is an exact path
	// or a directory whose contents are wanted, matched on slash boundaries,
	// and each is relative to the volume's root whatever else is set. One that
	// matches nothing fails the download rather than silently producing less
	// than was asked for. Empty downloads the whole volume, unless Ref carries
	// a path, which narrows the same way; see [PullVolumeOptions.Ref].
	Include []string

	// Restart discards a partly downloaded tree from an earlier attempt
	// instead of continuing it.
	Restart bool

	// StripRefPath makes DestDir stand for the directory Ref's path names, so
	// that directory's contents land directly in DestDir rather than under the
	// chain of directories that led to them. Pulling
	// "bdn:weights/llama/config/models" into "./out" writes
	// "./out/gpt2.json" with it set, and "./out/config/models/gpt2.json"
	// without it.
	//
	// Requires a path on Ref, since without one there is nothing for DestDir
	// to stand for. It changes only the layout, never what is downloaded.
	//
	// A symlink under the path whose target resolves outside it is refused:
	// stripping makes DestDir a subtree of the volume, and a link reaching
	// above that subtree is not the link the volume records. Entries selected
	// by Include from outside the path are refused for the same reason.
	StripRefPath bool

	// Hasher returns an unkeyed BLAKE3 hash with a 32-byte digest. Required:
	// every chunk is verified against the digest recorded for it, and
	// continuing an interrupted download works by hashing what is already on
	// disk. It is supplied here so this module carries no hashing library of
	// its own, and getting it wrong fails every verification. Both common
	// libraries produce the right hash when told the size explicitly or by
	// default:
	//
	//	github.com/zeebo/blake3:   func() hash.Hash { return blake3.New() }
	//	lukechampine.com/blake3:   func() hash.Hash { return blake3.New(32, nil) }
	//
	// A 64-byte extended output is the mistake to watch for; the hasher is
	// checked against the published test vectors before a transfer starts.
	Hasher func() hash.Hash

	// Store reads the volume's objects. Required: the service compresses
	// what it stores at its own discretion, so a reader has to be able to
	// download and decompress whatever comes back.
	Store VolumeObjectStore

	// Progress is called as the download proceeds, never concurrently with
	// itself.
	Progress func(VolumeProgress)

	// Concurrency overrides the transfer's concurrency limits. The zero
	// value means defaults.
	Concurrency VolumeConcurrencyOptions
}

PullVolumeOptions configures ManagementClient.PullVolume.

type PullVolumeResult added in v0.3.0 

type PullVolumeResult struct {
	// VersionRef is the volume pinned to the version that was downloaded,
	// which is the form to quote to get this exact tree again. Its Digest is
	// the complete sixty-four hex digits, never a prefix: an unkeyed BLAKE3
	// hash with a 32-byte output over the manifest's content bytes. It
	// carries no path whatever narrowed the download.
	VersionRef VolumeRef

	// Files and Bytes are what was written.
	Files int64
	Bytes int64

	// SelectedFiles and TotalFiles report what Include narrowed to. They are
	// equal when the whole volume was downloaded.
	SelectedFiles int64
	TotalFiles    int64

	// ChunksFetched and ChunksReused partition the chunks. Reused were
	// already on disk with the right contents, from an earlier attempt.
	ChunksFetched int64
	ChunksReused  int64

	// Warnings are the containment findings that did not stop the download —
	// a dangling link, a link through a file, an entry whose parent has no
	// record, a root-anchored path normalized on the way out. Volumes
	// published before the containment rule carry these; they are written
	// out faithfully and reported here rather than silently.
	Warnings []VolumeWarning
}

PullVolumeResult is what a download produced.

type PushModelOptions added in v0.3.0 

type PushModelOptions struct {
	// ModelID is an existing model to add a deployment to. When empty, a new
	// model is created, named by the `model_name` key of Config.
	ModelID string

	// TeamID is the team the new model belongs to. Only valid when creating a
	// model (ModelID empty); defaults to the organization's default team.
	TeamID string

	// Config is the parsed model config. Required, and must carry a non-empty
	// `model_name` when creating a model.
	Config map[string]any

	// RawConfig is the original config.yaml text, persisted verbatim on the
	// deployment for display and download. The server never builds from it, so
	// it may keep comments and formatting the parsed Config has lost. Optional.
	RawConfig string

	// Archive describes the tar to upload. Archive.Dir is required, and at
	// minimum holds the model's config.yaml.
	//
	// This package does not parse or serialize YAML, so callers whose config
	// has `external_package_dirs` must handle the coupling between the config
	// and the archive themselves: copy the value into
	// Archive.ExternalPackageDirs, set Archive.BundledPackagesDir from the
	// config's `bundled_packages_dir` (canonically "packages"), remove
	// `external_package_dirs` from Config, and set Archive.ConfigYAMLOverride
	// to that cleared config. The external directories are inlined into the
	// archive under the bundled packages dir, so a config that still names
	// them fails server-side validation on relative paths the extracted
	// archive does not contain.
	Archive modelarchive.BuildModelArchiveOptions

	// ModelUploader uploads the built archive to the S3 location the server
	// issued credentials for. Required unless DryRun is set. Callers supply
	// this so this package takes no dependency on an AWS SDK.
	//
	// It is not called for model formats built without an archive (for example
	// BIS-LLM), where the server issues no upload target.
	ModelUploader ModelUploader

	// DryRun validates the push without creating anything: the config is
	// checked server-side and the archive is built and read to completion, but
	// never uploaded. Building it exercises the ignore rules, the external
	// package dirs, and the archive paths, none of which the server can see.
	// The returned result carries ArchiveBytes but no model or deployment.
	DryRun bool

	// DeploymentName is an optional human-readable name for the deployment.
	// Names are unique per model, so a literal name can be pushed only once;
	// leave empty to let the server assign `deployment-N`.
	DeploymentName string

	// EnvironmentName is the stable environment to push to (for example
	// "production"). When empty, the deployment is created without
	// environment selection.
	EnvironmentName string

	// CreateEnvironmentIfMissing creates the environment named by
	// EnvironmentName when the model does not have it yet. Without it, a push
	// naming an environment that does not exist is rejected. Only meaningful
	// with EnvironmentName set to something other than "production", which
	// every model has.
	CreateEnvironmentIfMissing bool

	// Region is the region in which to deploy the model. When empty, the
	// server chooses. Unrelated to [ModelUpload.Region], which is the AWS
	// region of the archive's upload bucket.
	Region string

	// Labels are user-provided key-value labels for the deployment. They can
	// only be set at creation.
	Labels map[string]any

	// DeployTimeoutMinutes overrides the server's deploy timeout. Allowed
	// range is 10 to 1440; zero leaves the server default.
	DeployTimeoutMinutes int

	// PreserveEnvInstanceType retains the target environment's current
	// instance type instead of applying the one in Config. Only meaningful
	// with EnvironmentName set to an environment that already exists.
	PreserveEnvInstanceType bool

	// IsDevelopment pushes to the model's single mutable development slot,
	// created if absent and overwritten in place otherwise. DeploymentName,
	// EnvironmentName, and PreserveEnvInstanceType must be left unset.
	IsDevelopment bool

	// UserEnv is client environment metadata (for example client version,
	// Python version), validated server-side.
	UserEnv map[string]any

	// DisableArchiveDownload prevents the uploaded archive from being
	// downloadable later. Only valid when creating a model.
	DisableArchiveDownload bool
}

PushModelOptions configures ManagementClient.PushModel.

func (PushModelOptions) Validate added in v0.3.0 

func (o PushModelOptions) Validate() error

Validate reports whether the options describe a push that can be attempted, including the archive's own preconditions. ManagementClient.PushModel calls it before its first request, so a bad push fails before the server issues upload credentials.

type PushModelResult added in v0.3.0 

type PushModelResult struct {
	// Model is the model the deployment belongs to, whether newly created or
	// already existing. Nil for a dry run.
	Model *managementapi.Model

	// Deployment is the created deployment. It has only just been created, so
	// its status is an in-progress one; callers that need it live must poll.
	// Nil for a dry run.
	Deployment *managementapi.Deployment

	// ModelCreated reports whether the push created the model rather than
	// adding a deployment to an existing one.
	ModelCreated bool

	// ArchiveBytes is the size of the built archive: the bytes read by the
	// uploader, or by the dry run that discarded it. Zero for a push of a
	// format built without an archive, which never builds one.
	ArchiveBytes int64
}

PushModelResult is the outcome of ManagementClient.PushModel.

type PushVolumeOptions added in v0.3.0 

type PushVolumeOptions struct {
	// Ref names the volume to publish to. The volume is created if it does not
	// exist; the namespace must already.
	//
	// It must name a volume and nothing more specific. A path is refused
	// because a push publishes a whole tree rather than writing into one, and
	// a tag or digest is refused because a selector picks out an existing
	// version everywhere else, where here it would be setting one: apply tags
	// with Tags instead.
	//
	// Use [ParseVolumeRef] to read one from a string.
	Ref VolumeRef

	// SourceDir is the directory to push. It is walked without following
	// symlinks, which are recorded as links.
	SourceDir string

	// Tags are applied atomically with the publish, so a tag never points at
	// a version that is only partly uploaded. The reserved name "head" is
	// not a tag; see RequireHeadMove.
	Tags []string

	// RequireHeadMove fails the push before it uploads anything if the
	// credential does not carry permission to move the volume's head.
	// Without it, such a push still publishes the version, and refs without
	// a tag keep resolving to the previous one — reported as HeadMoveDenied.
	//
	// See [PushVolumeResult.HeadMoveDenied]: with a credential from this
	// client's own exchange, the condition this guards against is not
	// expected to arise.
	RequireHeadMove bool

	// Hasher returns an unkeyed BLAKE3 hash with a 32-byte digest. Required —
	// the digest is the whole content-addressing scheme, so getting it wrong
	// produces a volume no other client can read, and it is supplied here so
	// this module carries no hashing library of its own. Both common
	// libraries produce the right hash when told the size explicitly or by
	// default:
	//
	//	github.com/zeebo/blake3:   func() hash.Hash { return blake3.New() }
	//	lukechampine.com/blake3:   func() hash.Hash { return blake3.New(32, nil) }
	//
	// A 64-byte extended output is the mistake to watch for; the hasher is
	// checked against the published test vectors before a transfer starts.
	Hasher func() hash.Hash

	// Store lets the push read the volume's previous version, so a file
	// whose bytes have not changed is not uploaded again. Optional: without
	// it the push uploads everything, which is slower and produces the same
	// version.
	Store VolumeObjectStore

	// Progress is called as the push proceeds, never concurrently with
	// itself. It should return promptly.
	Progress func(VolumeProgress)

	// Concurrency overrides the transfer's concurrency limits. The zero
	// value means defaults.
	Concurrency VolumeConcurrencyOptions

	// SourceURI records where the tree came from, and is inside the bytes
	// the version's digest covers: two pushes of identical trees from
	// different paths are different versions. Empty derives it from
	// SourceDir, which means a push from a different absolute path publishes
	// a new version of an unchanged tree. Set it to a stable string to avoid
	// that.
	SourceURI string
}

PushVolumeOptions configures ManagementClient.PushVolume.

type PushVolumeResult added in v0.3.0 

type PushVolumeResult struct {
	// VersionRef is the volume pinned to the version this push published,
	// which is what to pull to get this exact tree back. Its Digest is the
	// complete sixty-four hex digits, never a prefix: an unkeyed BLAKE3 hash
	// with a 32-byte output over the manifest's content bytes.
	VersionRef VolumeRef

	// Sequence is the volume's snapshot sequence at this publish.
	Sequence int64

	// HeadUpdated reports whether this push moved head. It is false when
	// head already pointed at this exact version, which is what re-pushing
	// an unchanged tree does — so false does not mean untagged refs resolve
	// elsewhere, only that nothing had to move.
	HeadUpdated bool

	// HeadMoveDenied reports that the push did not ask to move head, because
	// the credential's grants do not cover it. The version was published and
	// can be reached by digest or tag; what did not happen is head moving to
	// it.
	//
	// Not expected to occur with a credential obtained the way this client
	// obtains one. The exchange grants push and tag together or refuses
	// outright, and the credential it mints covers every volume and tag in
	// the namespaces it names — so a credential that can push can also move
	// head. The field describes a real state of the underlying protocol,
	// which admits credentials minted other ways, and reports what this
	// client actually asked for rather than asserting what the service
	// would allow.
	HeadMoveDenied bool

	TagsApplied []string

	// Files and Bytes are what the source tree held.
	Files int64
	Bytes int64

	// Chunks counts every object the push accounted for, and Unique, Reused,
	// and Existing partition it. Reused never reached the network, because a
	// previous version or an earlier file in the same push already had those
	// bytes; Existing was sent and the service already had it.
	Chunks   int64
	Unique   int64
	Reused   int64
	Existing int64
}

PushVolumeResult is what a push published.

type SandboxClient added in v0.4.0 

type SandboxClient struct {
	// contains filtered or unexported fields
}

SandboxClient provides access to the Baseten sandbox API for a specific sandbox.

func NewSandboxClient added in v0.4.0 

func NewSandboxClient(opts SandboxClientOptions) (*SandboxClient, error)

NewSandboxClient creates a new SandboxClient.

func (*SandboxClient) API added in v0.4.0 

func (c *SandboxClient) API() *sandboxapi.Client

API returns the underlying generated sandbox API client. The generated API surface is not covered by Go compatibility guarantees and may change between versions. The caller must close the body of a response returned by a Raw method.

type SandboxClientOptions added in v0.4.0 

type SandboxClientOptions struct {
	// Token is the sandbox token used for authentication, minted from an API
	// key through the management API. Required unless DeferAuth is true.
	Token string

	// BaseURL is the sandbox URL returned by the management API. Required.
	BaseURL string

	// HTTPClient overrides the default HTTP client. If nil, http.DefaultClient
	// is used.
	HTTPClient interface {
		Do(*http.Request) (*http.Response, error)
	}

	// DeferAuth, when true, skips the Token requirement and does not set any
	// Authorization header. The caller is expected to provide an HTTPClient
	// that injects the appropriate auth header.
	DeferAuth bool

	// Headers are added to every request. The map is cloned at construction,
	// so later mutations by the caller do not affect the live client.
	Headers http.Header
}

SandboxClientOptions configures a SandboxClient.

type VolumeConcurrencyOptions added in v0.3.0 

type VolumeConcurrencyOptions struct {
	// FileJobs is how many files are processed concurrently on push. Zero
	// means the default.
	FileJobs int

	// ChunkOperations pins how many object operations may be in flight. Zero
	// does not mean "some default": it means the limit is not pinned, and a
	// transfer adapts it to what the origin will bear. Setting it caps the
	// load a transfer places on a shared machine or a metered link, and is
	// honoured exactly.
	ChunkOperations int

	// MaxBytesInFlight caps the chunk data resident in memory. Zero means
	// the two-gibibyte default.
	MaxBytesInFlight int64
}

VolumeConcurrencyOptions is a transfer's concurrency limits. The zero value means defaults: concurrent files follow the pinned object-operation count when one is set and otherwise default to 256, in-flight object operations adapt to what the origin will bear, and two gibibytes of chunk data may be resident in memory.

A wide fan-out is served best by an injected HTTPClient whose transport raises MaxIdleConnsPerHost toward the operation ceiling: the default transport parks only two idle connections per host, so most requests in a wide wave open a fresh connection instead of reusing one.

type VolumeEntry added in v0.3.0 

type VolumeEntry struct {
	// Path is the entry's full path within the version, slash-prefixed and
	// with no trailing slash, spelled exactly as [VolumeRef.Path] spells one.
	// So building a ref for an entry is an assignment and needs no separator
	// of its own:
	//
	//	ref := manifest.Ref
	//	ref.Path = entry.Path
	//	fmt.Println(ref) // bdn:weights/llama@a1b2c3/config/model.json
	//
	// A directory carries no trailing slash either; Kind is what says it is
	// one, and a trailing slash would not survive the round trip through a ref.
	Path string

	Kind VolumeEntryKind

	// Size is a file's length in bytes, and zero for the other kinds: a
	// directory has no size of its own, and a symlink's would be its target's,
	// which a volume does not record.
	Size int64

	// Mode is the recorded permission bits, including setuid, setgid, and
	// sticky, which a container root filesystem legitimately carries. Zero
	// when nothing in the volume describes the entry, which happens for a
	// directory implied only by the paths beneath it.
	Mode uint32

	// ModTime is the modification time recorded when the version was
	// published, and zero when none was recorded.
	ModTime time.Time

	// LinkTarget is a symlink's target exactly as it was recorded, and empty
	// for every other kind. It may point outside the volume, so a caller
	// reproducing it somewhere is responsible for whatever that means there.
	LinkTarget string
}

VolumeEntry describes one entry of a version of a volume: what it is and what it looks like, not how its bytes are stored.

There is deliberately no content digest. A version's own digest names the whole tree, but a per-file one is not something every entry has: a file small enough to fit one chunk carries that chunk's digest, which is its content, while a larger file carries the digest of the document listing its chunks, which is not. A field meaning two different things and absent for the common case would be worse than none.

type VolumeEntryKind added in v0.3.0 

type VolumeEntryKind string

VolumeEntryKind says what an entry is. 

const (
	VolumeEntryKindFile      VolumeEntryKind = "file"
	VolumeEntryKindDirectory VolumeEntryKind = "directory"
	VolumeEntryKindSymlink   VolumeEntryKind = "symlink"
)

type VolumeError added in v0.3.0 

type VolumeError struct {
	// Reason is the stable constant to branch on. A message is for humans
	// and changes between releases.
	Reason VolumeErrorReason

	// Message is the service's human-readable description.
	Message string

	// Err is the underlying error, when there is one.
	Err error
}

VolumeError is what a volume operation returns when the volume service refused or failed it. Match with errors.As: 

var ve *client.VolumeError
if errors.As(err, &ve) && ve.Reason == client.VolumeErrorReasonCASConflict { ... }

Errors that are not the service's own — a local filesystem failure, a cancelled context, a malformed response — come back as themselves, wrapped with context, not as a VolumeError with an invented reason.

func (*VolumeError) Error added in v0.3.0 

func (e *VolumeError) Error() string

func (*VolumeError) Unwrap added in v0.3.0 

func (e *VolumeError) Unwrap() error

type VolumeErrorReason added in v0.3.0 

type VolumeErrorReason string

VolumeErrorReason is the stable constant naming what specifically went wrong with a volume operation. It mirrors the service's own reason strings, so a reason the service adds tomorrow flows through a VolumeError without new API here — the constants below are the ones a caller has a reason to branch on, not the whole registry. 

const (
	// VolumeErrorReasonUploadSessionExpired means the push outlived its
	// session. There is no way to extend one, so the work has to start over.
	VolumeErrorReasonUploadSessionExpired VolumeErrorReason = "UPLOAD_SESSION_EXPIRED"

	// VolumeErrorReasonCASConflict means someone else published while this
	// push was running: the view of the volume was stale.
	VolumeErrorReasonCASConflict VolumeErrorReason = "CAS_CONFLICT"

	VolumeErrorReasonNotFound           VolumeErrorReason = "NOT_FOUND"
	VolumeErrorReasonPermissionDenied   VolumeErrorReason = "PERMISSION_DENIED"
	VolumeErrorReasonAmbiguousPrefix    VolumeErrorReason = "AMBIGUOUS_PREFIX"
	VolumeErrorReasonRateLimited        VolumeErrorReason = "RATE_LIMITED"
	VolumeErrorReasonChunkTooLarge      VolumeErrorReason = "CHUNK_TOO_LARGE"
	VolumeErrorReasonManifestTooLarge   VolumeErrorReason = "MANIFEST_TOO_LARGE"
	VolumeErrorReasonServiceUnavailable VolumeErrorReason = "UNAVAILABLE"
)

type VolumeManifest added in v0.3.0 

type VolumeManifest struct {
	// VersionRef is the volume pinned to the version that was read, carrying
	// no path, so naming one of the entries below is an assignment:
	//
	//	ref := manifest.VersionRef
	//	ref.Path = manifest.Entries[0].Path
	//
	// It is the form to quote to read this exact version again. Its Digest is
	// the complete sixty-four hex digits, never a prefix: an unkeyed BLAKE3
	// hash with a 32-byte output over the manifest's content bytes.
	VersionRef VolumeRef

	// EntryCount and TotalSize describe the WHOLE version, as the manifest
	// itself accounts for it, whatever a filter or a ref path narrowed the
	// entries below to. That is what they are for: with a narrowing they
	// cannot be recovered from Entries, and recovering them would mean
	// holding every entry, which is what narrowing avoids. TotalSize sums the
	// files; directories and symlinks have no size of their own.
	EntryCount int64
	TotalSize  int64

	// Entries are the entries that survived the narrowing, in path order:
	// every entry follows its parent and a subtree is contiguous, so one pass
	// can list a directory's children or walk a tree. A directory implied only
	// by the paths beneath it is not an entry of its own.
	Entries []VolumeEntry
}

VolumeManifest is what one version of a volume holds.

type VolumeObjectCredentials added in v0.3.0 

type VolumeObjectCredentials struct {
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
}

VolumeObjectCredentials are the short-lived read-only credentials the volume service leases for reading a namespace's objects.

type VolumeObjectDownload added in v0.3.0 

type VolumeObjectDownload struct {
	// Endpoint is empty for AWS itself and a base URL otherwise. The two
	// address buckets differently, which is why the distinction is passed
	// through rather than resolved here.
	Endpoint string
	Region   string
	Bucket   string
	Key      string

	Credentials VolumeObjectCredentials

	// ExpectedSize is the object's size when it is known, and zero when it
	// is not. A chunk's length comes from the manifest, so it is known; a
	// manifest's own size is not.
	ExpectedSize int64
}

VolumeObjectDownload names one object to read from the store.

type VolumeObjectResult added in v0.3.0 

type VolumeObjectResult struct {
	Body io.ReadCloser

	// ContentType is the stored media type, and is the only thing that says
	// how the bytes are encoded. The service decides at write time whether
	// to compress an object, and the key does not record what it chose, so
	// a reader that guessed from the key would eventually guess wrong.
	ContentType string

	// Size is the stored length, which for a compressed object is the
	// compressed length rather than the object's own.
	Size int64
}

VolumeObjectResult is an open object.

type VolumeObjectStore added in v0.3.0 

type VolumeObjectStore interface {
	// DownloadObject opens one object for reading.
	DownloadObject(ctx context.Context, req VolumeObjectDownload) (*VolumeObjectResult, error)

	// Decompressor wraps a reader of zstd-compressed bytes in one that
	// yields the original bytes.
	Decompressor(r io.Reader) (io.ReadCloser, error)
}

VolumeObjectStore reads stored objects. The caller supplies it so this module needs no cloud SDK or compression library of its own; the two methods are one interface because they are only ever usable together — the service decides per object whether to compress what it stores, so a reader that can download but not decompress will fail on an arbitrary subset of objects. aws-sdk-go-v2's GetObject and github.com/klauspost/compress/zstd fill the two methods.

DownloadObject owns its own retrying: an error returned from it has already exhausted whatever budget the implementation has, and ends the operation that asked for the object.

type VolumePhase added in v0.3.0 

type VolumePhase string

VolumePhase names the part of a transfer that progress is being reported for. 

const (
	VolumePhaseScan     VolumePhase = "scan"
	VolumePhaseUpload   VolumePhase = "upload"
	VolumePhaseCommit   VolumePhase = "commit"
	VolumePhaseResolve  VolumePhase = "resolve"
	VolumePhaseDownload VolumePhase = "download"
	VolumePhasePublish  VolumePhase = "publish"
)

type VolumeProgress added in v0.3.0 

type VolumeProgress struct {
	Phase VolumePhase

	Files      int64
	TotalFiles int64

	// Bytes counts logical file bytes processed, including reused chunks,
	// rather than network traffic. Uploads and downloads to disk advance it
	// as chunks complete.
	Bytes      int64
	TotalBytes int64
}

VolumeProgress is a transfer's state at one moment. Counts within a phase only ever increase.

type VolumePulledEntry added in v0.3.0 

type VolumePulledEntry struct {
	VolumeEntry

	// Reader is the entry's contents. It is never nil, and reports EOF
	// immediately for a directory or a symlink, so a handler that copies
	// before switching on Kind produces nothing rather than panicking.
	//
	// Valid only for the duration of the handler call: the buffer behind it
	// and its share of the transfer's in-flight byte budget are reclaimed as
	// soon as the handler returns, so a retained Reader reads freed memory.
	// There is nothing to close, and a handler that reads none of it, or stops
	// partway, costs nothing and leaks nothing.
	Reader io.Reader
}

VolumePulledEntry is one entry delivered to a PullVolumeOptions.EntryHandler, together with the means to read it.

type VolumeRef added in v0.3.0 

type VolumeRef struct {
	// Namespace is set on every parsed ref. The zero VolumeRef is not a ref.
	Namespace string

	// Volume is empty on a namespace ref.
	Volume string

	// Tag and Digest are the selector, and at most one is set. Digest is 1 to
	// 64 lowercase hex characters, optionally preceded by "b3:", and it is
	// held and rendered in whichever of those two spellings it was read in.
	// Every digest this package writes for itself carries the prefix and the
	// whole 64, which is the form a REST response carries and so is directly
	// comparable to one. Fewer than 64 characters is a prefix; whether one is
	// long enough to accept, and whether it matches more than one version, are
	// decided where it is resolved, so a prefix this type accepts can still be
	// refused by the service.
	Tag    string
	Digest string

	// Path is the position within the point, spelled the way a URL spells a
	// path rather than the way a manifest entry does: it is slash-prefixed,
	// so the empty string means the ref names no path at all and "/" means
	// the tree's root. Segments are percent-decoded, and neither "." nor
	// ".." can appear. A trailing slash is accepted on input and dropped,
	// since no operation distinguishes "x" from "x/".
	//
	// [VolumeEntry.Path] is spelled the same way, so assigning one here builds
	// the ref for that entry with no separator to add.
	Path string
}

VolumeRef names a namespace, a volume, a point within a volume, or a position within the tree that point presents.

A ref with no selector names the volume's head, which moves. A ref with a tag names whatever that tag currently points at, which also moves. Only a ref pinned to a digest names one fixed version, which is why a transfer resolves once and then works from the pin: a tag that moved partway through would otherwise produce a tree assembled from two versions.

func ParseVolumeRef added in v0.3.0 

func ParseVolumeRef(s string) (VolumeRef, error)

ParseVolumeRef reads a ref in the grammar the volume service documents.

Namespace and volume are accepted in any case and folded to lowercase, as the service folds them before validation, storage, and authorization. Tags are never folded and are case-sensitive.

func (VolumeRef) Level added in v0.3.0 

func (r VolumeRef) Level() VolumeRefLevel

Level reports the most specific component the ref names.

func (VolumeRef) ShorthandString added in v0.3.0 

func (r VolumeRef) ShorthandString() string

ShorthandString renders the ref the way VolumeRef.String does, except that a whole digest is shortened to its first 12 hex characters and loses its "b3:" prefix, which is how a digest prefix is conventionally written. It is for a display where the whole 64 would crowd out everything beside it; the result still names the same version and still parses.

A digest that is already a prefix is left alone, since it is as short as the ref's author was willing to make it. Everything else renders identically, so this shortens the digest and nothing else.

func (VolumeRef) String added in v0.3.0 

func (r VolumeRef) String() string

String renders the ref canonically, which is the form to print, to hand back in a result, and to paste into a config. A namespace ref renders with a trailing slash, names render lowercase, a digest renders in the spelling the ref holds, and path segments are percent-encoded again.

type VolumeRefLevel added in v0.3.0 

type VolumeRefLevel int

VolumeRefLevel is the most specific component a ref names. Every operation that takes a ref accepts some levels and rejects others, so the level is what decides whether a ref is usable before anything is fetched. 

const (
	// VolumeRefLevelNamespace: "bdn:ns".
	VolumeRefLevelNamespace VolumeRefLevel = iota + 1
	// VolumeRefLevelVolume: "bdn:ns/vol", naming the volume itself.
	VolumeRefLevelVolume
	// VolumeRefLevelPoint: "bdn:ns/vol:tag" or "bdn:ns/vol@digest", naming one
	// point within the volume.
	VolumeRefLevelPoint
	// VolumeRefLevelPath: "bdn:ns/vol/x/y" or "bdn:ns/vol:tag/x/y", naming a
	// position within the tree the point presents.
	VolumeRefLevelPath
)

func (VolumeRefLevel) String added in v0.3.0 

func (l VolumeRefLevel) String() string

type VolumeWarning added in v0.3.0 

type VolumeWarning struct {
	// Path is the entry the finding is about.
	Path string
	// Kind is the finding, typed.
	Kind VolumeWarningKind
	// Detail says what was found, in prose.
	Detail string
}

VolumeWarning is a containment finding that did not stop a download: harmless to write out, present in volumes published before the containment rule, and worth telling the caller about.

func (VolumeWarning) String added in v0.3.0 

func (w VolumeWarning) String() string

String renders the finding for a human; the typed fields are the API.

type VolumeWarningKind added in v0.3.0 

type VolumeWarningKind uint8

VolumeWarningKind names what a download's containment check found, so a caller can branch without parsing prose. 

const (
	// VolumeWarningKindDanglingLink: a symlink's target resolves to a path
	// the volume has nothing at.
	VolumeWarningKindDanglingLink VolumeWarningKind = iota + 1
	// VolumeWarningKindLinkThroughFile: a symlink resolves through an entry
	// recorded as a file, which a real filesystem answers with ENOTDIR.
	VolumeWarningKindLinkThroughFile
	// VolumeWarningKindParentUnrecorded: an entry's ancestors up to the
	// nearest recorded one are all implicit — nothing records the parent
	// directory.
	VolumeWarningKindParentUnrecorded
	// VolumeWarningKindPathNormalized: the entry's path was recorded
	// root-anchored — "/a/b" — and was normalized to a relative one on
	// download. Only volumes published before the containment rule carry
	// that shape; pushing it is refused, so this warning marks a legacy
	// volume, never a fresh push.
	VolumeWarningKindPathNormalized
)

Source Files

View all Source files

Directories

Path Synopsis
Package inferenceapi is a generated client for the Baseten inferenceapi.
 Package inferenceapi is a generated client for the Baseten inferenceapi.
Package managementapi is a generated client for the Baseten managementapi.
 Package managementapi is a generated client for the Baseten managementapi.
Package modelarchive builds uncompressed tar archives of model directories for upload to Baseten.
 Package modelarchive builds uncompressed tar archives of model directories for upload to Baseten.
Package modelconfig contains generated types for Baseten model configuration (also known as Truss config).
 Package modelconfig contains generated types for Baseten model configuration (also known as Truss config).
Package sandboxapi is a generated client for the Baseten sandboxapi.
 Package sandboxapi is a generated client for the Baseten sandboxapi.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.