storage

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 8, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package storage provides the multi-copy upload orchestration service.

The central types are *Service (high-level upload/download operations), *Context (per-provider store/pull/commit/download operations), and *ServiceResolver (selection + dataset-reuse wiring against warmstorage and spregistry services).

Manager-level operations

Service also exposes manager-level operations:

Per-context manager operations live on *Context: Context.Upload (single-copy), Context.DeletePieceByID, Context.DeletePiece, Context.PieceStatus, Context.GetScheduledRemovals and Context.Terminate.

Upload Flow

The multi-copy upload follows a store → pull → commit pipeline:

  1. Store: Upload data to the primary storage provider.
  2. Pull: Secondary providers fetch data from the primary (SP-to-SP).
  3. Commit: All providers call AddPieces on-chain with EIP-712 signatures.

The Service handles orchestration of the full multi-copy flow, while ServiceResolver reuses provider-local datasets only when metadata matches exactly and the warmstorage-approved provider set intersects active PDP providers from spregistry.

Downloads are validated as they stream so callers can keep io.Reader-style boundaries without skipping PieceCID verification. Context.Download can use a CDN-backed retriever first when the context has CDN enabled, then fall back to provider PDP retrieval on ordinary CDN failures. By default the HTTP download client refuses to dial loopback, link-local, or private (RFC1918 / ULA) addresses to guard against SSRF, and it ignores environment-variable proxies for the same reason; set [Options.AllowPrivateNetworks] when connecting to trusted private infrastructure, or provide [Options.HTTPClient] if you need explicit proxy control. Bound the number of bytes accepted per URL-based Service.Download call via [Options.DownloadMaxBytes]; Context.Download is not subject to this cap.

UploadOptions exposes per-upload lifecycle hooks covering the full store → pull → commit pipeline:

  • [UploadOptions.OnProgress] — bytes streamed to the primary provider.
  • [UploadOptions.OnStored] — primary provider confirmed storage.
  • [UploadOptions.OnPiecesAdded] — on-chain AddPieces transaction submitted (batch-shaped: carries a []SubmittedPiece per provider per commit).
  • [UploadOptions.OnPiecesConfirmed] — on-chain AddPieces transaction confirmed (batch-shaped: carries a []ConfirmedPiece with assigned on-chain IDs).
  • [UploadOptions.OnCopyComplete] — secondary SP-to-SP pull succeeded.
  • [UploadOptions.OnCopyFailed] — a secondary SP-to-SP copy attempt failed (presign failures remain FailedAttempts-only).
  • [UploadOptions.OnPullProgress] — per-piece status update during a secondary pull.

Service.Upload and Context.Upload recover and ignore UploadOptions callback panics. If a logger is configured, the first panic per callback name in an upload is logged as a warning. Direct staged hooks on StoreOptions, PullRequest, and CommitRequest are invoked as-is and are not covered by this recovery guarantee.

Context.Pull checks that each requested piece resolves to a non-empty source URL. The PDP provider performs stricter source-URL validation before executing the provider-to-provider pull.

Callers that need restartable staged uploads can split secondary creation from piece registration: build a context with Service.CreateContext, call Context.CreateDataSet, persist the CreateDataSetSubmission from [CreateDataSetOptions.OnSubmitted], resume with Context.WaitForDataSetCreated if needed, then run Context.Pull and Context.Commit.

Stability

0.x phase: public API may change between minor releases.

Example

Example demonstrates uploading a payload via a storage.Service. In practice a Service is obtained from synapse.Client.Storage after calling synapse.New; this example assumes svc is already wired. 

var svc *storage.Service // obtained from synapse.Client.Storage()

ctx := context.Background()
payload := bytes.NewReader(bytes.Repeat([]byte("data"), 64))

res, err := svc.Upload(ctx, payload, &storage.UploadOptions{
	OnProgress: func(uploaded int64) {
		log.Printf("uploaded %d bytes", uploaded)
	},
	OnStored: func(providerID types.BigInt, pieceCID cid.Cid) {
		log.Printf("stored %s on provider %s", pieceCID, providerID)
	},
	OnPiecesConfirmed: func(dataSetID, providerID types.BigInt, pieces []storage.ConfirmedPiece) {
		log.Printf("provider %s confirmed %d piece(s) in dataset %s", providerID, len(pieces), dataSetID)
	},
})
if err != nil {
	log.Fatal(err)
}
fmt.Println(res.PieceCID)

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrClosed = lifecycle.ErrClosed

ErrClosed is returned when a method is called after the owning Client has been closed. It aliases the shared closed-client sentinel.

View Source 
var ErrInvalidArgument = errors.New("storage: invalid argument")

ErrInvalidArgument is returned, wrapped via fmt.Errorf with %w, when a caller passes an argument that violates a precondition (nil pointer, zero ID, empty required URL, undefined pieceCID, etc.). Match with errors.Is(err, storage.ErrInvalidArgument).

Business / server-invariant errors (no approved providers, duplicate dataset IDs, server returned zero dataSetID, etc.) are intentionally left as plain errors so that errors.Is against ErrInvalidArgument only matches genuine caller-supplied validation failures.

View Source 
var ErrInvalidDownloadOptions = errors.New("storage: invalid download options")

ErrInvalidDownloadOptions is returned when DownloadOptions is nil, empty, or specifies more than one download source. Service.Download wraps it together with ErrInvalidArgument, so callers may match either.

View Source 
var ErrMaxBytesExceeded = errors.New("storage: download exceeded MaxBytes")

ErrMaxBytesExceeded is returned by Service.Download when the response body exceeds Options.DownloadMaxBytes. The error is surfaced either eagerly (via Content-Length when supplied by the server) or via the terminal Read on the returned reader.

View Source 
var ErrPrivateNetwork = errors.New("storage: private / local network address disallowed")

ErrPrivateNetwork is returned by Service.Download when the target URL resolves to a loopback / link-local / RFC1918 / ULA / multicast / unspecified address and the Service was constructed without Options.AllowPrivateNetworks. It prevents SDK callers from being used as SSRF egress against internal networks.

View Source 
var ErrUninitialized = errors.New("storage: service not initialized; use storage.New")

ErrUninitialized is returned when a method is invoked on a zero-value Service (one that was not constructed via New).

View Source 
var ErrUnsupportedScheme = errors.New("storage: unsupported URL scheme")

ErrUnsupportedScheme is returned when the URL passed to Service.Download uses a scheme other than http or https.

Functions

This section is empty.

Types

type Allowances 

type Allowances struct {
	Service         common.Address
	IsApproved      bool
	RateAllowance   *big.Int
	LockupAllowance *big.Int
	RateUsed        *big.Int
	LockupUsed      *big.Int
	MaxLockupPeriod *big.Int
}

Allowances is the client-specific operator-approval view embedded in StorageInfo. Nil when the caller has no wallet or the lookup failed.

type CDNRetriever added in v0.2.0 

type CDNRetriever interface {
	DownloadPiece(context.Context, cid.Cid) (io.ReadCloser, error)
}

CDNRetriever provides optional CDN-backed piece retrieval for Context.

type CIDMismatchError 

type CIDMismatchError struct {
	Expected   cid.Cid
	ComputedV1 cid.Cid
	ComputedV2 cid.Cid
}

CIDMismatchError is returned when the piece CID computed from the downloaded bytes does not match the expected CID. Use errors.As to retrieve the computed and expected CIDs for diagnostic purposes.

func (*CIDMismatchError) Error 

func (e *CIDMismatchError) Error() string

type CommitError 

type CommitError struct {
	ProviderID types.BigInt
	Endpoint   string
	Cause      error
}

CommitError is returned when all on-chain commit attempts fail and no copies are stored. Individual per-provider failures are reported in UploadResult.FailedAttempts.

func (*CommitError) Error 

func (e *CommitError) Error() string

func (*CommitError) Unwrap 

func (e *CommitError) Unwrap() error

type CommitRequest 

type CommitRequest struct {
	Pieces    []PieceInput
	ExtraData []byte // EIP-712 signed payload; nil for the primary (create-or-add path)
	// OnSubmitted is invoked with the transaction hash immediately after the
	// on-chain AddPieces transaction is submitted, before confirmation. It may
	// be nil. Direct Commit calls do not recover callback panics.
	OnSubmitted func(txHash string)
}

CommitRequest triggers on-chain registration of pieces for one provider.

type CommitResult 

type CommitResult struct {
	TransactionID string       // on-chain transaction hash
	DataSetID     types.BigInt // data set that now holds the piece
	PieceIDs      []types.BigInt
	IsNewDataSet  bool // true when a new data set was created by this commit
}

CommitResult is returned by a successful Commit call.

type ConfirmedPiece 

type ConfirmedPiece struct {
	PieceID  types.BigInt
	PieceCID cid.Cid
}

ConfirmedPiece carries the on-chain identity reported by an OnPiecesConfirmed callback.

type Context 

type Context struct {
	// contains filtered or unexported fields
}

Context represents a specific provider + data-set pair and handles storage operations (store, pull, and/or commit) for one upload copy. It is safe for concurrent use.

func NewContext 

func NewContext(provider Provider, client PDPProviderClient, evmSigner signer.EVMSigner, opts ...ContextOption) (*Context, error)

NewContext creates a Context for the given provider and PDP client. provider.ID, provider.ServiceURL, and client are validated here. Signing prerequisites (such as a non-nil signer plus chain/payer/record-keeper options) are validated by the write paths that need them, e.g. PresignForCommit.

func (*Context) Commit 

func (c *Context) Commit(ctx context.Context, req CommitRequest) (*CommitResult, error)

Commit calls the provider's AddPieces or CreateDataSet+AddPieces API and waits for on-chain confirmation. When req.ExtraData is empty, PresignForCommit is called internally to produce the signed payload.

func (*Context) CreateDataSet added in v0.2.0 

func (c *Context) CreateDataSet(ctx context.Context, opts *CreateDataSetOptions) (*CreateDataSetResult, error)

CreateDataSet creates an empty data set for this context's provider and binds the confirmed DataSetID to the context.

func (*Context) DataSetID 

func (c *Context) DataSetID() *types.BigInt

DataSetID returns the Context's bound data set ID, or nil if the Context targets a data set that does not yet exist (will be created on first upload).

func (*Context) DeletePiece 

func (c *Context) DeletePiece(ctx context.Context, pieceCID cid.Cid) (*sdktypes.WriteResult, error)

DeletePiece schedules removal of the first piece matching pieceCID from this context's data set.

A data set can contain multiple piece IDs with the same piece CID. This method resolves pieceCID with PDPVerifier.findPieceIdsByCid using limit=1 and deletes the first returned piece ID. Prefer Context.DeletePieceByID when the on-chain piece ID is available.

The implementation:

  1. Resolves one on-chain pieceID via PDPVerifier.findPieceIdsByCid.
  2. Signs an EIP-712 SchedulePieceRemovals message over (clientDataSetID, pieceID) and ABI-encodes the signature as bytes.
  3. Calls the provider's DELETE /pdp/data-sets/{id}/pieces/{pieceId} endpoint via PDPProviderClient.SchedulePieceDeletion.

The returned WriteResult carries only the transaction hash; there is no on-chain wait.

func (*Context) DeletePieceByID added in v0.2.2 

func (c *Context) DeletePieceByID(ctx context.Context, pieceID sdktypes.BigInt) (*sdktypes.WriteResult, error)

DeletePieceByID schedules removal of the piece identified by its on-chain piece ID from this context's data set.

Prefer this method when the piece ID is available, because piece CID is not guaranteed to be unique within a data set.

func (*Context) Download 

func (c *Context) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)

Download retrieves a piece from CDN when enabled and available, otherwise from the storage provider. Validation is streaming: the integrity check runs at EOF, so callers must inspect the terminal error returned by the last Read (or io.ReadAll).

pieceCID must be a PieceCIDv2. PieceCIDv1 is not accepted on this path because PDP provider only accepts v2 and the raw size needed to normalise v1→v2 is not available here. Use Service.Download with a URL if you only have v1.

func (*Context) GetProviderInfo 

func (c *Context) GetProviderInfo() Provider

GetProviderInfo returns a copy of the Provider that this Context was constructed with. Safe for concurrent use; the returned struct is a deep copy for all non-address fields.

func (*Context) GetScheduledRemovals 

func (c *Context) GetScheduledRemovals(ctx context.Context) ([]types.BigInt, error)

GetScheduledRemovals returns the list of piece ids that have been scheduled for removal from this data set but have not yet been processed.

func (*Context) PieceStatus 

func (c *Context) PieceStatus(ctx context.Context, pieceCID cid.Cid) (*PieceStatus, error)

PieceStatus returns the current status of pieceCID relative to this context's data set and the proving schedule. If the piece is not present in the data set, Exists is false and the other fields are zero-valued.

The call performs up to five concurrent reads: findPieceIdsByCid, getNextChallengeEpoch, BlockNumber, getPDPConfig, and provider info. Any individual failure surfaces as a wrapped error; a nil pdpConfig is tolerated (proof-timing fields remain zero).

func (*Context) PieceURL 

func (c *Context) PieceURL(pieceCID cid.Cid) string

PieceURL returns the HTTPS retrieval URL for the given piece CID on this provider.

func (*Context) PresignForCommit 

func (c *Context) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)

PresignForCommit produces the EIP-712–signed extraData payload for Commit. For a new data set it signs both CreateDataSet and AddPieces; for an existing data set it signs only AddPieces. The returned bytes are opaque to callers.

The operation is CPU/crypto-bound and performs no I/O, but ctx is honoured before each signing step so callers can cancel long batches.

func (*Context) ProviderID 

func (c *Context) ProviderID() types.BigInt

ProviderID returns the provider's numeric ID.

func (*Context) Pull 

func (c *Context) Pull(ctx context.Context, req PullRequest) (*PullResult, error)

Pull asks this provider to fetch pieces from another provider (SP-to-SP transfer). req.ExtraData must be the payload returned by PresignForCommit on this context.

func (*Context) ServiceURL 

func (c *Context) ServiceURL() string

ServiceURL returns the base URL of the provider's PDP HTTP API.

func (*Context) Store 

func (c *Context) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)

Store streams data to the provider and waits for it to be parked. The reader is consumed in a single pass. If opts.PieceCID is defined, the client skips inline commP calculation; otherwise commP is computed during the upload via TeeReader. opts may be nil.

func (*Context) Terminate 

func (c *Context) Terminate(ctx context.Context, opts ...warmstorage.WriteOption) (*types.WriteResult, error)

Terminate schedules termination of this context's data set via the FWSS terminateService entry point. On success the provider stops proving the data set and all contained pieces will be removed on-chain.

opts are forwarded to warmstorage.Service.TerminateDataSet (wait / confirmations / etc.).

func (*Context) Upload 

func (c *Context) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)

Upload stores a single copy of data on this context's provider and commits it on-chain. It is Store + Commit — no fan-out, no Pull — and returns the canonical UploadResult shape used elsewhere in the SDK.

opts may be nil. PieceCID, OnProgress, PieceMetadata, OnStored, OnPiecesAdded, and OnPiecesConfirmed are honoured when present; other UploadOptions fields related to provider selection are ignored because this path does not touch provider selection.

Lifecycle callbacks fired (when opts provides them):

  • OnStored after Store succeeds
  • OnPiecesAdded when the commit transaction is submitted
  • OnPiecesConfirmed after commit is confirmed

func (*Context) WaitForDataSetCreated added in v0.2.0 

func (c *Context) WaitForDataSetCreated(ctx context.Context, submission CreateDataSetSubmission) (*CreateDataSetResult, error)

WaitForDataSetCreated waits for a previously submitted create-dataset transaction and binds the confirmed DataSetID to the context.

func (*Context) WithCDN 

func (c *Context) WithCDN() bool

WithCDN reports whether CDN services are enabled for this Context.

type ContextCostRef 

type ContextCostRef struct {
	DataSetID               *types.BigInt
	Provider                Provider
	CurrentDataSetSizeBytes *big.Int
	WithCDN                 bool
}

ContextCostRef references one prospective upload target for multi-context cost aggregation. A nil DataSetID means "a new data set will be created on this provider". CurrentDataSetSizeBytes is used only when DataSetID is non-nil; zero means the caller does not have the current size handy and wants the floor-price rate. WithCDN determines whether the CDN-fixed lockup is included for this ref.

type ContextFactory 

type ContextFactory func(ResolvedUploadContext, *UploadOptions) (UploadContext, error)

ContextFactory builds an UploadContext from a resolved selection.

type ContextOption 

type ContextOption func(*Context)

ContextOption configures a Context.

func WithCDN 

func WithCDN(enabled bool) ContextOption

WithCDN enables CDN services for the data set and CDN-first downloads when a retriever is configured. When true, a "withCDN" metadata marker is added to the EIP-712 dataset-creation message; the contract activates CDN and applies its configured lockup upon seeing it.

func WithCDNRetriever added in v0.2.0 

func WithCDNRetriever(r CDNRetriever) ContextOption

WithCDNRetriever injects the optional CDN retriever used by Context.Download.

func WithChainID 

func WithChainID(chainID types.ChainID) ContextOption

WithChainID sets the EIP-155 chain ID used for EIP-712 domain separation.

func WithClientDataSetID 

func WithClientDataSetID(id types.BigInt) ContextOption

WithClientDataSetID sets a caller-chosen data-set identifier included in EIP-712 messages. If not provided, a random value is generated on the first PresignForCommit call and reused for the lifetime of this Context.

func WithDataSetID 

func WithDataSetID(id types.BigInt) ContextOption

WithDataSetID pins the context to an existing on-chain data set. When set, Commit issues an AddPieces call instead of CreateDataSet+AddPieces.

func WithDataSetMetadata 

func WithDataSetMetadata(metadata map[string]string) ContextOption

WithDataSetMetadata sets the key-value metadata stored with the data set on creation.

func WithFWSSTerminator 

func WithFWSSTerminator(t FWSSTerminator) ContextOption

WithFWSSTerminator injects the terminator used by Context.Terminate.

func WithLogger added in v0.2.0 

func WithLogger(logger *slog.Logger) ContextOption

WithLogger sets the logger used for internal warnings.

func WithPDPConfigReader 

func WithPDPConfigReader(r PDPConfigReader) ContextOption

WithPDPConfigReader injects a reader for FWSSView PDPConfig. Required by Context.PieceStatus for challenge-window math.

func WithPDPVerifierReader 

func WithPDPVerifierReader(r PDPVerifierReader) ContextOption

WithPDPVerifierReader injects a reader for PDPVerifier contract state. Required by Context.GetScheduledRemovals, Context.PieceStatus and Context.DeletePiece; callers that only Store/Pull/Commit/DeletePieceByID may leave it nil.

func WithPayer 

func WithPayer(payer common.Address) ContextOption

WithPayer sets the EVM address that pays for storage.

func WithRecordKeeper 

func WithRecordKeeper(addr common.Address) ContextOption

WithRecordKeeper sets the FWSS contract address (record-keeper) used for EIP-712 signing and passed to the PDP provider for Pull and dataset creation.

type CopyResult 

type CopyResult struct {
	ProviderID   types.BigInt
	DataSetID    types.BigInt
	PieceID      types.BigInt
	Role         CopyRole
	RetrievalURL string // HTTPS retrieval URL for this piece on the provider.
	IsNewDataSet bool
}

CopyResult describes one successfully committed copy.

type CopyRole 

type CopyRole string

CopyRole identifies the role a provider plays in a multi-copy upload. 

const (
	// CopyRolePrimary is the provider that received the original store.
	CopyRolePrimary CopyRole = "primary"
	// CopyRoleSecondary is a provider that pulled data from the primary.
	CopyRoleSecondary CopyRole = "secondary"
)

type CopyStage 

type CopyStage string

CopyStage identifies the pipeline stage at which a provider attempt failed. 

const (
	CopyStageStore   CopyStage = "store"
	CopyStagePull    CopyStage = "pull"
	CopyStagePresign CopyStage = "presign"
	CopyStageCommit  CopyStage = "commit"
)

type CreateContextOptions 

type CreateContextOptions struct {
	ProviderIDs        []types.BigInt
	DataSetIDs         []types.BigInt
	ExcludeProviderIDs []types.BigInt
	DataSetMetadata    map[string]string
	WithCDN            *bool
}

CreateContextOptions configures Service.CreateContext — the single-copy variant. Same knobs as CreateContextsOptions minus Copies.

WithCDN follows the same tri-state convention as CreateContextsOptions.WithCDN.

type CreateContextsOptions 

type CreateContextsOptions struct {
	Copies             int
	ProviderIDs        []types.BigInt
	DataSetIDs         []types.BigInt
	ExcludeProviderIDs []types.BigInt
	DataSetMetadata    map[string]string
	WithCDN            *bool
}

CreateContextsOptions configures Service.CreateContexts.

Copies controls how many provider copies to create. When zero, the resolver uses its default (two copies when no explicit providers or datasets are pinned; otherwise len(ProviderIDs) / len(DataSetIDs)).

WithCDN is tri-state: nil means inherit the Client-level default configured via synapse.WithCDN; non-nil explicitly overrides for this call. Declare a local variable to take its address: 

b := true
opts := &storage.CreateContextsOptions{WithCDN: &b}

type CreateDataSetOptions added in v0.2.0 

type CreateDataSetOptions struct {
	// OnSubmitted is invoked after the create transaction is submitted and
	// before waiting for confirmation. It may be nil.
	OnSubmitted func(CreateDataSetSubmission)
}

CreateDataSetOptions configures Context.CreateDataSet.

type CreateDataSetResult added in v0.2.0 

type CreateDataSetResult struct {
	TransactionID   string
	DataSetID       types.BigInt
	ClientDataSetID types.BigInt
}

CreateDataSetResult is returned after standalone dataset creation confirms.

type CreateDataSetSubmission added in v0.2.0 

type CreateDataSetSubmission struct {
	TransactionID string
	StatusURL     string
	// ClientDataSetID must be non-nil when resuming a submitted create.
	ClientDataSetID *types.BigInt
}

CreateDataSetSubmission identifies a submitted create-dataset transaction. Persist and restore all fields together; incomplete submissions are rejected.

type DataSetCatalog 

type DataSetCatalog interface {
	GetApprovedProviderIDs(context.Context, types.ListOptions) ([]types.BigInt, error)
	GetClientDataSets(context.Context, common.Address, types.ListOptions) ([]*warmstorage.DataSetInfo, error)
	GetDataSet(context.Context, types.BigInt) (*warmstorage.DataSetInfo, error)
	GetAllDataSetMetadata(context.Context, types.BigInt) (map[string]string, error)
}

DataSetCatalog is the subset of warmstorage.Service used by ServiceResolver.

type DataSetFinder 

type DataSetFinder interface {
	FindDataSets(ctx context.Context, payer common.Address, onlyManaged bool) ([]*DataSetInfo, error)
}

DataSetFinder lists the enriched data sets owned by `payer`. Satisfied by *warmstorage.Service via GetClientDataSetsWithDetails.

type DataSetInfo 

type DataSetInfo = warmstorage.EnhancedDataSetInfo

DataSetInfo aliases warmstorage.EnhancedDataSetInfo, the richer record returned by Service.FindDataSets. Kept as an alias (not a copy) so that callers that already hold *warmstorage.EnhancedDataSetInfo can pass values through without conversion.

type DataSetSizeReader 

type DataSetSizeReader interface {
	GetDataSetSizeBytes(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
}

DataSetSizeReader returns the current on-chain size (bytes) of an existing data set, used by Service.Prepare to price lockup accurately for add-pieces scenarios. Satisfied by an adapter around PDPVerifier.getDataSetLeafCount (leafCount * 32).

type DownloadContext 

type DownloadContext interface {
	Download(context.Context, cid.Cid) (io.ReadCloser, error)
}

DownloadContext provides piece retrieval from a known storage provider.

type DownloadError 

type DownloadError struct {
	URL        string
	StatusCode int   // zero when the failure occurred before receiving a response
	Cause      error // non-nil when the failure has an underlying error (e.g. a network or transport-level error)
}

DownloadError is returned when an HTTP download request fails, either due to a network error or a non-2xx HTTP status code. Use errors.As to access the URL and status code.

func (*DownloadError) Error 

func (e *DownloadError) Error() string

func (*DownloadError) Unwrap 

func (e *DownloadError) Unwrap() error

type DownloadOptions 

type DownloadOptions struct {
	Context DownloadContext // when set, delegates to DownloadContext.Download; mutually exclusive with URL
	URL     string          // direct HTTP or HTTPS URL; validated against pieceCID on read completion
}

DownloadOptions configures a Service.Download call. Exactly one of Context or URL must be set; supplying both, or neither, returns an error matching both ErrInvalidDownloadOptions and ErrInvalidArgument.

type FWSSDataSetReader 

type FWSSDataSetReader interface {
	GetDataSet(ctx context.Context, dataSetID sdktypes.BigInt) (*warmstorage.DataSetInfo, error)
}

FWSSDataSetReader reads an existing data set's on-chain record from the FWSSView contract. Used by Service.CreateContext / Service.CreateContexts to auto-fetch the on-chain ClientDataSetID when the resolver path did not already supply one. Satisfied by *warmstorage.Service (see GetDataSet).

type FWSSTerminator 

type FWSSTerminator interface {
	TerminateDataSet(ctx context.Context, dataSetID sdktypes.BigInt, opts ...warmstorage.WriteOption) (*sdktypes.WriteResult, error)
}

FWSSTerminator terminates an on-chain data set via FWSS.TerminateService. Satisfied by *warmstorage.Service (see TerminateDataSet).

type FailedAttempt 

type FailedAttempt struct {
	ProviderID types.BigInt
	Role       CopyRole
	Stage      CopyStage // pipeline stage where the failure occurred
	Err        error
	Explicit   bool // true when the provider was caller-specified (no auto-retry)
}

FailedAttempt records a provider attempt that did not produce a copy.

type FindDataSetsOptions 

type FindDataSetsOptions struct {
	// Payer overrides the default signer address. Zero means "use the
	// signer configured via Options.SignerAddress".
	Payer common.Address
	// OnlyManaged restricts the returned set to data sets whose
	// record-keeper is the configured FWSS contract.
	OnlyManaged bool
}

FindDataSetsOptions configures Service.FindDataSets. A nil pointer or zero value selects the configured default signer as the payer and disables the "only managed" filter.

type GetStorageInfoOptions 

type GetStorageInfoOptions struct {
	// Client overrides the default signer. Zero means "use the signer
	// configured via Options.SignerAddress"; if that too is zero the
	// allowances section of the returned StorageInfo will be nil.
	Client common.Address
}

GetStorageInfoOptions configures Service.GetStorageInfo. A nil pointer selects the configured default signer as the client address.

type MultiContextCosts 

type MultiContextCosts struct {
	RatePerEpoch         *big.Int
	RatePerMonth         *big.Int
	DepositNeeded        *big.Int
	NeedsFWSSMaxApproval bool
	Ready                bool
}

MultiContextCosts is the aggregate cost view across N upload targets. It is a flat summary rather than a per-context breakdown.

type MultiCostCalculator 

type MultiCostCalculator interface {
	CalculateMultiContextCosts(ctx context.Context, payer common.Address, dataSizeBytes *big.Int, refs []ContextCostRef, opts MultiCostOptions) (*MultiContextCosts, error)
}

MultiCostCalculator computes an upload-cost summary for a fan-out across multiple prospective contexts.

type MultiCostOptions 

type MultiCostOptions struct {
	// EnableCDN forces CDN pricing on every ref (in addition to any
	// per-ref `WithCDN` flag) and governs whether the CDN-fixed lockup
	// is added for new datasets.
	EnableCDN bool
	// ExtraRunwayEpochs is additional runway (epochs) on top of the
	// minimum lockup period. Defaults to 0 when unset.
	ExtraRunwayEpochs int64
	// BufferEpochs is the deposit cushion above current lockup usage to
	// cover transaction latency. Zero uses the cost service default
	// (5 epochs).
	BufferEpochs int64
}

MultiCostOptions customises the multi-context cost calculation.

type Options 

type Options struct {
	// Resolver selects providers for each upload and supplies replacement
	// candidates when a secondary provider fails. A nil resolver is allowed
	// so the Service can still serve DownloadFromContext / download-by-URL
	// calls; Upload then returns a clean validation error.
	Resolver UploadResolver

	// HTTPClient is used for URL-based downloads. nil installs a client with
	// a 24-hour timeout — long enough for multi-GiB transfers over typical
	// storage networks while preventing indefinite hangs. The default client
	// also disables environment-variable proxies to avoid proxy-assisted SSRF
	// bypass. When set, the SDK's built-in SSRF protection is bypassed
	// entirely: the provided Transport is responsible for implementing
	// equivalent safeguards (private-network rejection, DNS-rebind close
	// window, redirect filtering). AllowPrivateNetworks has no effect in this
	// case.
	HTTPClient *http.Client

	// Source is the application identifier for dataset namespace isolation.
	// Datasets with different Source values are treated as separate
	// namespaces; reuse only occurs within the same Source.
	Source string

	// DefaultWithCDN is the Client-level CDN default applied when an
	// UploadOptions.WithCDN / CreateContextsOptions.WithCDN /
	// CreateContextOptions.WithCDN field is nil. Ignored when the
	// per-op field is non-nil.
	DefaultWithCDN bool

	// MaxSecondaryAttempts caps the number of provider candidates tried for
	// each secondary copy slot before giving up. Values <= 0 select the
	// default of 5.
	MaxSecondaryAttempts int

	// CommitConcurrency caps the number of concurrent on-chain Commit RPCs
	// issued across primary + secondary copies. Values <= 0 select the
	// default of 4. The cap only matters when RequestedCopies exceeds it.
	CommitConcurrency int

	// AllowPrivateNetworks disables the default SSRF protection applied to
	// URL-based Service.Download calls. When false (the default), the
	// built-in HTTP client refuses to dial loopback / link-local / RFC1918 /
	// ULA / multicast / unspecified addresses and returns ErrPrivateNetwork.
	// Set to true only when you knowingly need to download from a private
	// network (e.g. in-cluster storage). Ignored when HTTPClient is set.
	AllowPrivateNetworks bool

	// DownloadMaxBytes caps the number of bytes a single URL-based
	// Service.Download will return. Zero (the default) disables the cap.
	// Exceeding it returns ErrMaxBytesExceeded either eagerly (via
	// Content-Length) or at the terminal Read of the returned reader.
	DownloadMaxBytes int64

	// Logger receives internal warnings. nil disables logging.
	Logger *slog.Logger

	// Lifecycle, when non-nil, ties this Service to the owning Client's
	// close state. After the Lifecycle is closed, every method returns
	// ErrClosed. Nil is allowed for standalone use.
	Lifecycle *lifecycle.Lifecycle

	// DataSetFinder backs Service.FindDataSets. Optional; when nil the
	// method returns an ErrUninitialized-wrapped error.
	DataSetFinder DataSetFinder

	// StorageInfoReader backs Service.GetStorageInfo. Optional.
	StorageInfoReader StorageInfoReader

	// DataSetTerminator backs Service.TerminateDataSet. Optional.
	DataSetTerminator FWSSTerminator

	// CostCalculator backs Service.CalculateMultiContextCosts and the
	// cost estimation inside Prepare. Optional.
	CostCalculator MultiCostCalculator

	// PaymentsFunder backs PrepareTransaction.Execute. Optional.
	PaymentsFunder PaymentsFunder

	// DataSetSizeReader backs the per-dataset size lookup performed by
	// Service.Prepare for existing-dataset contexts. Optional; when
	// nil, Prepare falls back to zero sizes (floor-price lockup). For
	// accurate add-pieces pricing, wire an implementation backed by
	// PDPVerifier.getDataSetLeafCount (leafCount * 32 bytes).
	DataSetSizeReader DataSetSizeReader

	// FWSSDataSetReader is used by Service.CreateContext /
	// Service.CreateContexts to auto-fetch the on-chain
	// ClientDataSetID when a context resolves with a pinned dataSetID
	// but the resolver did not already populate clientDataSetID. When
	// nil, the safety net is skipped (existing behaviour).
	FWSSDataSetReader FWSSDataSetReader

	// SignerAddress is the default payer/client used by manager-level
	// helpers (FindDataSets, GetStorageInfo, CreateContext{s}, Prepare)
	// when the caller does not explicitly supply one.
	SignerAddress common.Address
}

Options configures a Service. Unset fields fall back to sensible defaults.

type PDPConfigReader 

type PDPConfigReader interface {
	GetPDPConfig(ctx context.Context) (*warmstorage.PDPConfig, error)
}

PDPConfigReader returns the proving-period configuration from the FWSSView contract. Satisfied by *warmstorage.Service.

type PDPProviderClient added in v0.2.0 

type PDPProviderClient interface {
	UploadPieceStreaming(context.Context, io.Reader, pdp.UploadPieceStreamingOptions) (*pdp.UploadStreamingResult, error)
	DownloadPiece(context.Context, cid.Cid) (io.ReadCloser, int64, error)
	WaitForPieceParked(context.Context, cid.Cid, time.Duration) error
	WaitForPullComplete(context.Context, pdp.PullRequest, time.Duration, func(*pdp.PullResult)) (*pdp.PullResult, error)
	AddPieces(context.Context, types.BigInt, []pdp.AddPieceInput, []byte) (*pdp.AddPiecesResult, error)
	WaitForPiecesAdded(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
	CreateDataSet(context.Context, common.Address, []byte) (*pdp.CreateDataSetResult, error)
	WaitForDataSetCreated(context.Context, string, time.Duration) (*pdp.CreateDataSetStatus, error)
	CreateDataSetAndAddPieces(context.Context, common.Address, []pdp.AddPieceInput, []byte) (*pdp.CreateDataSetResult, error)
	WaitForCreateDataSetAndAddPieces(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
	SchedulePieceDeletion(ctx context.Context, dataSetID, pieceID types.BigInt, extraData []byte) (common.Hash, error)
}

PDPProviderClient is the provider HTTP API surface required by Context. It is injectable for tests and alternate provider clients.

type PDPProviderSource 

type PDPProviderSource interface {
	GetPDPProvider(context.Context, types.BigInt) (*spregistry.PDPProvider, error)
	SelectActivePDPProviders(context.Context, spregistry.ProviderFilter) ([]spregistry.PDPProvider, error)
}

PDPProviderSource is the subset of spregistry.Service used by ServiceResolver.

type PDPVerifierReader 

type PDPVerifierReader interface {
	FindPieceIdsByCid(ctx context.Context, dataSetID sdktypes.BigInt, pieceCID cid.Cid, start, limit uint64) ([]sdktypes.BigInt, error)
	GetScheduledRemovals(ctx context.Context, dataSetID sdktypes.BigInt) ([]sdktypes.BigInt, error)
	GetNextChallengeEpoch(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
	BlockNumber(ctx context.Context) (uint64, error)
}

PDPVerifierReader is the read-only PDPVerifier surface required by Context for piece lifecycle queries (scheduled removals, id lookup, next challenge epoch) and by PieceStatus for proving-window calculations. Implementations convert between sdktypes.BigInt / cid.Cid and the abigen-native types (`*big.Int`, `pdpverifier.CidsCid`).

type PaymentsFunder 

type PaymentsFunder interface {
	FundSync(ctx context.Context, amount *big.Int, opts ...payments.WriteOption) (*sdktypes.WriteResult, error)
}

PaymentsFunder tops up the Payments contract for an upload. Narrow view of payments.Service used by PrepareTransaction.Execute.

type PieceInput 

type PieceInput struct {
	PieceCID      cid.Cid
	PieceMetadata map[string]string // optional key-value metadata stored with the piece
}

PieceInput describes a single piece being committed on-chain.

type PieceStatus 

type PieceStatus struct {
	// Exists reports whether the piece is currently present in the data set.
	Exists bool

	// PieceID is the piece's on-chain numeric id. Only meaningful when
	// Exists is true.
	PieceID types.BigInt

	// DataSetLastProven is the wall-clock time of the most recent
	// proof submission for the enclosing data set. Zero time indicates
	// "never proven" or "unknown".
	DataSetLastProven time.Time

	// DataSetNextProofDue is the wall-clock deadline by which the next
	// proof must be submitted. Zero time indicates "unknown".
	DataSetNextProofDue time.Time

	// RetrievalURL is the HTTPS URL at which the piece can be retrieved
	// from the provider. Empty when provider info is not available.
	RetrievalURL string

	// InChallengeWindow reports whether the data set is currently inside
	// its challenge window (proof submission is required).
	InChallengeWindow bool

	// HoursUntilChallengeWindow is the number of hours between now and
	// the start of the next challenge window. Zero when inside or past
	// the window.
	HoursUntilChallengeWindow float64

	// IsProofOverdue reports whether the proving deadline has passed
	// without a proof being submitted.
	IsProofOverdue bool
}

PieceStatus captures the current state of a piece relative to its data set's proving schedule.

PieceID is zero when the piece is not present in the data set (Exists = false); callers should treat (Exists == false) as the "null" TS return.

type PrepareOptions 

type PrepareOptions struct {
	// DataSize is the payload size (bytes) used to derive per-upload
	// costs when Costs is not pre-supplied.
	DataSize uint64
	// Contexts (optional) restricts cost calculation to the given set
	// of upload contexts. Each element should be a *Context.
	Contexts []UploadContext
	// Costs (optional) short-circuits calculation when supplied.
	Costs *MultiContextCosts
	// EnableCDN is tri-state:
	//   nil         → inherit the Client-level default (synapse.WithCDN)
	//   &true/&false → explicit per-Prepare override
	// Only consulted on the auto-create-context branch (i.e. when neither
	// Costs nor Contexts is supplied); once Contexts are provided, each
	// context's own WithCDN() takes precedence.
	EnableCDN *bool
	// ExtraRunwayEpochs is additional runway (epochs) above the
	// minimum lockup period passed through to the cost calculator. Defaults
	// to 0 when unset.
	ExtraRunwayEpochs int64
	// BufferEpochs is the deposit cushion above current lockup usage
	// used to absorb transaction-latency epochs. Zero falls back to
	// the cost service default (5 epochs).
	BufferEpochs int64
}

PrepareOptions configures Service.Prepare. At most one of Costs or Contexts should be set; when neither is provided, Prepare will call CreateContexts to obtain a default set.

type PrepareResult 

type PrepareResult struct {
	// Costs is the aggregated cost calculation that drove the decision.
	Costs *MultiContextCosts
	// Transaction is non-nil only when funding is required (Ready=false).
	Transaction *PrepareTransaction
}

PrepareResult is the value returned by Service.Prepare.

type PrepareTransaction 

type PrepareTransaction struct {
	// DepositAmount is the USDFC amount that will be moved into the
	// payments account.
	DepositAmount *big.Int
	// IncludesApproval reports whether the call will also set the FWSS
	// operator to max allowance.
	IncludesApproval bool
	// Execute performs the funding operation. Additional
	// payments.WriteOption values are appended to the built-in
	// WithFundNeedsFwssApproval option when applicable.
	Execute func(ctx context.Context, opts ...payments.WriteOption) (*types.WriteResult, error)
}

PrepareTransaction is the deferred funding step returned by Prepare when the account is not yet Ready. Execute performs the top-up.

type PricePerTiB 

type PricePerTiB struct {
	PerMonth *big.Int
	PerDay   *big.Int
	PerEpoch *big.Int
}

PricePerTiB holds the effective price per TiB for one provisioning profile at three time granularities.

type PricingInfo 

type PricingInfo struct {
	NoCDN        PricePerTiB
	WithCDN      PricePerTiB
	TokenAddress common.Address
	TokenSymbol  string
}

PricingInfo is the pricing sub-structure of StorageInfo.

type Provider 

type Provider struct {
	ID              types.BigInt   // provider ID from SPRegistry
	ServiceURL      string         // base URL of the provider's PDP HTTP API
	ServiceProvider common.Address // provider's EVM address
	Payee           common.Address // address that receives payments
}

Provider holds the on-chain identity of a storage provider.

type PullPieceResult 

type PullPieceResult struct {
	PieceCID cid.Cid
	Status   PullStatus
}

PullPieceResult is the per-piece outcome within a PullResult.

type PullRequest 

type PullRequest struct {
	Pieces    []cid.Cid
	From      func(cid.Cid) string // returns the HTTPS URL for a given piece CID
	ExtraData []byte               // EIP-712 signed payload authorising the pull
	// OnProgress is invoked after each piece status update during the pull.
	// It may be nil. Direct Pull calls do not recover callback panics.
	OnProgress func(pieceCID cid.Cid, status PullStatus)
}

PullRequest asks a secondary provider to pull pieces from a primary.

type PullResult 

type PullResult struct {
	Status PullStatus
	Pieces []PullPieceResult
}

PullResult is the aggregate outcome of a pull operation.

type PullStatus 

type PullStatus string

PullStatus is the per-piece or overall status returned by a pull operation. 

const (
	PullStatusPending    PullStatus = "pending"
	PullStatusInProgress PullStatus = "inProgress"
	PullStatusRetrying   PullStatus = "retrying"
	PullStatusComplete   PullStatus = "complete"
	PullStatusFailed     PullStatus = "failed"
)

type ResolvedUploadContext 

type ResolvedUploadContext struct {
	Provider        Provider
	DataSetID       *types.BigInt     // nil when a new data set will be created
	ClientDataSetID *types.BigInt     // stable caller-chosen ID reused across commits; nil for new datasets
	DataSetMetadata map[string]string // metadata carried into the new data set if created
}

ResolvedUploadContext is the pre-selection result for one provider copy.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service orchestrates multi-copy uploads and downloads. Create with New; configure via the Options struct.

func New 

func New(opts Options) (*Service, error)

New creates a Service from the given Options. The returned error is always nil today; callers should still check it for forward compatibility.

func (*Service) CalculateMultiContextCosts 

func (s *Service) CalculateMultiContextCosts(ctx context.Context, dataSizeBytes uint64, refs []ContextCostRef, opts MultiCostOptions, payer common.Address) (*MultiContextCosts, error)

CalculateMultiContextCosts fans out the cost calculation across the given refs and returns an aggregate result.

func (*Service) CreateContext 

func (s *Service) CreateContext(ctx context.Context, opts *CreateContextOptions) (*Context, error)

CreateContext is the single-copy convenience wrapper around CreateContexts.

func (*Service) CreateContexts 

func (s *Service) CreateContexts(ctx context.Context, opts *CreateContextsOptions) ([]*Context, error)

CreateContexts provisions one or more concrete storage contexts without uploading. It picks providers, reuses or creates data sets according to opts, and returns the resulting contexts. When opts is nil or opts.Copies is zero the resolver default (two copies in auto-select) applies.

func (*Service) Download 

func (s *Service) Download(ctx context.Context, pieceCID cid.Cid, opts *DownloadOptions) (io.ReadCloser, error)

Download retrieves a piece by URL or via a DownloadContext. When URL is used, the response body is streamed through a validating reader; the terminal read error from io.ReadAll (or any last Read call that returns io.EOF) carries the integrity check result — callers must not discard it.

func (*Service) FindDataSets 

func (s *Service) FindDataSets(ctx context.Context, opts *FindDataSetsOptions) ([]*DataSetInfo, error)

FindDataSets returns the enriched list of data sets owned by the caller or by the payer in opts.

func (*Service) GetDefaultContext 

func (s *Service) GetDefaultContext(ctx context.Context) (*Context, error)

GetDefaultContext returns a single auto-selected context using resolver defaults.

func (*Service) GetStorageInfo 

func (s *Service) GetStorageInfo(ctx context.Context, opts *GetStorageInfoOptions) (*StorageInfo, error)

GetStorageInfo returns the comprehensive chain-wide storage view needed to author an upload.

func (*Service) Prepare 

func (s *Service) Prepare(ctx context.Context, opts *PrepareOptions) (*PrepareResult, error)

Prepare returns the funding transaction needed (if any) to cover the upload costs across the given or auto-selected contexts.

func (*Service) TerminateDataSet 

func (s *Service) TerminateDataSet(ctx context.Context, dataSetID types.BigInt, opts *TerminateDataSetOptions) (*types.WriteResult, error)

TerminateDataSet terminates an FWSS-managed data set by ID.

func (*Service) Upload 

func (s *Service) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)

Upload runs the multi-copy upload pipeline streaming from r in a single pass. opts may be nil (defaults apply). Returns UploadResult whose Complete field indicates whether all requested copies were committed on-chain.

The reader is consumed once by the primary provider; secondary copies are populated via server-to-server Pulls. On success the reader is fully drained; on error it may be only partially consumed.

Timeouts and cancellation: Upload honours ctx for every step — presign, store, pull and on-chain commit wait. To bound the total upload time (including the blockchain confirmation that populates the returned [UploadResult.Copies]), wrap ctx with context.WithTimeout; the Service itself does not impose an internal wait deadline. The built-in 24h HTTP timeout on Service only affects URL-based downloads; Upload, Pull, and Commit use the UploadContext implementation's own HTTP client configuration.

type ServiceParameters 

type ServiceParameters struct {
	EpochsPerMonth int64
	EpochsPerDay   int64
	EpochDuration  int64 // seconds per epoch
	MinUploadSize  int64
	MaxUploadSize  int64
}

ServiceParameters captures the immutable chain/service geometry that StorageInfo callers want to show alongside pricing.

type ServiceResolver 

type ServiceResolver struct {
	// contains filtered or unexported fields
}

ServiceResolver selects providers and data sets for each upload, reusing existing data sets when metadata matches exactly.

func NewServiceResolver 

func NewServiceResolver(opts ServiceResolverOptions) (*ServiceResolver, error)

NewServiceResolver constructs a ServiceResolver. All fields in opts are required.

func (*ServiceResolver) ResolveUploadContexts 

func (r *ServiceResolver) ResolveUploadContexts(ctx context.Context, opts *UploadOptions) ([]UploadContext, bool, error)

ResolveUploadContexts returns one UploadContext per requested copy. When neither ProviderIDs nor DataSetIDs are set, providers are auto-selected from the warmstorage-approved and active-PDP intersection. The second return value is true when providers were explicitly specified by opts.

func (*ServiceResolver) SelectReplacement 

func (r *ServiceResolver) SelectReplacement(ctx context.Context, usedProviders map[string]types.BigInt, opts *UploadOptions) (UploadContext, error)

SelectReplacement picks a single unused provider to replace a failed attempt.

type ServiceResolverOptions 

type ServiceResolverOptions struct {
	Payer       common.Address // EVM address of the paying account
	SPRegistry  PDPProviderSource
	WarmStorage DataSetCatalog
	NewContext  ContextFactory // called per-provider to construct an UploadContext
}

ServiceResolverOptions configures a ServiceResolver.

type StorageInfo 

type StorageInfo struct {
	Pricing           PricingInfo
	Providers         []spregistry.PDPProvider
	ServiceParameters ServiceParameters
	Allowances        *Allowances
}

StorageInfo aggregates the chain-wide storage view a client needs before authoring an upload.

type StorageInfoReader 

type StorageInfoReader interface {
	GetStorageInfo(ctx context.Context, client common.Address) (*StorageInfo, error)
}

StorageInfoReader returns the chain-wide StorageInfo view for the given client.

type StoreError 

type StoreError struct {
	ProviderID types.BigInt
	Endpoint   string
	Cause      error
}

StoreError is returned when the primary store operation fails.

func (*StoreError) Error 

func (e *StoreError) Error() string

func (*StoreError) Unwrap 

func (e *StoreError) Unwrap() error

type StoreOptions 

type StoreOptions struct {
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the client skips inline commP calculation; the server still
	// verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the reader, with
	// the cumulative bytes sent so far. It may be nil. Direct Store calls do
	// not recover callback panics.
	OnProgress func(bytesUploaded int64)
}

StoreOptions configures a single Context.Store call.

type StoreResult 

type StoreResult struct {
	PieceCID cid.Cid // PieceCIDv2 of the stored data
	Size     int64   // raw (unpadded) byte count
}

StoreResult is returned by a successful Store call.

type SubmittedPiece 

type SubmittedPiece struct {
	PieceCID cid.Cid
}

SubmittedPiece carries the piece identity reported by an OnPiecesAdded callback.

type TerminateDataSetOptions 

type TerminateDataSetOptions struct {
	// WriteOptions are forwarded to warmstorage.TerminateDataSet.
	WriteOptions []warmstorage.WriteOption
}

TerminateDataSetOptions configures Service.TerminateDataSet.

type UploadContext 

type UploadContext interface {
	ProviderID() types.BigInt
	ServiceURL() string
	PieceURL(cid.Cid) string
	Store(context.Context, io.Reader, *StoreOptions) (*StoreResult, error)
	PresignForCommit(context.Context, []PieceInput) ([]byte, error)
	Pull(context.Context, PullRequest) (*PullResult, error)
	Commit(context.Context, CommitRequest) (*CommitResult, error)
}

UploadContext abstracts a single provider's upload operations. Implementations are returned by UploadResolver and are safe for concurrent use.

type UploadOptions 

type UploadOptions struct {
	// Copies is the number of provider copies to store. Zero means the resolver
	// default: len(DataSetIDs) or len(ProviderIDs) when those are set, otherwise 2.
	Copies int
	// PieceMetadata is stored with each piece on-chain.
	PieceMetadata map[string]string
	// DataSetMetadata is stored with the data set on first creation.
	DataSetMetadata map[string]string
	// ProviderIDs pins the upload to specific providers by ID.
	ProviderIDs []types.BigInt
	// DataSetIDs pins the upload to specific existing data sets.
	DataSetIDs []types.BigInt
	// ExcludeProviderIDs skips these providers during auto-selection.
	ExcludeProviderIDs []types.BigInt
	// WithCDN is tri-state: nil inherits the Client-level default
	// configured via synapse.WithCDN; non-nil explicitly overrides
	// for this upload. Declare a local variable to take its address:
	//
	//	b := true
	//	opts := &storage.UploadOptions{WithCDN: &b}
	WithCDN *bool
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the primary provider client skips inline commP calculation;
	// the server still verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the upload
	// reader, with the cumulative bytes sent to the primary provider so
	// far. It may be nil.
	OnProgress func(bytesUploaded int64)
	// OnStored is invoked once the primary provider has confirmed storage of
	// the piece. It may be nil.
	OnStored func(providerID types.BigInt, pieceCID cid.Cid)
	// OnPiecesAdded is invoked after the on-chain AddPieces transaction is
	// submitted for a provider (primary or secondary), carrying the transaction
	// hash and the batch of pieces included in that transaction. During
	// Service.Upload, different providers may invoke this callback
	// concurrently when commitConcurrency > 1. It may be nil.
	OnPiecesAdded func(txHash string, providerID types.BigInt, pieces []SubmittedPiece)
	// OnPiecesConfirmed is invoked after the on-chain AddPieces transaction is
	// confirmed (CommitResult received) for a provider, carrying the assigned
	// on-chain IDs for each piece. During Service.Upload, this callback is
	// invoked sequentially after all commit workers finish. It may be nil.
	OnPiecesConfirmed func(dataSetID, providerID types.BigInt, pieces []ConfirmedPiece)
	// OnCopyComplete is invoked once a secondary provider's SP-to-SP pull
	// completes successfully. It is not fired for the primary (which stores
	// directly). It may be nil.
	OnCopyComplete func(providerID types.BigInt, pieceCID cid.Cid)
	// OnCopyFailed is invoked when a secondary provider's SP-to-SP copy
	// attempt fails. Presign failures are not copy attempts and still surface
	// only through FailedAttempts with CopyStagePresign. Primary store/commit
	// failures likewise surface through the Upload return value and
	// FailedAttempts. It may be nil.
	OnCopyFailed func(providerID types.BigInt, pieceCID cid.Cid, err error)
	// OnPullProgress is invoked for each piece status update during a
	// secondary-provider pull. It may be nil.
	OnPullProgress func(providerID types.BigInt, pieceCID cid.Cid, status PullStatus)
}

UploadOptions configures an Upload call.

Some lifecycle callbacks may be invoked from internal orchestration goroutines. Callers that share mutable state across callbacks must keep their handlers concurrency-safe. Service.Upload and Context.Upload recover and ignore callback panics; when a logger is configured, the first panic per callback name in an upload logs a warning. This recovery does not apply to direct StoreOptions, PullRequest, or CommitRequest hooks.

type UploadResolver 

type UploadResolver interface {
	ResolveUploadContexts(context.Context, *UploadOptions) ([]UploadContext, bool, error)
	SelectReplacement(context.Context, map[string]types.BigInt, *UploadOptions) (UploadContext, error)
}

UploadResolver selects the set of providers for an upload and provides replacement candidates when a secondary provider fails.

type UploadResult 

type UploadResult struct {
	PieceCID        cid.Cid // PieceCIDv2 of the stored data
	Size            int64   // raw (unpadded) byte count
	RequestedCopies int
	// Complete is true when all RequestedCopies were committed on-chain.
	// Equivalent to len(Copies) >= RequestedCopies.
	Complete       bool
	Copies         []CopyResult
	FailedAttempts []FailedAttempt
}

UploadResult is returned by a successful Upload call.

Use Complete to determine overall success: it is true when every requested copy was committed on-chain (equivalent to len(Copies) >= RequestedCopies). A non-empty FailedAttempts slice does NOT indicate overall failure — failed attempts may have been resolved by successful retries on other providers.

Example: 

result, err := m.Upload(ctx, r, opts)
if err != nil { ... }
if !result.Complete {
    log.Printf("partial upload: %d/%d copies", result.SuccessCount(), result.RequestedCopies)
}

func (*UploadResult) PartialSuccess 

func (r *UploadResult) PartialSuccess() bool

PartialSuccess reports whether at least one copy was committed on-chain but fewer than the requested number were obtained. Returns false when Complete is true or when no copies succeeded at all.

func (*UploadResult) PrimaryDataSetID 

func (r *UploadResult) PrimaryDataSetID() (types.BigInt, bool)

PrimaryDataSetID returns the DataSetID of the primary copy.

ok is false when no primary copy committed on-chain (even if secondaries did). Callers that need precise provenance should inspect [UploadResult.Copies] directly.

func (*UploadResult) SuccessCount 

func (r *UploadResult) SuccessCount() int

SuccessCount returns the number of copies that were successfully committed on-chain. Equivalent to len(Copies).

func (*UploadResult) SuccessfulProviderIDs 

func (r *UploadResult) SuccessfulProviderIDs() []types.BigInt

SuccessfulProviderIDs returns the ProviderID of every copy that committed on-chain, in the order the copies appear in [UploadResult.Copies].

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.