file

package
v1.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 8, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package file implements every "data goes in or out of Bee" endpoint: raw bytes (/bytes), files (/bzz), chunks (/chunks), single-owner chunks (/soc), feeds (/feeds), and tar-packed collections (/bzz on a directory).

Get a *Service handle from github.com/ethswarm-tools/bee-go.Client.File.

Headline pieces:

  • Upload / download primitives — UploadData, DownloadData, UploadFile, DownloadFile, UploadChunk, DownloadChunk, UploadSOC, ProbeData.
  • Collections — UploadCollection (filesystem walk + tar), in-memory UploadCollectionEntries, offline HashDirectory / HashCollectionEntries (compute the manifest reference without uploading), and StreamDirectory / StreamCollectionEntries for chunk-by-chunk uploads with a per-chunk progress callback.
  • Feeds — MakeFeedReader, MakeFeedWriter, MakeFeedIdentifier, FeedUpdateChunkReference, IsFeedRetrievable, AreAllSequentialFeedsUpdateRetrievable, FetchLatestFeedUpdate.
  • Single-owner chunks — MakeSOCReader, MakeSOCWriter wrappers around pkg/swarm.MakeSingleOwnerChunk.

Mirrors bee-js's Bee.uploadData / uploadFile / uploadFiles / uploadFilesFromDirectory / streamDirectory / uploadChunk / uploadSOC / downloadData / downloadFile / downloadChunk / makeFeedReader / makeFeedWriter / fetchLatestFeedUpdate / makeSOCReader / makeSOCWriter fan-out.

Streaming vs. buffered transfers

Downloads return io.ReadCloser backed by the live HTTP body. Drain them with io.Copy for large payloads — io.ReadAll buffers the full reference in memory and will OOM on multi-GB downloads. Always Close the returned reader.

Uploads accept io.Reader and stream the body to Bee. The chunk-by-chunk variants (Service.StreamDirectory and Service.StreamCollectionEntries) bound peak memory at the BMT chunk size regardless of file size and emit a per-chunk progress callback; the tar-based Service.UploadCollection keeps the tar stream itself in memory while it is being assembled.

Cancellation

Cancelling the context.Context aborts the in-flight HTTP request. For Service.StreamDirectory / Service.StreamCollectionEntries, chunks already accepted by the local Bee node remain in the local reserve but the manifest is not finalized — the resulting orphan chunks are eventually pruned but cost reserve space until then.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CollectionSize 

func CollectionSize(entries []CollectionEntry) int64

CollectionSize returns the cumulative byte size of the entries. Mirrors bee-js getCollectionSize.

func FeedUpdateChunkReference 

func FeedUpdateChunkReference(owner swarm.EthAddress, topic swarm.Topic, index uint64) (swarm.Reference, error)

FeedUpdateChunkReference returns the SOC chunk address for the feed update at (owner, topic, index): keccak256(identifier || owner). Use this with DownloadChunk to verify retrievability of past updates. Mirrors bee-js getFeedUpdateChunkReference.

func HashCollectionEntries 

func HashCollectionEntries(entries []CollectionEntry) (swarm.Reference, error)

HashCollectionEntries chunks each entry through the streaming BMT chunker and assembles a Mantaray manifest, returning the manifest's root reference without uploading anything.

Mirrors bee-js hashDirectory but takes in-memory entries so callers can use it from any platform (no filesystem required).

func HashDirectory 

func HashDirectory(dir string) (swarm.Reference, error)

HashDirectory walks dir, chunks each file through the streaming BMT chunker, assembles a Mantaray manifest, and returns the manifest's root reference without uploading anything. Mirrors bee-js hashDirectory.

File contents are read in full per file (the chunker streams within a file but each file is read with os.Open + the chunker drives reads). Directory walk follows fs.WalkDir order.

func MakeFeedIdentifier 

func MakeFeedIdentifier(topic swarm.Topic, index uint64) (swarm.Identifier, error)

MakeFeedIdentifier returns the identifier for a feed update at the given topic + index: keccak256(topic || BE-uint64(index)). Mirrors bee-js makeFeedIdentifier.

Types

type CollectionEntry 

type CollectionEntry struct {
	Path string
	Data []byte
}

CollectionEntry is one file in an in-memory collection. Path is the relative tar entry path (e.g. "index.html" or "assets/logo.png"); Data is the file contents.

Mirrors the bee-js Collection entry shape (path + data) without the browser-only File object — Go callers supply bytes directly.

type FeedReader 

type FeedReader struct {
	Owner swarm.EthAddress
	Topic swarm.Topic
	// contains filtered or unexported fields
}

FeedReader reads updates from a feed identified by (Owner, Topic). Mirrors bee-js FeedReader. Construct via Service.MakeFeedReader.

func (*FeedReader) Download 

func (r *FeedReader) Download(ctx context.Context) (FeedUpdate, error)

Download returns the latest feed update.

func (*FeedReader) DownloadReference 

func (r *FeedReader) DownloadReference(ctx context.Context) (swarm.Reference, FeedUpdate, error)

DownloadReference returns the latest feed update interpreted as `timestamp(8) || reference(32)` and gives back the embedded reference. Returns an error if the payload is not the expected length.

Use this when the feed stores references (the common case for mutable content) — the writer side is UpdateFeedWithReference.

type FeedUpdate 

type FeedUpdate struct {
	Payload   []byte
	Index     uint64
	IndexNext uint64
}

FeedUpdate is the result of a feed lookup. Payload is the raw chunk payload (timestamp + data, when written via UpdateFeedWithIndex). Index is the feed index of the returned update; IndexNext is the index where the *next* update should be written (Index + 1 for sequential feeds).

Mirrors bee-js FeedPayloadResult.

type FeedWriter 

type FeedWriter struct {
	*FeedReader
	// contains filtered or unexported fields
}

FeedWriter is a FeedReader plus update methods. Construct via Service.MakeFeedWriter, which derives the owner from the signer.

func (*FeedWriter) Upload 

func (w *FeedWriter) Upload(ctx context.Context, batchID swarm.BatchID, data []byte) (api.UploadResult, error)

Upload writes `data` to the next feed index, wrapped as `timestamp(8) || data`. Mirrors bee-js FeedWriter.uploadPayload.

func (*FeedWriter) UploadAtIndex 

func (w *FeedWriter) UploadAtIndex(ctx context.Context, batchID swarm.BatchID, index uint64, data []byte) (api.UploadResult, error)

UploadAtIndex writes `data` at a specific feed index, bypassing FindNextIndex. Useful for replaying or seeding feeds at known indexes.

func (*FeedWriter) UploadReference 

func (w *FeedWriter) UploadReference(ctx context.Context, batchID swarm.BatchID, ref swarm.Reference) (api.UploadResult, error)

UploadReference writes a Reference to the next feed index. Mirrors bee-js FeedWriter.upload / .uploadReference.

type ReferenceInformation 

type ReferenceInformation struct {
	ContentLength int64
}

ReferenceInformation is the result of ProbeData: the size of the data at a /bytes reference, learned via HEAD without downloading the body.

type SOCReader 

type SOCReader struct {
	// contains filtered or unexported fields
}

SOCReader downloads SOCs for a known owner. Mirrors bee-js SOCReader.

func (*SOCReader) Download 

func (r *SOCReader) Download(ctx context.Context, id swarm.Identifier) (*swarm.SingleOwnerChunk, error)

Download fetches the SOC at keccak256(identifier || owner), parses the wire form, and verifies the recovered signer matches the owner.

func (*SOCReader) Owner 

func (r *SOCReader) Owner() swarm.EthAddress

Owner returns the address whose SOCs this reader downloads.

type SOCWriter 

type SOCWriter struct {
	*SOCReader
	// contains filtered or unexported fields
}

SOCWriter is a SOCReader that can also upload signed SOCs. Mirrors bee-js SOCWriter.

func (*SOCWriter) Upload 

func (w *SOCWriter) Upload(ctx context.Context, batchID swarm.BatchID, id swarm.Identifier, data []byte, opts *api.UploadOptions) (api.UploadResult, error)

Upload signs and uploads a SOC for identifier with the given payload. Returns the SOC reference (= keccak256(identifier || owner)).

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service is the data-transfer endpoint group: bytes, files, chunks, SOCs, feeds, and tar-packed collections. Get one from the top-level Client.File field rather than constructing it directly.

func NewService 

func NewService(baseURL *url.URL, httpClient *http.Client) *Service

NewService wires up a file.Service against a Bee base URL and HTTP client. The top-level bee.NewClient calls this for you.

func (*Service) AreAllSequentialFeedsUpdateRetrievable 

func (s *Service) AreAllSequentialFeedsUpdateRetrievable(ctx context.Context, owner swarm.EthAddress, topic swarm.Topic, index uint64, opts *api.DownloadOptions) (bool, error)

AreAllSequentialFeedsUpdateRetrievable verifies that every feed-update chunk from index 0 through `index` (inclusive) is currently retrievable from the network. Returns true only if all are present.

Used to validate that a feed can be replayed from its origin. Mirrors bee-js areAllSequentialFeedsUpdateRetrievable.

func (*Service) CreateFeedManifest 

func (s *Service) CreateFeedManifest(ctx context.Context, batchID swarm.BatchID, owner swarm.EthAddress, topic swarm.Topic) (swarm.Reference, error)

CreateFeedManifest creates a feed manifest for the (owner, topic) pair.

func (*Service) DownloadChunk 

func (s *Service) DownloadChunk(ctx context.Context, ref swarm.Reference, opts *api.DownloadOptions) ([]byte, error)

DownloadChunk fetches a single chunk's bytes.

func (*Service) DownloadData 

func (s *Service) DownloadData(ctx context.Context, ref swarm.Reference, opts *api.DownloadOptions) (io.ReadCloser, error)

DownloadData downloads raw bytes from Bee. nil opts means "use Bee defaults"; pass DownloadOptions to specify ACT, redundancy strategy or chunk-retrieval timeout.

The returned ReadCloser must be closed by the caller.

func (*Service) DownloadFile 

func (s *Service) DownloadFile(ctx context.Context, ref swarm.Reference, opts *api.DownloadOptions) (io.ReadCloser, api.FileHeaders, error)

DownloadFile downloads a file from Bee. Returns the body reader, the parsed file headers (Content-Disposition / Content-Type / Swarm-Tag-Uid) and any error. The caller must close the reader.

func (*Service) FetchLatestFeedUpdate 

func (s *Service) FetchLatestFeedUpdate(ctx context.Context, owner swarm.EthAddress, topic swarm.Topic) (FeedUpdate, error)

FetchLatestFeedUpdate downloads the most recent update for (owner, topic) by hitting GET /feeds. The response body is the wrapped chunk payload; swarm-feed-index / swarm-feed-index-next headers carry the indexes (BE-uint64 hex). Mirrors bee-js fetchLatestFeedUpdate.

func (*Service) FindNextIndex 

func (s *Service) FindNextIndex(ctx context.Context, owner swarm.EthAddress, topic swarm.Topic) (uint64, error)

FindNextIndex returns the index where the next feed update should be written. Returns 0 if the feed has no updates yet (Bee responds 404). Mirrors bee-js findNextIndex.

func (*Service) GetFeedLookup 

func (s *Service) GetFeedLookup(ctx context.Context, owner swarm.EthAddress, topic swarm.Topic) (swarm.Reference, error)

GetFeedLookup retrieves the latest feed update lookup for the (owner, topic) pair. Bee returns 200 OK with body {"reference": "..."}.

func (*Service) IsFeedRetrievable 

func (s *Service) IsFeedRetrievable(ctx context.Context, owner swarm.EthAddress, topic swarm.Topic, index *uint64, opts *api.DownloadOptions) (bool, error)

IsFeedRetrievable reports whether the feed at (owner, topic) currently resolves on the network.

If index is nil, only the latest feed update is checked (a weaker guarantee, since "latest" is observer-dependent). If index is non-nil, every sequential chunk up to and including that index is checked — see AreAllSequentialFeedsUpdateRetrievable.

Mirrors bee-js Bee.isFeedRetrievable: 404 / 500 from Bee become (false, nil); other errors propagate.

func (*Service) MakeFeedReader 

func (s *Service) MakeFeedReader(owner swarm.EthAddress, topic swarm.Topic) *FeedReader

MakeFeedReader returns a FeedReader bound to (owner, topic).

func (*Service) MakeFeedWriter 

func (s *Service) MakeFeedWriter(signer swarm.PrivateKey, topic swarm.Topic) *FeedWriter

MakeFeedWriter returns a FeedWriter for (signer, topic). The owner is derived from the signer's public key.

func (*Service) MakeSOCReader 

func (s *Service) MakeSOCReader(owner swarm.EthAddress) *SOCReader

MakeSOCReader returns a reader for SOCs owned by owner. Convenience wrapper around DownloadChunk + the keccak256(identifier || owner) addressing rule. Mirrors bee-js Bee.makeSOCReader.

func (*Service) MakeSOCWriter 

func (s *Service) MakeSOCWriter(signer *ecdsa.PrivateKey) (*SOCWriter, error)

MakeSOCWriter returns a writer that signs uploads with signer. The reader half is keyed off the owner address derived from signer. Mirrors bee-js Bee.makeSOCWriter.

func (*Service) ProbeData 

func (s *Service) ProbeData(ctx context.Context, ref swarm.Reference) (ReferenceInformation, error)

ProbeData fetches the content length for a /bytes reference using a HEAD request. Useful for sizing downloads or validating a ref points at retrievable data without paying the bandwidth cost.

Mirrors bee-js Bee.probeData.

func (*Service) StreamCollectionEntries 

func (s *Service) StreamCollectionEntries(ctx context.Context, batchID swarm.BatchID, entries []CollectionEntry, opts *StreamOptions) (api.UploadResult, error)

StreamCollectionEntries is the upload-as-you-go counterpart to HashCollectionEntries: each leaf and intermediate file chunk is uploaded as it's produced, then the Mantaray manifest is uploaded recursively. Mirrors bee-js streamFiles for the entries shape (the browser-only File[]/FileList path is intentionally omitted).

Useful when files are large enough that collecting them all in memory before tar-ing (the UploadCollectionEntries path) is wasteful or when callers want a deterministic per-file content reference.

func (*Service) StreamDirectory 

func (s *Service) StreamDirectory(ctx context.Context, batchID swarm.BatchID, dir string, opts *StreamOptions) (api.UploadResult, error)

StreamDirectory walks dir and streams each file through the BMT chunker, uploading each chunk as it's produced; finally the Mantaray manifest is uploaded recursively. Returns the manifest root. Mirrors bee-js streamDirectory.

func (*Service) UpdateFeed 

func (s *Service) UpdateFeed(ctx context.Context, batchID swarm.BatchID, signer swarm.PrivateKey, topic swarm.Topic, data []byte) (api.UploadResult, error)

UpdateFeed updates a feed at the next available index by first calling FindNextIndex. data is wrapped as `BE-uint64(timestamp) || data` in the SOC payload, exactly like UpdateFeedWithIndex. Mirrors bee-js updateFeedWithPayload.

func (*Service) UpdateFeedWithIndex 

func (s *Service) UpdateFeedWithIndex(ctx context.Context, batchID swarm.BatchID, signer swarm.PrivateKey, topic swarm.Topic, index uint64, data []byte) (api.UploadResult, error)

UpdateFeedWithIndex updates a feed at the specified index.

The feed identifier is keccak256(topic || BE-uint64(index)). The chunk payload is BE-uint64(timestamp) || data. The chunk is signed via SOC and uploaded.

func (*Service) UpdateFeedWithReference 

func (s *Service) UpdateFeedWithReference(ctx context.Context, batchID swarm.BatchID, signer swarm.PrivateKey, topic swarm.Topic, ref swarm.Reference, index *uint64) (api.UploadResult, error)

UpdateFeedWithReference updates a feed to point at the given Swarm reference. The chunk payload is `timestamp(8) || reference(32)` — matches bee-js updateFeedWithReference. If index is nil, FindNextIndex is called.

func (*Service) UploadChunk 

func (s *Service) UploadChunk(ctx context.Context, batchID swarm.BatchID, data []byte, opts *api.UploadOptions) (api.UploadResult, error)

UploadChunk uploads a single raw chunk (span + payload, up to 4096 bytes of payload).

func (*Service) UploadCollection 

func (s *Service) UploadCollection(ctx context.Context, batchID swarm.BatchID, dir string, opts *api.CollectionUploadOptions) (api.UploadResult, error)

UploadCollection uploads a directory tree as a tar stream via POST /bzz. indexFile (e.g. "index.html") is the document served when the collection root is requested; opts may further specify an error document.

func (*Service) UploadCollectionEntries 

func (s *Service) UploadCollectionEntries(ctx context.Context, batchID swarm.BatchID, entries []CollectionEntry, opts *api.CollectionUploadOptions) (api.UploadResult, error)

UploadCollectionEntries packages the given entries as a tar stream and uploads them via POST /bzz, same as UploadCollection but without touching the filesystem. Useful for programmatic site generation, test fixtures, or callers that already hold the files in memory.

Mirrors bee-js makeCollectionFromFileList + bzz.uploadCollection.

func (*Service) UploadData 

func (s *Service) UploadData(ctx context.Context, batchID swarm.BatchID, data io.Reader, opts *api.RedundantUploadOptions) (api.UploadResult, error)

UploadData uploads raw bytes to Bee. The body is sent as application/octet-stream; for typed files use UploadFile.

Returns an UploadResult that exposes the content reference, the optional auto-created tag UID, and (when ACT was requested) the history address.

func (*Service) UploadFile 

func (s *Service) UploadFile(ctx context.Context, batchID swarm.BatchID, data io.Reader, name string, contentType string, opts *api.FileUploadOptions) (api.UploadResult, error)

UploadFile uploads a single file via POST /bzz. name is the displayed filename (sent as the `name` query parameter); contentType becomes the stored MIME type and may be overridden by FileUploadOptions.ContentType.

If contentType is empty and opts does not specify one, application/ octet-stream is used.

func (*Service) UploadSOC 

func (s *Service) UploadSOC(ctx context.Context, batchID swarm.BatchID, owner swarm.EthAddress, id swarm.Identifier, signature swarm.Signature, data []byte, opts *api.UploadOptions) (api.UploadResult, error)

UploadSOC uploads a Single Owner Chunk. The owner / id / signature triple uniquely addresses the chunk on the network.

type StreamOptions 

type StreamOptions struct {
	api.CollectionUploadOptions
	OnProgress func(UploadProgress)
}

StreamOptions tunes the Stream* methods. Inherits CollectionUploadOptions for index/error documents and adds a progress callback.

type UploadProgress 

type UploadProgress struct {
	Total     int
	Processed int
}

UploadProgress is the per-chunk progress signal passed to Stream* callers. Total / Processed are running counters of leaf+intermediate chunks; Total may be 0 if the chunker doesn't precompute it.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.