piecehub

README

piecehub

A Filecoin piece storage server for curio market.

Installation

go install github.com/strahe/piecehub/cmd/piecehub@latest

Quick Start

1. Simple Directory Mode:

piecehub dir /data/pieces1 [/data/pieces2 ...]

For more options:

piecehub dir -h

1. Simple S3 Mode:

piecehub s3 --endpioint xx --ak xx --sk xx bucket1 [bucket2 ...]

For more options:

piecehub s3 -h

1. Hybrid Storage Mode:

Create a configuration file config.toml:

[server]
address = ":8080"
read_timeout = 30
write_timeout = 30

[[disks]]
name = "local1"
root_dir = "/data/pieces1"
max_size = 1073741824  # 1GB
direct_io = true

[[disks]]
name = "local2"
root_dir = "/data/pieces2"
max_size = 1073741824  # 1GB
direct_io = true

[[s3s]]
name = "remote1"
endpoint = "https://s3.amazonaws.com"
region = "us-east-1"
bucket = "my-pieces2"
access_key = "xxx"
secret_key = "xxx"
use_ssl = false
prefix = ""

[[s3s]]
name = "remote2"
endpoint = "https://s3.amazonaws.com"
region = "us-east-1"
bucket = "my-pieces2"
access_key = "xxx"
secret_key = "xxx"
use_ssl = false
prefix = ""

Start the server:

piecehub -c config.toml

API

Check Piece Existence

HEAD /pieces?id=<pieceCid>

Get Piece Data

GET /pieces?id=<pieceCid>

List Storage Name

GET /storages

Examples

Using curl:

# Check if piece exists
curl -I "http://localhost:8080/pieces?id=<pieceCid>"

# Download piece
curl -O "http://localhost:8080/pieces?id=<pieceCid>"

# Generatge car file
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"size":268435456,"storageName":"test-storage-name"}' \
  http://localhost:8080/debug/generate-car

# Response
{
    "pieceCid":"baga6ea4seaqb46zh6n4fig7nuf5lmfylxr4flmzu2tgfjm6k4werggcnp3fvspy",
    "pieceSize":536870912,
    "payloadSize":268445499,
    "carSize":268445499,
    "carCid":"bafkreibq4fevl27rgurgnxbp7adh42aqiyd6ouflxhj3gzmcxcxzbh6lla"
}