bifrost-gateway

README

bifrost-gateway

A lightweight IPFS Gateway daemon backed by a remote data store.

• About
• Usage
  • Local build
  • Configuration
  • Docker
• FAQ
  • How to run with local gateway as a backend
  • How to run with Saturn CDN backend
  • How to debug
  • How to use tracing
  • How does this work at ipfs.io and dweb.link
• Contributing
• License

Maintainers

IPFS Stewards

About

bifrost-gateway provides a single binary daemon implementation of HTTP+Web Gateway Specs.

It is capable of serving requests to:

• path gateway at 127.0.0.1
• subdomain gateway requests with Host: localhost
• DNSLink gateway based on Host header

Supported response types include both deserialized flat files, and verifiable Block/CAR.

For more information about IPFS Gateways, see:

• Specifications at specs.ipfs.tech/http-gateways
• Conformance tests at ipfs/gateway-conformance
• Go implementation at boxo/gateway

Usage

Local build

$ go build
$ ./bifrost-gateway --help

Configuration

See ./bifrost-gateway --help and ./docs/environment-variables.md

Docker

Official Docker images are provided at hub.docker.com/r/ipfs/bifrost-gateway.

• 🟢 Releases
  • latest and release always point at the latest release
  • vN.N.N point at a specific release tag
• 🟠 Developer builds
  • main-latest always points at the HEAD of the main branch
  • main-YYYY-DD-MM-GITSHA points at a specific commit from the main branch
• ⚠️ Experimental, unstable builds
  • staging-latest always points at the HEAD of the staging branch
  • staging-YYYY-DD-MM-GITSHA points at a specific commit from the staging branch
  • This tag is used by developers for internal testing, not intended for end users

When using Docker, make sure to pass necessary config via -e:

$ docker pull ipfs/bifrost-gateway:release
$ docker run --rm -it --net=host -e PROXY_GATEWAY_URL=http://127.0.0.1:8080  ipfs/bifrost-gateway:release

See ./docs/environment-variables.md.

FAQ

How to use other gateway as a block backend

All you need is a trustless gateway endpoint that supports verifiable response types. The minimum requirement is support for GET /ipfs/cid with application/vnd.ipld.raw (block by block).

To run against a compatible, local trustless gateway provided by Kubo or IPFS Desktop:

$ PROXY_GATEWAY_URL="http://127.0.0.1:8080" ./bifrost-gateway

See Proxy Backend in ./docs/environment-variables.md

How to run with Saturn CDN backend

Saturn is a CDN that provides a pool of trustless gateways.

bifrost-gateway supports it via the Caboose backend, which takes care of discovering and evaluating Block/CAR gateways (in Saturn called L1 nodes/peers) for increased availability.

See Saturn Backend in ./docs/environment-variables.md

How to debug

See GOLOG_LOG_LEVEL.

How to use tracing

For tracing configuration, please check boxo/docs/tracing.md on how to generate the traceparent HTTP header in order to be able to easily identify specific requests.

How does this work at ipfs.io and dweb.link

This is WIP, but the high level architecture is to move from

Old Kubo-based architecture:

graph LR
    A(((fa:fa-person HTTP</br>clients)))
    K[[Kubo]]
    N(((BGP Anycast,<br>HTTP load-balancers,<br>TLS termination)))

    D(((DHT)))

    P((( IPFS<br>Peers)))

    A -->| Accept: text/html, *| N
    A -->| Accept: application/vnd.ipld.raw | N
    A -->| Accept: application/vnd.ipld.car | N
    A -->| Accept: application/vnd.ipld.dag-json | N
    A -->| Accept: application/vnd.ipld.dag-cbor | N
    A -->| Accept: application/json | N
    A -->| Accept: application/cbor | N
    A -->| Accept: application/x-tar | N
    A -->| Accept: application/vnd.ipfs.ipns-record | N
    A -->| DNSLink Host: en.wikipedia-on-ipfs.org | N
    A -->| Subdomain Host: cid.ipfs.dweb.link | N

    N ==>| fa:fa-link HTTP GET <br> Content Path | K

    K -.- D
    K ===|fa:fa-cube bitswapl | P
    P -.- D

New Rhea architecture:

graph LR
    A(((fa:fa-person HTTP</br>clients)))
    B[[bifrost-gateway]]
    N(((BGP Anycast,<br>HTTP load-balancers,<br>TLS termination)))
    S(((Saturn<br>CDN)))
    I[[IPNI]]
    D(((DHT)))

    P((( IPFS<br>Peers)))

    A -->| Accept: text/html, *| N
    A -->| Accept: application/vnd.ipld.raw | N
    A -->| Accept: application/vnd.ipld.car | N
    A -->| Accept: application/vnd.ipld.dag-json | N
    A -->| Accept: application/vnd.ipld.dag-cbor | N
    A -->| Accept: application/json | N
    A -->| Accept: application/cbor | N
    A -->| Accept: application/x-tar | N
    A -->| Accept: application/vnd.ipfs.ipns-record | N
    A -->| DNSLink Host: en.wikipedia-on-ipfs.org | N
    A -->| Subdomain Host: cid.ipfs.dweb.link | N

    N ==>| fa:fa-link HTTP GET <br> Content Path | B
    
    B ==>|fa:fa-cube HTTP GET <br> Blocks | S
    S -.- I 
    I -.- D 
    D -.- P -.- I
  
    P ===|fa:fa-cube the best block/dag <br> transfer protocol | S

bifrost-gateway nodes are responsible for processing requests to:

• path gateway at ipfs.io
• subdomain gateway at dweb.link
• selected DNSLink websites on their own domain names.

Caveats:

• IPFS Gateway interface based on reference implementation from boxo/gateway.
• IPFS Backend based on https://saturn.tech and HTTP client talking to it via caboose with STRN_LOGGER_SECRET.
  • Learn more at Project Rhea (decentralized IPFS gateway)
• Functional gaps facilitated by temporary delegation to legacy Kubo RPC (/api/v0) at https://node[0-3].delegate.ipfs.io infra (already used by js-ipfs).

How does high level overview look like

Some high level areas:

mindmap
  root[bifrost-gateway]
    (boxo/gateway.IPFSBackend)
        Block Backend
        CAR Backend
    Ephemeral Storage
        Block Cache
        Exchange Backend
            Plain HTTP Fetch
            Caboose Saturn Fetch
    Resolving Content Paths
        Raw
        CAR
        UnixFS
        IPLD Data Model
            [DAG-JSON]
            [DAG-CBOR]
        Web
            HTTP Host Header
            HTML dir listings
            index.html
            _redirects
            HTTP Range Requests
        Namesys
            DNSLink
                EoDoH<br>ENS over DNS over HTTPS
            IPNS Records
    Metrics and Tracing
        Prometheus
            Counters
            Histograms
        OpenTelemetry
            Spans
            Exporters
            Trace Context

Contributing

Contributions are welcome! This repository is part of the IPFS project and therefore governed by our contributing guidelines.

License

SPDX-License-Identifier: Apache-2.0 OR MIT