nopfs

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 29, 2023 License: Apache-2.0 Imports: 19 Imported by: 0

README

NOpfs!

NOPfs helps IPFS to say No!

NOPfs is an implementation of IPIP-383 which add supports for content blocking to the go-ipfs stack and particularly to Kubo.

NOPfs is quite alpha because:

The list of blocked content is managed via denylist files which have the following syntax:

version: 1
name: IPFSorp blocking list
description: A collection of bad things we have found in the universe
author: abuse-ipfscorp@example.com
hints:
  gateway_status: 410
  double_hash_fn: sha256
  double_hash_enc: hex
---
# Blocking by CID - blocks wrapped multihash.
# Does not block subpaths.
/ipfs/bafybeihvvulpp4evxj7x7armbqcyg6uezzuig6jp3lktpbovlqfkuqeuoq

# Block all subpaths
/ipfs/QmdWFA9FL52hx3j9EJZPQP1ZUH8Ygi5tLCX2cRDs6knSf8/*

# Block some subpaths (equivalent rules)
/ipfs/Qmah2YDTfrox4watLCr3YgKyBwvjq8FJZEFdWY6WtJ3Xt2/test*
/ipfs/QmTuvSQbEDR3sarFAN9kAeXBpiBCyYYNxdxciazBba11eC/test/*

# Block some subpaths with exceptions
/ipfs/QmUboz9UsQBDeS6Tug1U8jgoFkgYxyYood9NDyVURAY9pK/blocked*
+/ipfs/QmUboz9UsQBDeS6Tug1U8jgoFkgYxyYood9NDyVURAY9pK/blockednot
+/ipfs/QmUboz9UsQBDeS6Tug1U8jgoFkgYxyYood9NDyVURAY9pK/blocked/not
+/ipfs/QmUboz9UsQBDeS6Tug1U8jgoFkgYxyYood9NDyVURAY9pK/blocked/exceptions*

# Block IPNS domain name
/ipns/domain.example

# Block IPNS domain name and path
/ipns/domain2.example/path

# Block IPNS key - blocks wrapped multihash.
/ipns/k51qzi5uqu5dhmzyv3zac033i7rl9hkgczxyl81lwoukda2htteop7d3x0y1mf

# Block all mime types with exceptions
/mime/image/*
+/mime/image/gif

# Legacy CID double-hash block
# sha256(bafybeiefwqslmf6zyyrxodaxx4vwqircuxpza5ri45ws3y5a62ypxti42e/)
# blocks only this CID
//d9d295bde21f422d471a90f2a37ec53049fdf3e5fa3ee2e8f20e10003da429e7

# Legacy Path double-hash block
# Blocks bafybeiefwqslmf6zyyrxodaxx4vwqircuxpza5ri45ws3y5a62ypxti42e/path
# but not any other paths.
//3f8b9febd851873b3774b937cce126910699ceac56e72e64b866f8e258d09572

# Double hash CID block
# base58btc-sha256-multihash(QmVTF1yEejXd9iMgoRTFDxBv7HAz9kuZcQNBzHrceuK9HR)
# Blocks bafybeidjwik6im54nrpfg7osdvmx7zojl5oaxqel5cmsz46iuelwf5acja
# and QmVTF1yEejXd9iMgoRTFDxBv7HAz9kuZcQNBzHrceuK9HR etc. by multihash
//QmX9dhRcQcKUw3Ws8485T5a9dtjrSCQaUAHnG4iK9i4ceM

# Double hash Path block using blake3 hashing
# base58btc-blake3-multihash(gW7Nhu4HrfDtphEivm3Z9NNE7gpdh5Tga8g6JNZc1S8E47/path)
# Blocks /ipfs/bafyb4ieqht3b2rssdmc7sjv2cy2gfdilxkfh7623nvndziyqnawkmo266a/path
# /ipfs/bafyb4ieqht3b2rssdmc7sjv2cy2gfdilxkfh7623nvndziyqnawkmo266a/path
# /ipfs/f01701e20903cf61d46521b05f926ba1634628d0bba8a7ffb5b6d5a3ca310682ca63b5ef0/path etc...
# But not /path2
//QmbK7LDv5NNBvYQzNfm2eED17SNLt1yNMapcUhSuNLgkqz

Status

  • Support for blocking CIDs
  • Support for blocking IPFS Paths
  • Support for paths with wildcards (prefix paths)
  • Support for blocking legacy badbits anchors
  • Support for blocking double-hashed CIDs, IPFS paths, IPNS paths.
  • Support for blocking prefix and non-prefix sub-path
  • Support for denylist headers
  • Support for denylist rule hints
  • Support for allow rules (undo or create exceptions to earlier rules)
  • Live processing of appended rules to denylists
  • Content-blocking-enabled IPFS BlockService implementation
  • Content-blocking-enabled IPFS NameSystem implementation
  • Content-blocking-enabled IPFS Path resolver implementation
  • Mime-type blocking
  • Kubo plugin
  • Automatic, comprehensive testing of all rule types and edge cases
  • Work with a stable release of Kubo
  • Prebuilt plugin binaries

Using with Kubo

The plugin works with Kubo@9300769122d9151ea4e0861bce5dca9c1e411e96, with the following caveats:

  • go.mod has a replace directive. It should point to your local kubo git repository.
  • You should build the ipfs binary manually from the local kubo repo: cd cmd/ipfs; CGO_ENABLED=1 go build. Do not use make build.
  • Running make plugin-install (here on the nopfs repo) will build and install the nopfs.so plugin into ~/.ipfs/plugins.

From that point, starting Kubo should load the plugin and automatically work with denylists (files with extension .deny) found in /etc/ipfs/denylists and $XDG_CONFIG_HOME/ipfs/denylists (usually ~/.config/ipfs/denylists).

Documentation

Overview

Package nopfs implements content blocking for the IPFS stack.

nopfs provides an implementation of the compact denylist format (IPIP-383), with methods to check whether IPFS paths and CIDs are blocked.

In order to seamlessly be inserted into the IPFS stack, content-blocking wrappers for several components are provided (BlockService, NameSystem, Resolver...). A Kubo plugin (see kubo/) can be used to give Kubo content-blocking superpowers.

Index

Constants

This section is empty.

Variables

View Source 
var ErrHeaderNotFound = errors.New("header not found")

ErrHeaderNotFound is returned when no header can be Decoded.

Functions

func GetDenylistFiles 

func GetDenylistFiles() ([]string, error)

GetDenylistFiles returns a list of ".deny" files found in $XDG_CONFIG_HOME/ipfs/denylists and /etc/ipfs/denylists. The files are sortered by their names in their respective folders.

Types

type BlockedPath 

type BlockedPath struct {
	Path   string
	Prefix bool
}

BlockedPath represents the path part of a blocking rule.

func NewBlockedPath 

func NewBlockedPath(rawPath string) (BlockedPath, error)

NewBlockedPath takes a raw path, unscapes and sanitizes it, detecting and handling wildcards. It may also represent an "allowed" path on "allowed" rules.

func (BlockedPath) Matches 

func (bpath BlockedPath) Matches(path string) bool

Matches returns whether the given path matched the blocked (or allowed) path.

type Blocker 

type Blocker struct {
	Denylists map[string]*Denylist
}

A Blocker binds together multiple Denylists and can decide whether a path or a CID is blocked.

func NewBlocker 

func NewBlocker(files []string) (*Blocker, error)

NewBlocker creates a Blocker using the given denylist file paths. For default denylist locations, you can use GetDenylistFiles(). TODO: Options.

func (*Blocker) Close 

func (blocker *Blocker) Close() error

Close stops all denylists from being processed and watched for updates.

func (*Blocker) IsCidBlocked 

func (blocker *Blocker) IsCidBlocked(c cid.Cid) StatusResponse

IsCidBlocked returns blocking status for a CID. A CID is blocked when a Denylist reports it as blocked. A CID is not blocked when no denylist reports it as blocked or it is explicitally allowed. Lookup stops as soon as a defined "blocked" or "allowed" status is found.

Lookup for "allowed" or "blocked" status happens in order of the denylist, thus the denylist position during Blocker creation affects which one has preference.

Note that StatusResponse.Path will be unset. See Denylist.IsCidBlocked() for more info.

func (*Blocker) IsPathBlocked 

func (blocker *Blocker) IsPathBlocked(p path.Path) StatusResponse

IsPathBlocked returns blocking status for an IPFS Path. A Path is blocked when a Denylist reports it as blocked. A Path is not blocked when no denylist reports it as blocked or it is explicitally allowed. Lookup stops as soon as a defined "blocked" or "allowed" status is found.

Lookup for "allowed" or "blocked" status happens in order of the denylist, thus the denylist position during Blocker creation affects which one has preference.

Note that StatusResponse.Cid will be unset. See Denylist.IsPathBlocked() for more info.

type BlocksDB 

type BlocksDB struct {
	// contains filtered or unexported fields
}

BlocksDB is a key-value store of Entries. Keying may vary depending on whether we are indexing IPNS names, CIDs etc.

func (*BlocksDB) Load 

func (b *BlocksDB) Load(key string) (Entries, bool)

Load returns the Entries for a key.

func (*BlocksDB) Store 

func (b *BlocksDB) Store(key string, entry Entry)

Store stores a new entry with the given key. If there are existing Entries, the new Entry will be appended to them.

type Denylist 

type Denylist struct {
	Header   DenylistHeader
	Filename string

	Entries Entries

	IPFSBlocksDB       *BlocksDB
	IPNSBlocksDB       *BlocksDB
	DoubleHashBlocksDB map[uint64]*BlocksDB // codec -> blocks using that codec
	PathBlocksDB       *BlocksDB
	PathPrefixBlocks   Entries
	// contains filtered or unexported fields
}

A Denylist represents a denylist file and its rules. It can parse and follow a denylist file, and can answer questions about blocked or allowed items in this denylist.

func NewDenylist 

func NewDenylist(filepath string, follow bool) (*Denylist, error)

NewDenylist opens a denylist file and processes it (parses all its entries).

If follow is false, the file handle is closed.

If follow is true, the denylist file will be followed upon return. Any appended rules will be processed live-updated in the denylist. Denylist.Close() should be used when the Denylist or the following is no longer needed.

func NewDenylistReader 

func NewDenylistReader(r io.ReadSeekCloser) (*Denylist, error)

NewDenylistReader processes a denylist from the given reader (parses all its entries).

func (*Denylist) Close 

func (dl *Denylist) Close() error

Close closes the Denylist file handle and stops watching write events on it.

func (*Denylist) IsCidBlocked 

func (dl *Denylist) IsCidBlocked(c cid.Cid) StatusResponse

IsCidBlocked provides Blocking Status for a given CID. This is done by extracting the multihash and checking if it is blocked by any rule.

func (*Denylist) IsIPFSPathBlocked 

func (dl *Denylist) IsIPFSPathBlocked(cidStr, subpath string) StatusResponse

IsIPFSPathBlocked returns Blocking Status for a given IPFS CID and its subpath. The cidStr is NOT an "/ipns/cid" path, but just the cid.

func (*Denylist) IsIPLDPathBlocked 

func (dl *Denylist) IsIPLDPathBlocked(cidStr, subpath string) StatusResponse

IsIPLDPathBlocked returns Blocking Status for a given IPLD CID and its subpath. The cidStr is NOT an "/ipld/cid" path, but just the cid.

func (*Denylist) IsIPNSPathBlocked 

func (dl *Denylist) IsIPNSPathBlocked(name, subpath string) StatusResponse

IsIPNSPathBlocked returns Blocking Status for a given IPNS name and its subpath. The name is NOT an "/ipns/name" path, but just the name.

func (*Denylist) IsPathBlocked 

func (dl *Denylist) IsPathBlocked(p path.Path) StatusResponse

IsPathBlocked provides Blocking Status for a given path. This is done by interpreting the full path and checking for blocked Path, IPFS, IPNS or double-hashed items matching it.

Matching is more efficient if:

  • Paths in the form of /ipfs/Qm/... (sha2-256-multihash) are used rather than CIDv1.

  • A single double-hashing pattern is used.

  • A small number of path-only match rules using prefixes are used.

func (*Denylist) IsSubpathBlocked 

func (dl *Denylist) IsSubpathBlocked(subpath string) StatusResponse

IsSubpathBlocked returns Blocking Status for the given subpath.

type DenylistHeader 

type DenylistHeader struct {
	Version     int
	Name        string
	Description string
	Author      string
	Hints       map[string]string
	// contains filtered or unexported fields
}

DenylistHeader represents the header of a Denylist file.

func (*DenylistHeader) Decode 

func (h *DenylistHeader) Decode(r io.Reader) error

Decode decodes a DenlistHeader from a reader. Per the specification, the maximum size of a header is 1KiB. If no header is found, ErrHeaderNotFound is returned.

func (DenylistHeader) String 

func (h DenylistHeader) String() string

String provides a short string summary of the Header.

type Entries 

type Entries []Entry

Entries is a slice of Entry.

func (Entries) CheckPathStatus 

func (entries Entries) CheckPathStatus(p string) (Status, Entry)

CheckPathStatus returns whether the given path has a match in one of the Entries.

type Entry 

type Entry struct {
	Line      uint64
	AllowRule bool
	Hints     map[string]string
	RawValue  string
	Multihash multihash.Multihash // set for ipfs-paths mostly.
	Path      BlockedPath
}

Entry represents a rule (or a line) in a denylist file.

func (Entry) String 

func (e Entry) String() string

String provides a single-line representation of the Entry.

type Status 

type Status int

Status represent represents whether an item is blocked, allowed or simply not found in a Denylist. 

const (
	StatusNotFound Status = iota
	StatusBlocked
	StatusAllowed
	StatusErrored
)

Status values

func (Status) String 

func (st Status) String() string

type StatusError 

type StatusError struct {
	Response StatusResponse
}

StatusError implements the error interface and can be used to provide information about a blocked-status in the form of an error.

func (*StatusError) Error 

func (err *StatusError) Error() string

type StatusResponse 

type StatusResponse struct {
	Cid      cid.Cid
	Path     path.Path
	Status   Status
	Filename string
	Entry    Entry
	Error    error
}

StatusResponse provides full information for a content-block lookup, including the Filename and the Entry, when an associated rule is found.

func (StatusResponse) ToError 

func (r StatusResponse) ToError() error

ToError returns nil if the Status of the StatusResponse is Allowed or Not Found. When the status is Blocked or Errored, it returns a StatusError.

Source Files

View all Source files

Directories

Path Synopsis
cmd
nopfs
ipfs module
Package tester provides an implementation-agnostic way to test a Blocker.
 Package tester provides an implementation-agnostic way to test a Blocker.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.