README
NDN-DPDK: High-Speed Named Data Networking Forwarder
NDN-DPDK is a set of high-speed Named Data Networking (NDN) programs developed with the Data Plane Development Kit (DPDK). Included are a network forwarder and a traffic generator.
This software is developed at the Advanced Network Technologies Division of the National Institute of Standards and Technology. It is in pre-release stage and will continue to be updated.
Installation
Requirements
- Ubuntu 18.04 or Debian 10 on amd64 architecture
- Required APT packages:
build-essential clang-8 git libc6-dev-i386 libelf-dev libnuma-dev libssl-dev liburcu-dev pkg-config python3-distutils
- Optional APT packages:
clang-format-8 doxygen yamllint
(see "other build targets" for an explanation) - pip and
sudo pip install -U meson ninja
- Intel Multi-Buffer Crypto for IPsec Library v0.55 (optional)
- DPDK 20.11, configured with
meson -Ddebug=true -Doptimization=3 -Dtests=false --libdir=lib build
- SPDK 20.10, configured with
./configure --enable-debug --disable-tests --with-shared --with-dpdk=/usr/local --without-vhost --without-isal --without-fuse
- ubpf library, installed to
/usr/local
- Go 1.x
- Node.js 14.x
You can install the dependencies with ndndpdk-depends.sh.
NDN-DPDK requires hugepages to run. See huge-setup.sh for an example on how to setup hugepages.
Build steps
- Clone the repository.
- Execute
npm install
to download NPM dependencies. - Execute
make
to compile the project. - Execute
sudo make install
to install the programs to/usr/local
, andsudo make uninstall
to uninstall them.
Other build targets
make godeps
builds C objects and generates certain Go source files.make gopkg
builds all Go packages.make test
runs all unit tests. You can also executemk/gotest.sh <PKG>
to run the tests for a given package.make doxygen
builds C documentation (requires thedoxygen
dependency).- To view Go documentation, execute
godoc &
and access the website on port 6060. You may need to install godoc command:sudo GO111MODULE=off GOBIN=$(go env GOROOT)/bin $(which go) get -u golang.org/x/tools/cmd/godoc
make lint
fixes code style issues before committing (requires theclang-format-8
andyamllint
dependencies).
Compile-time settings
You can change compile-time settings by setting these environment variables:
NDNDPDK_MK_RELEASE=1
selects release mode that disables assertions and verbose logging in C code.NDNDPDK_MK_THREADSLEEP=1
insertsnanosleep(1ns)
to each thread. This reduces performance significantly, but is occasionally useful when running on a machine with fewer CPU cores.- C code other than strategy is compiled with
gcc
by default; you can override this by setting theCC
environment variable. - Strategy code is compiled with
clang-8
by default; you can override this by setting theBPFCC
environment variable.
You must run make clean
when switching compile-time settings.
Docker packaging
- Build the image:
docker build -t ndn-dpdk .
- Configure hugepages on the host machine:
echo 8 | sudo tee /sys/devices/system/node/node*/hugepages/hugepages-1048576kB/nr_hugepages && sudo mkdir -p /mnt/huge1G && sudo mount -t hugetlbfs nodev /mnt/huge1G -o pagesize=1G
- Launch a container in privileged mode:
docker run --rm -it --privileged --network host --mount type=bind,source=/mnt/huge1G,target=/mnt/huge1G ndn-dpdk
- Run NDN-DPDK service inside the container:
ndndpdk-svc
- Or run unit tests:
export PATH=$PATH:/usr/local/go/bin; cd /root/ndn-dpdk; make test
Note that DPDK is compiled with -march=native
flag, so that the Docker image only works on machines with the same CPU model.
Code Organization
- ndn: NDN library in pure Go.
- mk: build helper scripts.
- csrc: C source code.
- js: TypeScript source code.
- core: common shared code.
- dpdk: Go bindings for DPDK and SPDK.
- ndni: NDN packet representation for internal use.
- iface: network interfaces.
- container: data structures.
- strategy: forwarding strategy BPF programs.
- app: applications, including the forwarder dataplane.
- cmd: executables.
Directories
Path | Synopsis |
---|---|
app
|
|
fetch
Package fetch simulates bulk file transfer traffic patterns.
|
Package fetch simulates bulk file transfer traffic patterns. |
fwdp
Package fwdp implements the forwarder's data plane.
|
Package fwdp implements the forwarder's data plane. |
tg
Package tg is a traffic generator.
|
Package tg is a traffic generator. |
tgconsumer
Package tgconsumer implements a traffic generator consumer.
|
Package tgconsumer implements a traffic generator consumer. |
tgproducer
Package tgproducer implements a traffic generator producer.
|
Package tgproducer implements a traffic generator producer. |
tgtestenv
Package tgtestenv provides facility to test the traffic generator.
|
Package tgtestenv provides facility to test the traffic generator. |
cmd
|
|
ndndpdk-ctrl
Command ndndpdk-ctrl controls the NDN-DPDK service.
|
Command ndndpdk-ctrl controls the NDN-DPDK service. |
ndndpdk-godemo
Command ndndpdk-godemo demonstrates NDNgo library features.
|
Command ndndpdk-godemo demonstrates NDNgo library features. |
ndndpdk-hrlog2histogram
Command ndndpdk-hrlog2histogram extracts latency histograms from high resolution per-packet logs.
|
Command ndndpdk-hrlog2histogram extracts latency histograms from high resolution per-packet logs. |
ndndpdk-jrproxy
Command ndndpdk-jrproxy exposes NDN-DPDK GraphQL API as JSON-RPC 2.0 management API (2019).
|
Command ndndpdk-jrproxy exposes NDN-DPDK GraphQL API as JSON-RPC 2.0 management API (2019). |
ndndpdk-svc
Command ndndpdk-svc executes the NDN-DPDK service.
|
Command ndndpdk-svc executes the NDN-DPDK service. |
container
|
|
cs
Package cs implements the Content Store.
|
Package cs implements the Content Store. |
cs/cscnt
Package cscnt provides Content Store counters.
|
Package cscnt provides Content Store counters. |
diskstore
Package diskstore provides a disk-based Data packet store.
|
Package diskstore provides a disk-based Data packet store. |
fib
Package fib implements the Forwarding Information Base.
|
Package fib implements the Forwarding Information Base. |
fib/fibdef
Package fibdef declares common data structures for FIB.
|
Package fibdef declares common data structures for FIB. |
fib/fibreplica
Package fibreplica controls a FIB replica in C.Fib struct.
|
Package fibreplica controls a FIB replica in C.Fib struct. |
fib/fibtestenv
Package fibtestenv provides utilities for FIB unit tests.
|
Package fibtestenv provides utilities for FIB unit tests. |
fib/fibtree
Package fibtree organizes logical FIB entries in a name hierarchy.
|
Package fibtree organizes logical FIB entries in a name hierarchy. |
hrlog
Package hrlog collects high resolution per-packet logs.
|
Package hrlog collects high resolution per-packet logs. |
ndt
Package ndt implements the Name Dispatch Table.
|
Package ndt implements the Name Dispatch Table. |
pcct
Package pcct implements the PIT-CS Composite Table.
|
Package pcct implements the PIT-CS Composite Table. |
pit
Package pit implements the Pending Interest Table.
|
Package pit implements the Pending Interest Table. |
strategycode
Package strategycode manages forwarding strategy BPF programs.
|
Package strategycode manages forwarding strategy BPF programs. |
core
|
|
cptr
Package cptr handles C void* pointers.
|
Package cptr handles C void* pointers. |
dlopen
Package dlopen allows preloading dynamic libraries.
|
Package dlopen allows preloading dynamic libraries. |
events
Package events provides a simple event emitter.
|
Package events provides a simple event emitter. |
gqlclient
Package gqlclient provides a GraphQL client.
|
Package gqlclient provides a GraphQL client. |
gqlserver
Package gqlserver provides a GraphQL server.
|
Package gqlserver provides a GraphQL server. |
logger
Package logger is a thin wrapper of logrus library.
|
Package logger is a thin wrapper of logrus library. |
macaddr
Package macaddr validates and classifies MAC-48 addresses.
|
Package macaddr validates and classifies MAC-48 addresses. |
nnduration
Package nnduration provides JSON-friendly non-negative duration types.
|
Package nnduration provides JSON-friendly non-negative duration types. |
runningstat
Package runningstat implements Knuth and Welford's method for computing the standard deviation.
|
Package runningstat implements Knuth and Welford's method for computing the standard deviation. |
testenv
Package testenv provides general test utilities.
|
Package testenv provides general test utilities. |
urcu
Package urcu is a thin wrapper of Userspace RCU library.
|
Package urcu is a thin wrapper of Userspace RCU library. |
dpdk
|
|
bdev
Package bdev contains bindings of SPDK block device layer.
|
Package bdev contains bindings of SPDK block device layer. |
cryptodev
Package cryptodev contains bindings of DPDK crypto device.
|
Package cryptodev contains bindings of DPDK crypto device. |
eal
Package eal contains bindings of DPDK Environment Abstraction Layer.
|
Package eal contains bindings of DPDK Environment Abstraction Layer. |
ealconfig
Package ealconfig prepares EAL parameters.
|
Package ealconfig prepares EAL parameters. |
ealinit
Package ealinit initializes DPDK EAL and SPDK main thread.
|
Package ealinit initializes DPDK EAL and SPDK main thread. |
ealtestenv
Package ealtestenv initializes EAL for unit testing.
|
Package ealtestenv initializes EAL for unit testing. |
ealthread
Package ealthread provides a thread abstraction bound to an DPDK LCore.
|
Package ealthread provides a thread abstraction bound to an DPDK LCore. |
ethdev
Package ethdev contains bindings of DPDK Ethernet device.
|
Package ethdev contains bindings of DPDK Ethernet device. |
mempool
Package mempool contains bindings of DPDK memory pool.
|
Package mempool contains bindings of DPDK memory pool. |
pktmbuf
Package pktmbuf contains bindings of DPDK mbuf library.
|
Package pktmbuf contains bindings of DPDK mbuf library. |
pktmbuf/mbuftestenv
Package mbuftestenv contains helper functions to construct mbufs in test code.
|
Package mbuftestenv contains helper functions to construct mbufs in test code. |
ringbuffer
Package ringbuffer contains bindings of DPDK ring library.
|
Package ringbuffer contains bindings of DPDK ring library. |
spdkenv
Package spdkenv contains bindings of SPDK environment and threads.
|
Package spdkenv contains bindings of SPDK environment and threads. |
Package iface implements basics of the face system.
|
Package iface implements basics of the face system. |
ethface
Package ethface implements Ethernet faces using DPDK Ethernet devices.
|
Package ethface implements Ethernet faces using DPDK Ethernet devices. |
ifacetestenv
Package ifacetestenv provides a test fixture for a face type.
|
Package ifacetestenv provides a test fixture for a face type. |
intface
Package intface implements an internal face for internal applications.
|
Package intface implements an internal face for internal applications. |
socketface
Package socketface implements UDP/TCP socket faces using Go net.Conn type.
|
Package socketface implements UDP/TCP socket faces using Go net.Conn type. |
mk
|
|
enumgen
Command enumgen generates C headers from Go enum declarations.
|
Command enumgen generates C headers from Go enum declarations. |
version
Package version records NDN-DPDK version information.
|
Package version records NDN-DPDK version information. |
Package ndn implements Named Data Networking (NDN) packet semantics.
|
Package ndn implements Named Data Networking (NDN) packet semantics. |
an
Package an contains Assigned Numbers in Named Data Networking (NDN).
|
Package an contains Assigned Numbers in Named Data Networking (NDN). |
endpoint
Package endpoint implements basic consumer and producer functionality.
|
Package endpoint implements basic consumer and producer functionality. |
keychain
Package keychain implements signing and verification on NDN packets.
|
Package keychain implements signing and verification on NDN packets. |
keychain/eckey
Package eckey implements SigSha256WithEcdsa signature type.
|
Package eckey implements SigSha256WithEcdsa signature type. |
keychain/rsakey
Package rsakey implements SigSha256WithRsa signature type.
|
Package rsakey implements SigSha256WithRsa signature type. |
l3
Package l3 defines a network layer face abstraction.
|
Package l3 defines a network layer face abstraction. |
memiftransport
Package memiftransport implements a transport over a shared memory packet interface (memif).
|
Package memiftransport implements a transport over a shared memory packet interface (memif). |
mgmt
Package mgmt defines interface of forwarder management features.
|
Package mgmt defines interface of forwarder management features. |
mgmt/gqlmgmt
Package gqlmgmt provides access to NDN-DPDK GraphQL API.
|
Package gqlmgmt provides access to NDN-DPDK GraphQL API. |
ndntestenv
Package ndntestenv contains helper functions to validate NDN packets in test code.
|
Package ndntestenv contains helper functions to validate NDN packets in test code. |
ndntestvector
Package ndntestvector contains test vectors of NDN packets.
|
Package ndntestvector contains test vectors of NDN packets. |
packettransport
Package packettransport implements a transport based on gopacket library.
|
Package packettransport implements a transport based on gopacket library. |
packettransport/afpacket
Package afpacket implements a transport that communicates over AF_PACKET sockets.
|
Package afpacket implements a transport that communicates over AF_PACKET sockets. |
sockettransport
Package sockettransport implements a transport based on stream or datagram sockets.
|
Package sockettransport implements a transport based on stream or datagram sockets. |
tlv
Package tlv implements Type-Length-Value (TLV) encoding in Named Data Networking (NDN).
|
Package tlv implements Type-Length-Value (TLV) encoding in Named Data Networking (NDN). |
Package ndni implements NDN layer 2 and layer 3 packet representations for internal use.
|
Package ndni implements NDN layer 2 and layer 3 packet representations for internal use. |
ndnitestenv
Package ndnitestenv contains helper functions to construct NDN packets in test code.
|
Package ndnitestenv contains helper functions to construct NDN packets in test code. |