zmemif

README

zMemif

zMemif is a native golang based library for memif to interworking with dpdk. it can simply provide 20Mpps recv and 10Mpps xmit capability.

• The golang lib is forked from vpp gomemif example code, and modified the "Interface" to "Port" which make golang developers more clear about the definition. it also change the "master/slave" to "server/client" to align with existing dpdk library.
• we write a dpdk based light-weight forwarder to replace vpp on data path, provide simple virtio-user interface for kernel service and tcp based remote mgmt usage, all udp packet will redirect to memif which can be directly serve by native golang.

Motivation

Many applications like network telemetry analysis, video processing and other udp based application may require high performance I/O support in native-golang enviroment. DPDK may useful to accelerate packet proccessing in these case, intel open souce nff-go project few years ago, however the cgo and flow based abstraction may useful for NFV developer, but many application developer require a simple native go based solution.

This project wil be used for ruta-io and netDAM. We may integrate zMemif with go-quic in the future to provide quic based segment routing(quic-sr) or Segment Routing over UDP(SRoU).

Introduction

zmemif is a memory interface implementation for accelerate golang. You could build your application pure in golang mode, and use DPDK as memif server.

Usage

1. native golang server <----memif----> native golang client

example could be found /example/simple_echo and /example/bw_test

2. DPDK based server <----memif----> native golang client

dpdk forwarder server could be found /dpdk folder.

main.c defined the RX/TX core and queue per core, will add by cli args in the future

    unsigned int first_lcore = 25;
    char *ipaddr = "11.11.11.11";
    char *netmask = "255.255.255.0";
    int num_memif = 16;
    int num_rx_core = 4;
    int num_tx_core = 2;
    int server_mode = 1;

how to compile dpdk forwarder

1. install and setup dpdk env could be found in the following url

https://github.com/zartbot/learn_dpdk/tree/main/a1_setup_mlx5_sriov_env

1. compile and run

cd dpdk
make
sudo ./build/run

1. about unix sock file by default, it will create /tmp/memif.sock file for unix socket to communicate with golang client, this unix socket is used to allocate memory region and ring buffer. DPDK forwarder need to run in root mode and auto create this file, so this file must provide access priviledge for golang client.

sudo chmod 777 /tmp/memif.sock 

1. run native golang client

cd /example/dpdk_co_worker

run send/recv examples..

Roadmap

1. reliable transmit on datapath
2. simple udp level send/recv warpper for application migration.
3. In the future , we will use the netDAM DPU hardware to replace the dpdk based forwarder and provide full native userspace memif access.

Reference

• memif

• virtio-user dpdk usage

• netDAM

• ruta-io

Documentation

Index

• Constants
• func RoleToString(isServer bool) string
• type ConnectedFunc
• type DisconnectedFunc
• type MemoryConfig
• type MsgAddRegion
• type MsgAddRing
• type MsgConnect
• type MsgConnected
• type MsgDisconnect
• type MsgHello
• type MsgInit
• type Port
  • func NewPort(socket *Socket, cfg *PortCfg, extendData interface{}) (*Port, error)
  • func (p *Port) Delete() (err error)
  • func (p *Port) Disconnect() (err error)
  • func (p *Port) GetExtendData() interface{}
  • func (p *Port) GetId() uint32
  • func (p *Port) GetMemoryConfig() MemoryConfig
  • func (p *Port) GetName() string
  • func (p *Port) GetPeerName() string
  • func (p *Port) GetRemoteName() string
  • func (p *Port) GetRxQueue(qid int) (*Queue, error)
  • func (p *Port) GetSocket() *Socket
  • func (p *Port) GetTxQueue(qid int) (*Queue, error)
  • func (p *Port) IsConnected() bool
  • func (p *Port) IsConnecting() bool
  • func (p *Port) IsServer() bool
  • func (p *Port) RequestConnection() error
  • func (p *Port) String() string
• type PortCfg
• type Queue
  • func (q *Queue) GetEventFd() (int, error)
  • func (q *Queue) ReadPacket(pkt []byte) (int, error)
  • func (q *Queue) WritePacket(pkt []byte) int
• type Socket
  • func NewSocket(appName string, filename string) (socket *Socket, err error)
  • func (socket *Socket) AddListener(fd int) (err error)
  • func (socket *Socket) Delete() (err error)
  • func (socket *Socket) GetFilename() string
  • func (socket *Socket) NewPort(cfg *PortCfg) (*Port, error)
  • func (socket *Socket) StartPolling()
  • func (socket *Socket) StopPolling() error

Constants

const (
	DefaultSocketFilename   = "/tmp/memif.sock"
	DefaultNumQueuePairs    = 1
	DefaultLog2RingSize     = 10
	DefaultPacketBufferSize = 2048
)

const Version = ((VersionMajor << 8) | VersionMinor)

Version is memif protocols version as uint16 (M-Major m-minor: MMMMMMMMmmmmmmmm)

const VersionMajor = 2

VersionMajor is memif protocols major version

const VersionMinor = 0

VersionMinor is memif protocols minor version

Functions

func RoleToString

func RoleToString(isServer bool) string

RoleToString returns 'Server' if isServer os true, else returns 'Client'

Types

type ConnectedFunc

type ConnectedFunc func(p *Port) error

ConnectedFunc is a callback called when an interface is connected

type DisconnectedFunc

type DisconnectedFunc func(p *Port) error

DisconnectedFunc is a callback called when an interface is disconnected

type MemoryConfig

type MemoryConfig struct {
	NumQueuePairs    uint16 // number of queue pairs
	Log2RingSize     uint8  // ring size as log2
	PacketBufferSize uint32 // size of single packet buffer
}

MemoryConfig represents shared memory configuration

type MsgAddRegion

type MsgAddRegion struct {
	Index uint16
	Size  uint64
}

type MsgAddRing

type MsgAddRing struct {
	Flags          uint16
	Index          uint16
	Region         uint16
	Offset         uint32
	RingSizeLog2   uint8
	PrivateHdrSize uint16
}

type MsgConnect

type MsgConnect struct {
	// interface name
	Name [32]byte
}

type MsgConnected

type MsgConnected struct {
	// interface name
	Name [32]byte
}

type MsgDisconnect

type MsgDisconnect struct {
	Code   uint32
	String [96]byte
}

type MsgHello

type MsgHello struct {
	// app name
	Name            [32]byte
	VersionMin      uint16
	VersionMax      uint16
	MaxRegion       uint16
	MaxRingM2S      uint16
	MaxRingS2M      uint16
	MaxLog2RingSize uint8
}

type MsgInit

type MsgInit struct {
	Version uint16
	Id      uint32
	Mode    portMode
	Secret  [24]byte
	// app name
	Name [32]byte
}

type Port

type Port struct {
	ExtendData interface{}

	ErrChan  chan error
	QuitChan chan struct{}
	Wg       sync.WaitGroup
	// contains filtered or unexported fields
}

Port represents memif network interface

func NewPort

func NewPort(socket *Socket, cfg *PortCfg, extendData interface{}) (*Port, error)

func (*Port) Delete

func (p *Port) Delete() (err error)

Delete deletes the port

func (*Port) Disconnect

func (p *Port) Disconnect() (err error)

Disconnect disconnects the port

func (*Port) GetExtendData

func (p *Port) GetExtendData() interface{}

GetExtendData returns interfaces extend data

func (*Port) GetId

func (p *Port) GetId() uint32

GetId returns interfaces id

func (*Port) GetMemoryConfig

func (p *Port) GetMemoryConfig() MemoryConfig

GetMemoryConfig returns interfaces active memory config. If Port is not connected the config is invalid.

func (*Port) GetName

func (p *Port) GetName() string

GetName returens interfaces name

func (*Port) GetPeerName

func (p *Port) GetPeerName() string

GetPeerName returns peer interfaces name

func (*Port) GetRemoteName

func (p *Port) GetRemoteName() string

GetRemoteName returns the name of the application on which the peer interface exists

func (*Port) GetRxQueue

func (p *Port) GetRxQueue(qid int) (*Queue, error)

GetRxQueue returns an rx queue specified by queue index

func (*Port) GetSocket

func (p *Port) GetSocket() *Socket

GetSocket returns the socket the interface belongs to

func (*Port) GetTxQueue

func (p *Port) GetTxQueue(qid int) (*Queue, error)

GetRxQueue returns a tx queue specified by queue index

func (*Port) IsConnected

func (p *Port) IsConnected() bool

IsConnected returns true if the port is connected

func (*Port) IsConnecting

func (p *Port) IsConnecting() bool

IsConnecting returns true if the port is connecting

func (*Port) IsServer

func (p *Port) IsServer() bool

IsServer returns true if the interfaces role is server, else returns false

func (*Port) RequestConnection

func (p *Port) RequestConnection() error

RequestConnection is used by client port to connect to a socket and create a control channel

func (*Port) String

func (p *Port) String() string

type PortCfg

type PortCfg struct {
	Id               uint32 // Port identifier unique across socket. Used to identify peer Port when connecting
	IsServer         bool   // Port role server/client
	Name             string
	Secret           [24]byte // optional parameter, secrets of the Ports must match if they are to connect
	MemoryConfig     MemoryConfig
	ConnectedFunc    ConnectedFunc    // callback called when Port changes status to connected
	DisconnectedFunc DisconnectedFunc // callback called when Port changes status to disconnected
	ExtendData       interface{}      // ExtendData used by client program
}

PortCfg represent port configuration

type Queue

type Queue struct {
	// contains filtered or unexported fields
}

Queue represents rx or tx queue

func (*Queue) GetEventFd

func (q *Queue) GetEventFd() (int, error)

GetEventFd returns queues interrupt event fd

func (*Queue) ReadPacket

func (q *Queue) ReadPacket(pkt []byte) (int, error)

ReadPacket reads one packet form the shared memory and returns the number of bytes read

func (*Queue) WritePacket

func (q *Queue) WritePacket(pkt []byte) int

WritePacket writes one packet to the shared memory and returns the number of bytes written

type Socket

type Socket struct {
	ErrChan chan error
	// contains filtered or unexported fields
}

Socket represents a UNIX domain socket used for communication between memif peers

func NewSocket

func NewSocket(appName string, filename string) (socket *Socket, err error)

NewSocket returns a new Socket

func (*Socket) AddListener

func (socket *Socket) AddListener(fd int) (err error)

AddListener adds a lisntener to the socket. The fd must describe a UNIX domain socket already bound to a UNIX domain filename and marked as listener

func (*Socket) Delete

func (socket *Socket) Delete() (err error)

Delete deletes the socket

func (*Socket) GetFilename

func (socket *Socket) GetFilename() string

GetFilename returns sockets filename

func (*Socket) NewPort

func (socket *Socket) NewPort(cfg *PortCfg) (*Port, error)

NewPort returns a new memif network port. When creating an port it's id must be unique across socket with the exception of loopback interface in which case the id is the same but role differs

func (*Socket) StartPolling

func (socket *Socket) StartPolling()

StartPolling starts polling and handling events on the socket, enabling communication between memif peers

func (*Socket) StopPolling

func (socket *Socket) StopPolling() error

StopPolling stops polling events on the socket