spdk

README

Go language bindings for the SPDK API

This is a Go interface for SPDK which is also a work in progress.

To clone the 18.07 SPDK branch:

git clone --single-branch --branch v18.07.x git@github.com:spdk/spdk.git

Current Status

• Initial support will be for NVMe driver utilities.

Using these bindings assumes SPDK shared lib is installed in ${SPDK_REPO}/install/lib/libspdk.so. In order to use some of the SPDK API, please also follow Hugepages and Device Binding.

How to build binding requirements

Setup environment:

export GOSPDK=${DAOS_ROOT}/src/control/lib/spdk
export SPDK_LIB=${DAOS_ROOT}/opt/spdk
export LD_LIBRARY_PATH=${SPDK_LIB}/build/lib:${SPDK_LIB}/include:${GOSPDK}/spdk:${LD_LIBRARY_PATH}
export CGO_CFLAGS="-I${SPDK_LIB}/include"
export CGO_LDFLAGS="-L${SPDK_LIB}/build/lib -lspdk"

Build NVMe libs:

cd ${GOSPDK}
gcc ${CGO_LDFLAGS} ${CGO_CFLAGS} -Werror -g -Wshadow -Wall -Wno-missing-braces -c -fpic -Iinclude src/*.c -lspdk
gcc ${CGO_LDFLAGS} ${CGO_CFLAGS} -shared -o libnvme_control.so *.o

Build go spdk bindings:

cd ${GOSPDK}
sudo CGO_LDFLAGS=${CGO_LDFLAGS} CGO_CFLAGS=${CGO_CFLAGS} go build -v -i

Run Go Unit Tests:

cd ${GOSPDK}
go test -v

cmocka Test Env:

export TESTLIBS="-lspdk -lcmocka -lnvme_control"
export GCCFLAGS="-g -fpic -Wall -Werror -Wshadow -Wno-missing-braces"
export TESTFLAGS="${CGO_LDFLAGS} ${CGO_CFLAGS} ${GCCFLAGS} ${TESTLIBS}"

Build cmocka Tests:

cd ${GOSPDK}/ctests
gcc ${TESTFLAGS} -I../include -L../. nvme_control_ut.c ../src/*.c -o nvme_control_ctests

Documentation

Overview

(C) Copyright 2020 Intel Corporation.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

GOVERNMENT LICENSE RIGHTS-OPEN SOURCE SOFTWARE The Government's rights to use, modify, reproduce, release, perform, display, or disclose this software are subject to the terms of the Apache License as provided in Contract No. 8F-30005. Any reproduction of computer software, computer software documentation, or portions thereof marked with this legend must also reproduce the markings.

Package spdk provides Go bindings for SPDK

Index

• Variables
• func FaultBindingFailed(rc int, errMsg string) *fault.Fault
• func Rc2err(label string, rc C.int) error
• type Controller
• type DeviceHealth
• type ENV
• type Env
  • func (e *Env) InitSPDKEnv(shmID int) (err error)
• type NVME
• type Namespace
• type Nvme
  • func (n *Nvme) CleanLockfiles(log logging.Logger, pciAddrs ...string) error
  • func (n *Nvme) Cleanup()
  • func (n *Nvme) Discover(log logging.Logger) ([]Controller, error)
  • func (n *Nvme) Format(log logging.Logger, ctrlrPciAddr string) (err error)
  • func (n *Nvme) Update(log logging.Logger, ctrlrPciAddr string, path string, slot int32) (ctrlrs []Controller, err error)

Variables

var (
	FaultUnknown = spdkFault(
		code.SpdkUnknown,
		"unknown SPDK bindings error",
		"",
	)
	FaultCtrlrNoHealth = spdkFault(
		code.SpdkCtrlrNoHealth,
		"NVMe controller details are missing health statistics",
		"",
	)
	FaultBindingRetNull = spdkFault(
		code.SpdkBindingRetNull,
		"SPDK binding unexpectedly returned NULL",
		"",
	)
)

Functions

func FaultBindingFailed

func FaultBindingFailed(rc int, errMsg string) *fault.Fault

func Rc2err

func Rc2err(label string, rc C.int) error

Rc2err returns an failure if rc != 0.

TODO: If err is already set then it is wrapped, otherwise it is ignored. e.g. func Rc2err(label string, rc C.int, err error) error {

Types

type Controller

type Controller struct {
	Model       string
	Serial      string
	PCIAddr     string
	FWRev       string
	SocketID    int32
	Namespaces  []*Namespace
	HealthStats *DeviceHealth
}

Controller struct mirrors C.struct_ctrlr_t and describes a NVMe controller.

TODO: populate implicitly using inner member: +inner C.struct_ctrlr_t

type DeviceHealth

type DeviceHealth struct {
	Temp            uint32
	TempWarnTime    uint32
	TempCritTime    uint32
	CtrlBusyTime    uint64
	PowerCycles     uint64
	PowerOnHours    uint64
	UnsafeShutdowns uint64
	MediaErrors     uint64
	ErrorLogEntries uint64
	TempWarn        bool
	AvailSpareWarn  bool
	ReliabilityWarn bool
	ReadOnlyWarn    bool
	VolatileWarn    bool
}

DeviceHealth struct mirrors C.struct_dev_health_t and describes the raw SPDK device health stats of a controller (NVMe SSD).

type ENV

type ENV interface {
	InitSPDKEnv(int) error
}

ENV is the interface that provides SPDK environment management.

type Env

type Env struct{}

Env is a simple ENV implementation.

func (*Env) InitSPDKEnv

func (e *Env) InitSPDKEnv(shmID int) (err error)

InitSPDKEnv initializes the SPDK environment.

SPDK relies on an abstraction around the local environment named env that handles memory allocation and PCI device operations. The library must be initialized first.

type NVME

type NVME interface {
	// Discover NVMe controllers and namespaces, and device health info
	Discover(logging.Logger) ([]Controller, error)
	// Format NVMe controller namespaces
	Format(logging.Logger, string) error
	// Cleanup NVMe object references
	Cleanup()
	// CleanLockfiles removes SPDK lockfiles for specific PCI addresses
	CleanLockfiles(logging.Logger, ...string)
}

NVME is the interface that provides SPDK NVMe functionality.

type Namespace

type Namespace struct {
	ID   uint32
	Size uint64
}

Namespace struct mirrors C.struct_ns_t and describes a NVMe Namespace tied to a controller.

TODO: populate implicitly using inner member: +inner C.struct_ns_t

type Nvme

type Nvme struct{}

Nvme is an NVME interface implementation.

func (*Nvme) CleanLockfiles

func (n *Nvme) CleanLockfiles(log logging.Logger, pciAddrs ...string) error

CleanLockfiles removes SPDK lockfiles after binding operations.

func (*Nvme) Cleanup

func (n *Nvme) Cleanup()

Cleanup unlinks and detaches any controllers or namespaces, as well as cleans up optional device health information.

func (*Nvme) Discover

func (n *Nvme) Discover(log logging.Logger) ([]Controller, error)

Discover NVMe devices accessible by SPDK on a given host.

Calls C.nvme_discover which returns pointers to single linked list of ctrlr_t structs. These are converted and returned as Controller slices containing any Namespace and DeviceHealth structs. Afterwards remove lockfile for each discovered device.

func (*Nvme) Format

func (n *Nvme) Format(log logging.Logger, ctrlrPciAddr string) (err error)

Format device at given pci address, destructive operation!

Attempt wipe of namespace #1 LBA-0 and falls back to full controller format if quick format failed. Afterwards remove lockfile for formatted device.

func (*Nvme) Update

func (n *Nvme) Update(log logging.Logger, ctrlrPciAddr string, path string, slot int32) (ctrlrs []Controller, err error)

Update calls C.nvme_fwupdate to update controller firmware image.

Retrieves image from path and updates given firmware slot/register then remove lockfile for updated device.