mccl

Documentation

Overview

Package mccl provides a collective communication library implementation.

Index

• func AllReduceRing(d *driver.Driver, comms []*Communicator, data []driver.Ptr, dataSize int, ...)
• func BroadcastRing(d *driver.Driver, comms []*Communicator, root int, data []driver.Ptr, ...)
• type Communicator
  • func CommInitAll(nDev int, driver *driver.Driver, ctx *driver.Context, gpuIDs []int) []*Communicator
  • func CommInitAllMultipleContexts(nDev int, driver *driver.Driver, ctxs []*driver.Context, gpuIDs []int) []*Communicator
  • func NewCommunicator(driver *driver.Driver, ctx *driver.Context, gpuID int, rank uint32, ...) *Communicator

Functions

func AllReduceRing

func AllReduceRing(
	d *driver.Driver,
	comms []*Communicator,
	data []driver.Ptr,
	dataSize int,
	bufs []driver.Ptr,
	sizePerBuf int,
)

AllReduceRing performs AllReduce average operation.

func BroadcastRing

func BroadcastRing(
	d *driver.Driver,
	comms []*Communicator,
	root int,
	data []driver.Ptr,
	dataSize int,
)

BroadcastRing broadcast data from the root to other GPUs.

Types

type Communicator

type Communicator struct {
	Driver    *driver.Driver
	Ctx       *driver.Context
	GPUID     int
	Rank      uint32
	GroupID   int
	GroupSize uint32
}

Communicator is a struct that helps gpu to communicate.

func CommInitAll

func CommInitAll(
	nDev int,
	driver *driver.Driver,
	ctx *driver.Context,
	gpuIDs []int,
) []*Communicator

CommInitAll initiates a list of communicators.

func CommInitAllMultipleContexts

func CommInitAllMultipleContexts(
	nDev int,
	driver *driver.Driver,
	ctxs []*driver.Context,
	gpuIDs []int,
) []*Communicator

CommInitAllMultipleContexts creates a list of communicators with one context per communicator.

func NewCommunicator

func NewCommunicator(
	driver *driver.Driver,
	ctx *driver.Context,
	gpuID int,
	rank uint32,
	groupID int,
	groupSize uint32,
) *Communicator

NewCommunicator creates a new communicator rank is the id of this gpu