voxels

package
v0.0.0-...-e545f54 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 6, 2014 License: BSD-3-Clause Imports: 16 Imported by: 0

Documentation

Overview

Package voxels implements DVID support for data using voxels as elements.

Index

Constants

View Source 
const (
	Version = "0.8"
	RepoUrl = "github.com/janelia-flyem/dvid/datatype/voxels"

	// Don't allow requests for more than this many voxels, which is larger than
	// 1000 x 1000 x 1000 volume, or 30000 x 30000 image.
	MaxVoxelsRequest = dvid.Giga
)
View Source 
const HelpMessage = `` /* 7699-byte string literal not displayed */
View Source 
const KVWriteSize = 500

KVWriteSize is the # of key/value pairs we will write as one atomic batch write.

Variables

View Source 
var (
	// DefaultBlockSize specifies the default size for each block of this data type.
	DefaultBlockSize int32 = 32

	DefaultRes float32 = 8

	DefaultUnits = "nanometers"
)

Functions

func ComputeTransform 

func ComputeTransform(v ExtHandler, block *Block, blockSize dvid.Point) (blockBeg, dataBeg, dataEnd dvid.Point, err error)

func GetImage 

func GetImage(uuid dvid.UUID, i IntHandler, e ExtHandler) (*dvid.Image, error)

GetImage retrieves a 2d image from a version node given a geometry of voxels.

func GetVolume 

func GetVolume(uuid dvid.UUID, i IntHandler, e ExtHandler) ([]byte, error)

GetVolume retrieves a n-d volume from a version node given a geometry of voxels.

func GetVoxels 

func GetVoxels(uuid dvid.UUID, i IntHandler, e ExtHandler) error

GetVoxels copies voxels from an IntHandler for a version to an ExtHandler, e.g., a requested subvolume or 2d image.

func LoadImages 

func LoadImages(i IntHandler, uuid dvid.UUID, offset dvid.Point, filenames []string) error

LoadImages bulk loads images using different techniques if it is a multidimensional file like HDF5 or a sequence of PNG/JPG/TIF images.

func PutVoxels 

func PutVoxels(uuid dvid.UUID, i IntHandler, e ExtHandler) error

PutVoxels copies voxels from an ExtHander (e.g., subvolume or 2d image) into an IntHandler for a version. Since chunk sizes can be larger than the PUT data, this also requires integrating the PUT data into current chunks before writing the result. There are two passes: 

Pass one: Retrieve all available key/values within the PUT space.
Pass two: Merge PUT data into those key/values and store them.

func ReadFromBlock 

func ReadFromBlock(v ExtHandler, block *Block, blockSize dvid.Point) error

func WriteToBlock 

func WriteToBlock(v ExtHandler, block *Block, blockSize dvid.Point) error

Types

type Axis 

type Axis struct {
	Label      string
	Resolution float32
	Units      string
	Size       int32
	Offset     int32
}

type Block 

type Block storage.KeyValue

Block is the basic key/value for the voxel type. The value is a slice of bytes corresponding to data within a block.

type Blocks 

type Blocks []Block

Blocks is a slice of Block.

type Data 

type Data struct {
	datastore.Data
	Properties
}

Data embeds the datastore's Data and extends it with voxel-specific properties.

func (*Data) BlankImage 

func (d *Data) BlankImage(dstW, dstH int32) (*dvid.Image, error)

BlankImage initializes a blank image of appropriate size and depth for the current data values. Returns an error if the geometry is not 2d.

func (*Data) BlockSize 

func (d *Data) BlockSize() dvid.Point

func (*Data) DataID 

func (d *Data) DataID() datastore.DataID

func (*Data) DoHTTP 

func (d *Data) DoHTTP(uuid dvid.UUID, w http.ResponseWriter, r *http.Request) error

DoHTTP handles all incoming HTTP requests for this data.

func (*Data) DoRPC 

func (d *Data) DoRPC(request datastore.Request, reply *datastore.Response) error

DoRPC acts as a switchboard for RPC commands.

func (*Data) Extents 

func (d *Data) Extents() *Extents

func (*Data) HandleIsotropy2D 

func (d *Data) HandleIsotropy2D(geom dvid.Geometry, isotropic bool) (dvid.Geometry, error)

Returns the image size necessary to compute an isotropic slice of the given dimensions. If isotropic is false, simply returns the original slice geometry. If isotropic is true, uses the higher resolution dimension.

func (*Data) JSONString 

func (d *Data) JSONString() (jsonStr string, err error)

JSONString returns the JSON for this Data's configuration

func (*Data) ModifyConfig 

func (d *Data) ModifyConfig(config dvid.Config) error

func (*Data) NewExtHandler 

func (d *Data) NewExtHandler(geom dvid.Geometry, img interface{}) (ExtHandler, error)

NewExtHandler returns an ExtHandler given some geometry and optional image data. If img is passed in, the function will initialize the ExtHandler with data from the image. Otherwise, it will allocate a zero buffer of appropriate size.

func (*Data) ProcessChunk 

func (d *Data) ProcessChunk(chunk *storage.Chunk)

ProcessChunk processes a chunk of data as part of a mapped operation. The data may be thinner, wider, and longer than the chunk, depending on the data shape (XY, XZ, etc). Only some multiple of the # of CPU cores can be used for chunk handling before it waits for chunk processing to abate via the buffered server.HandlerToken channel.

func (*Data) PutLocal 

func (d *Data) PutLocal(request datastore.Request, reply *datastore.Response) error

PutLocal adds image data to a version node, altering underlying blocks if the image intersects the block.

The image filename glob MUST BE absolute file paths that are visible to the server. This function is meant for mass ingestion of large data files, and it is inappropriate to read gigabytes of data just to send it over the network to a local DVID.

func (*Data) String 

func (d *Data) String() string

func (*Data) Values 

func (d *Data) Values() dvid.DataValues

type Datatype 

type Datatype struct {
	datastore.Datatype
	// contains filtered or unexported fields
}

Datatype embeds the datastore's Datatype to create a unique type with voxel functions. Refinements of general voxel types can be implemented by embedding this type, choosing appropriate # of values and bytes/value, overriding functions as needed, and calling datastore.RegisterDatatype(). Note that these fields are invariant for all instances of this type. Fields that can change depending on the type of data (e.g., resolution) should be in the Data type.

func NewDatatype 

func NewDatatype(values dvid.DataValues, interpolable bool) (dtype *Datatype)

NewDatatype returns a pointer to a new voxels Datatype with default values set.

func (*Datatype) Help 

func (dtype *Datatype) Help() string

func (*Datatype) NewData 

func (dtype *Datatype) NewData(id *datastore.DataID, config dvid.Config) (*Data, error)

NewData returns a pointer to a new Voxels with default values.

func (*Datatype) NewDataService 

func (dtype *Datatype) NewDataService(id *datastore.DataID, config dvid.Config) (datastore.DataService, error)

NewData returns a pointer to a new Voxels with default values.

type ExtHandler 

type ExtHandler interface {
	VoxelHandler

	Index(p dvid.ChunkPoint) dvid.Index

	IndexIterator(chunkSize dvid.Point) (dvid.IndexIterator, error)

	// DownRes reduces the image data by the integer scaling for each dimension.
	DownRes(downmag dvid.Point) error

	// Returns a 2d image suitable for external DVID use
	GetImage2d() (*dvid.Image, error)
}

ExtHandler provides the shape, location (indexing), and data of a set of voxels connected with external usage. It is the type used for I/O from DVID to clients, e.g., 2d images, 3d subvolumes, etc. These user-facing data must be converted to and from internal DVID representations using key/value pairs where the value is a block of data, and the key contains some spatial indexing.

We can read/write different external formats through the following steps:

  1. Create a data type package (e.g., datatype/labels64) and define a ExtHandler type where the data layout (i.e., the values in a voxel) is identical to the targeted DVID IntHandler.
  2. Do I/O for external format (e.g., Raveler's superpixel PNG images with implicit Z) and convert external data to the ExtHandler instance.
  3. Pass ExtHandler to voxels package-level functions.

type Extents 

type Extents struct {
	MinPoint dvid.Point
	MaxPoint dvid.Point

	MinIndex dvid.ChunkIndexer
	MaxIndex dvid.ChunkIndexer
	// contains filtered or unexported fields
}

Extents holds the extents of a volume in both absolute voxel coordinates and lexicographically sorted chunk indices.

func (*Extents) AdjustIndices 

func (ext *Extents) AdjustIndices(indexBeg, indexEnd dvid.ChunkIndexer) bool

AdjustIndices modifies extents based on new block indices in concurrency-safe manner.

func (*Extents) AdjustPoints 

func (ext *Extents) AdjustPoints(pointBeg, pointEnd dvid.Point) bool

AdjustPoints modifies extents based on new voxel coordinates in concurrency-safe manner.

type IntHandler 

type IntHandler interface {
	NewExtHandler(dvid.Geometry, interface{}) (ExtHandler, error)

	DataID() datastore.DataID

	UseCompression() dvid.Compression

	UseChecksum() dvid.Checksum

	Values() dvid.DataValues

	BlockSize() dvid.Point

	Extents() *Extents

	VersionMutex(dvid.VersionLocalID) *sync.Mutex

	ProcessChunk(*storage.Chunk)
}

IntHandler implementations handle internal DVID voxel representations, knowing how to break data into chunks (blocks for voxels). Typically, each voxels-oriented package has a Data type that fulfills the IntHandler interface.

type Metadata 

type Metadata struct {
	Axes   []Axis
	Values dvid.DataValues
}

type OpType 

type OpType int
const (
	GetOp OpType = iota
	PutOp
)

func (OpType) String 

func (o OpType) String() string

type Operation 

type Operation struct {
	ExtHandler
	OpType
}

Operation holds Voxel-specific data for processing chunks.

type Properties 

type Properties struct {
	// Values describes the data type/label for each value within a voxel.
	Values dvid.DataValues

	// Interpolable is true if voxels can be interpolated when resizing.
	Interpolable bool

	// Block size for this dataset
	BlockSize dvid.Point

	// The endianness of this loaded data.
	ByteOrder binary.ByteOrder

	Resolution
	Extents
}

func (*Properties) NdDataMetadata 

func (props *Properties) NdDataMetadata() (string, error)

NdDataSchema returns the metadata in JSON for this Data

func (*Properties) SetByConfig 

func (props *Properties) SetByConfig(config dvid.Config) error

SetByConfig sets Voxels properties based on type-specific keywords in the configuration. Any property not described in the config is left as is. See the Voxels help for listing of configurations.

func (*Properties) SetDefault 

func (props *Properties) SetDefault(values dvid.DataValues, interpolable bool) error

SetDefault sets Voxels properties to default values.

type Resolution 

type Resolution struct {
	// Resolution of voxels in volume
	VoxelSize dvid.NdFloat32

	// Units of resolution, e.g., "nanometers"
	VoxelUnits dvid.NdString
}

type VoxelGetter 

type VoxelGetter interface {
	dvid.Geometry

	Values() dvid.DataValues

	Stride() int32

	ByteOrder() binary.ByteOrder

	Data() []byte

	Interpolable() bool
}

type VoxelHandler 

type VoxelHandler interface {
	VoxelGetter
	VoxelSetter
}

VoxelHandlers can get and set n-D voxels.

type VoxelSetter 

type VoxelSetter interface {
	SetGeometry(geom dvid.Geometry)

	SetValues(values dvid.DataValues)

	SetStride(stride int32)

	SetByteOrder(order binary.ByteOrder)

	SetData(data []byte)
}

type Voxels 

type Voxels struct {
	dvid.Geometry
	// contains filtered or unexported fields
}

Voxels represents subvolumes or slices.

func NewVoxels 

func NewVoxels(geom dvid.Geometry, values dvid.DataValues, data []byte, stride int32,
	byteOrder binary.ByteOrder) *Voxels

func (*Voxels) ByteOrder 

func (v *Voxels) ByteOrder() binary.ByteOrder

func (*Voxels) BytesPerVoxel 

func (v *Voxels) BytesPerVoxel() int32

func (*Voxels) Data 

func (v *Voxels) Data() []byte

func (*Voxels) DownRes 

func (v *Voxels) DownRes(magnification dvid.Point) error

DownRes downsamples 2d Voxels data by averaging where the down-magnification are integers. If the source image size in Voxels is not an integral multiple of the reduction factor, the edge voxels on the right and bottom side are truncated. This function modifies the Voxels data. An error is returned if a non-2d Voxels receiver is used.

func (*Voxels) GetImage2d 

func (v *Voxels) GetImage2d() (*dvid.Image, error)

GetImage2d returns a 2d image suitable for use external to DVID. TODO -- Create more comprehensive handling of endianness and encoding of multibytes/voxel data into appropriate images.

func (*Voxels) Index 

func (v *Voxels) Index(c dvid.ChunkPoint) dvid.Index

func (*Voxels) IndexIterator 

func (v *Voxels) IndexIterator(chunkSize dvid.Point) (dvid.IndexIterator, error)

IndexIterator returns an iterator that can move across the voxel geometry, generating indices or index spans.

func (*Voxels) Interpolable 

func (v *Voxels) Interpolable() bool

func (*Voxels) SetByteOrder 

func (v *Voxels) SetByteOrder(order binary.ByteOrder)

func (*Voxels) SetData 

func (v *Voxels) SetData(data []byte)

func (*Voxels) SetGeometry 

func (v *Voxels) SetGeometry(geom dvid.Geometry)

func (*Voxels) SetStride 

func (v *Voxels) SetStride(stride int32)

func (*Voxels) SetValues 

func (v *Voxels) SetValues(values dvid.DataValues)

func (*Voxels) Stride 

func (v *Voxels) Stride() int32

func (*Voxels) String 

func (v *Voxels) String() string

func (*Voxels) Values 

func (v *Voxels) Values() dvid.DataValues

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.