data

Documentation

Overview

Package data provides a data pipeline for feeding arrays to models, analogous to tf.data.Dataset.

Index

• type Dataset
  • func FromNDArraySlice(arrays []*numgo.NDArray) *Dataset
  • func FromSlices(a *numgo.NDArray) (*Dataset, error)
  • func (d *Dataset) Batch(batchSize int) *Dataset
  • func (d *Dataset) Iterator() *Iterator
  • func (d *Dataset) Len() int
  • func (d *Dataset) Map(fn func(*numgo.NDArray) (*numgo.NDArray, error)) *Dataset
  • func (d *Dataset) Repeat(count int) *Dataset
  • func (d *Dataset) Shuffle(seed int64) *Dataset
  • func (d *Dataset) Take(count int) *Dataset
• type Iterator
  • func (it *Iterator) Next() ([]*numgo.NDArray, error)

Types

type Dataset

type Dataset struct {
	// contains filtered or unexported fields
}

Dataset represents a pipeline of array batches. Analogous to tf.data.Dataset.

func FromNDArraySlice

func FromNDArraySlice(arrays []*numgo.NDArray) *Dataset

FromNDArraySlice creates a Dataset from a slice of NDArrays directly.

func FromSlices

func FromSlices(a *numgo.NDArray) (*Dataset, error)

FromSlices creates a Dataset from an NDArray by slicing along axis 0. Each element in the dataset is a single sample. Analogous to tf.data.Dataset.from_tensor_slices.

func (*Dataset) Batch

func (d *Dataset) Batch(batchSize int) *Dataset

Batch sets the batch size for the dataset. Analogous to tf.data.Dataset.batch.

func (*Dataset) Iterator

func (d *Dataset) Iterator() *Iterator

Iterator creates an iterator over the dataset.

func (*Dataset) Len

func (d *Dataset) Len() int

Len returns the number of raw elements in the dataset.

func (*Dataset) Map

func (d *Dataset) Map(fn func(*numgo.NDArray) (*numgo.NDArray, error)) *Dataset

Map applies a transformation function to each element. Analogous to tf.data.Dataset.map.

func (*Dataset) Repeat

func (d *Dataset) Repeat(count int) *Dataset

Repeat sets the dataset to repeat the given number of times. count=-1 means repeat indefinitely (use Take to limit). Analogous to tf.data.Dataset.repeat.

func (*Dataset) Shuffle

func (d *Dataset) Shuffle(seed int64) *Dataset

Shuffle marks the dataset for shuffling with the given seed. Analogous to tf.data.Dataset.shuffle.

func (*Dataset) Take

func (d *Dataset) Take(count int) *Dataset

Take limits the dataset to the first count elements. Analogous to tf.data.Dataset.take.

type Iterator

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator iterates over dataset batches.

func (*Iterator) Next

func (it *Iterator) Next() ([]*numgo.NDArray, error)

Next returns the next batch of arrays. Returns nil, nil when the iterator is exhausted.