dldataset

README

DLDataset

Download the ImageNet dataset

The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset has 1000 categories and 1.2 million images. The images do not need to be preprocessed or packaged in any database, but the validation images need to be moved into appropriate subfolders.

1. Download the images from http://image-net.org/download-images

2. Extract the training data:

mkdir train && mv ILSVRC2012_img_train.tar train/ && cd train
tar -xvf ILSVRC2012_img_train.tar && rm -f ILSVRC2012_img_train.tar
find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
cd ..

1. Extract the validation data and move images to subfolders:

mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash

Todo

• ImageNet Validation Dataset

• http://mscoco.org/dataset/

Documentation

Index

• Variables
• func Datasets() []string
• func Register(d Dataset)
• type Dataset
  • func Get(category, name string) (Dataset, error)
• type LabeledData

Variables

var (
	// Config holds the data read by rai-project/config
	Config = &dldatasetConfig{
		done: make(chan struct{}),
	}
)

Config ...

Functions

func Datasets

func Datasets() []string

Datasets ...

func Register

func Register(d Dataset)

Register ...

Types

type Dataset

type Dataset interface {
	New(ctx context.Context) (Dataset, error)
	Category() string
	Name() string
	CanonicalName() string
	Download(ctx context.Context) error
	List(ctx context.Context) ([]string, error)
	Load(ctx context.Context) error
	Get(ctx context.Context, name string) (LabeledData, error)
	Next(ctx context.Context) (LabeledData, error)
	io.Closer
}

Dataset ...

func Get

func Get(category, name string) (Dataset, error)

Get ...

type LabeledData

type LabeledData interface {
	Label() string
	Data() (interface{}, error)
}

LabeledData ...