README
¶
Tile coding (for reinforcement learning)
See indexingTiler_test.go for examples of how to use the indexing tiler. The package documentation describes the other types of Tilers.
Documentation
¶
Index ¶
Examples ¶
Constants ¶
const UnlimitedIndices = math.MaxInt64
UnlimitedIndices can be provided to NewIndexingTiler to indicate there is no maximum number of indices.
Variables ¶
This section is empty.
Functions ¶
func MaxIndices ¶
MaxIndices returns the maximum number of indices with a given maximum range of input data, number of dimensions in the input data, and number of tilings. It assumes the number of tilings is the same in each dimension. For example, when tiling a 4-dimensional input, with each input value ranging from -3 to 6, and 32 tilings, the call would be `MaxIndices(6-(-3), 4, 32)`.
func MaxIndicesForRanges ¶
MaxIndicesForRanges returns the maximum number of indices with a given maximum range of input data and number of tilings. The ranges slice contains the maximum range for each of the input values being hashed.
Types ¶
type AggregateTiler ¶
type AggregateTiler struct {
// contains filtered or unexported fields
}
AggregateTiler is used for tile coding when multiple Tilers must work together.
func NewAggregateTiler ¶
func NewAggregateTiler(tils []Tiler) (*AggregateTiler, error)
NewAggregateTiler creates a new Tiler which returns all of the hashes provided by the individual Tilers.
func NewPairsTiler ¶
func NewPairsTiler(numDims, numTilings int) (*AggregateTiler, error)
NewPairsTiler creates a new Tiler which tiles each pair of dimensions.
func NewSinglesTiler ¶
func NewSinglesTiler(numDims, numTilings int) (*AggregateTiler, error)
NewSinglesTiler creates a new Tiler which tiles each dimension individually.
func (*AggregateTiler) Tile ¶
func (til *AggregateTiler) Tile(data []float64) []uint64
Tile returns a vector of indices describing the input data.
Example ¶
til, _ := newAggregateTiler()
it, _ := NewIndexingTiler(til, UnlimitedIndices)
test := [][]float64{
{2, 4},
{2.3, 4},
{2.7, 4},
{2, 4.3},
{2.3, 4.3},
{2.7, 4.3},
{2, 4.7},
{2.3, 4.7},
{2.7, 4.7},
}
for _, data := range test {
fmt.Println("The index for", data, "is", it.Tile(data))
}
Output: The index for [2 4] is [0 1 2 3 4 5] The index for [2.3 4] is [0 6 2 3 4 5] The index for [2.7 4] is [0 6 7 3 4 8] The index for [2 4.3] is [0 1 2 9 4 5] The index for [2.3 4.3] is [0 6 2 9 4 5] The index for [2.7 4.3] is [0 6 7 9 4 8] The index for [2 4.7] is [0 1 10 9 4 11] The index for [2.3 4.7] is [0 6 10 9 4 11] The index for [2.7 4.7] is [0 6 12 9 4 13]
type HashTiler ¶
type HashTiler struct {
// contains filtered or unexported fields
}
HashTiler is used for tile coding.
func NewHashTiler ¶
NewHashTiler creates a new tile coder with a unique random seed. The `numTilings` argument determines the number of tilings that will be calculated. Tiling is uniform with the displacement vector (1,-1).
func (HashTiler) Tile ¶
Tile returns a vector of length equal to `numTilings` (the argument to `NewHashTiler`). That vector contains hashes describing the input data. The length of the input data is not checked, but it is generally expected that the input length should always be the same for calls to the same HashTiler.
type IndexTiler ¶
type IndexTiler interface {
// Tile returns a vector of indices describing the input data. The length of the input data is not checked,
// but it is generally expected that the input length should always be the same for calls to the same IndexTiler.
Tile(data []float64) []int
// CheckError returns an error if any errors have occurred.
CheckError() error
}
type IndexingTiler ¶
type IndexingTiler struct {
// contains filtered or unexported fields
}
IndexingTiler is used for tile coding when a slice of indexes is desired. It runs slower than HashTiler.
func NewIndexingTiler ¶
func NewIndexingTiler(til Tiler, indexSize int) (*IndexingTiler, error)
NewIndexingTiler creates a new Indexing Tiler, which returns a slice of indexes based on the tiles' hashes. Hashes are calculated by HashTiler. See its documentation for further details regarding usage. If indexSize is UnlimitedIndices, then the number of indices is unlimited. Otherwise, the error is provided through CheckError().
func NewIndexingTilerWithOffset ¶
func NewIndexingTilerWithOffset(til Tiler, offset, indexSize int) (*IndexingTiler, error)
NewIndexingTilerWithOffset creates a new indexing tiler, but with an offset added to each provided index. Indices output by Tile will be in the range [offset, indexSize+offset).
func (IndexingTiler) CheckError ¶
func (it IndexingTiler) CheckError() error
CheckError returns an error if more indices were used than expected. There is no reason to check it if indexSize is UnlimitedIndices.
func (*IndexingTiler) Tile ¶
func (it *IndexingTiler) Tile(data []float64) []int
Tile returns a vector of indices describing the input data. The indices range from 0 to indexSize-1 (where indexSize was an argument to NewIndexingTiler). The length of the input data is not checked, but it is generally expected that the input length should always be the same for calls to the same IndexingTiler.
Example ¶
ht, err := newUnlimitedIndexTiler(1)
if err != nil {
fmt.Println(err.Error()) // HashTiler/IndexingTiler test code should have caught all errors
}
for _, data := range [][]float64{{3.14, 2.718}, {4, 2}, {3, 3}, {3, 2}} {
fmt.Println("The index for", data, "is", ht.Tile(data))
}
Output: The index for [3.14 2.718] is [0] The index for [4 2] is [1] The index for [3 3] is [2] The index for [3 2] is [0]
Example (Second) ¶
ht, err := newUnlimitedIndexTiler(4)
if err != nil {
fmt.Println(err.Error()) // HashTiler/IndexingTiler test code should have caught all errors
}
for _, data := range [][]float64{{4.99}, {5.24}, {5.25}, {5.49}} {
fmt.Println("The indices for", data, "are", ht.Tile(data))
}
Output: The indices for [4.99] are [0 1 2 3] The indices for [5.24] are [4 1 2 3] The indices for [5.25] are [4 5 2 3] The indices for [5.49] are [4 5 2 3]
Example (Third) ¶
// Test indexing with a constant offset added to each output.
til, err := NewHashTiler(4)
if err != nil {
fmt.Println(err.Error()) // HashTiler test code should have caught all errors
}
ht, _ := NewIndexingTilerWithOffset(til, 15, UnlimitedIndices)
for _, data := range [][]float64{{4.99}, {5.24}, {5.25}, {5.49}} {
fmt.Println("The indices for", data, "are", ht.Tile(data))
}
Output: The indices for [4.99] are [15 16 17 18] The indices for [5.24] are [19 16 17 18] The indices for [5.25] are [19 20 17 18] The indices for [5.49] are [19 20 17 18]
type InvalidNumTilingsError ¶
InvalidNumTilingsError is returned
func (InvalidNumTilingsError) Error ¶
func (err InvalidNumTilingsError) Error() string
Source Files