collections

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2023 License: Apache-2.0 Imports: 13 Imported by: 124

README

Collections

Collections is a library meant to simplify the experience with respect to module state handling.

Cosmos SDK modules handle their state using the KVStore interface. The problem with working with KVStore is that it forces you to think of state as a bytes KV pairings when in reality the majority of state comes from complex concrete golang objects (strings, ints, structs, etc.).

Collections allows you to work with state as if they were normal golang objects and removes the need for you to think of your state as raw bytes in your code.

It also allows you to migrate your existing state without causing any state breakage that forces you into tedious and complex chain state migrations.

Installation

To install collections in your cosmos-sdk chain project, run the following command:

go get cosmossdk.io/collections

Core types

Collections offers 5 different APIs to work with state, which will be explored in the next sections, these APIs are:

  • Map: to work with typed arbitrary KV pairings.
  • KeySet: to work with just typed keys
  • Item: to work with just one typed value
  • Sequence: which is a monotonically increasing number.
  • IndexedMap: which combines Map and KeySet to provide a Map with indexing capabilities.

Preliminary components

Before exploring the different collections types and their capability it is necessary to introduce the three components that every collection shares. In fact when instantiating a collection type by doing, for example, collections.NewMap/collections.NewItem/... you will find yourself having to pass them some common arguments.

For example, in code:

package collections

import (
    "cosmossdk.io/collections"
    storetypes "cosmossdk.io/store/types"
    sdk "github.com/cosmos/cosmos-sdk/types"
)

var AllowListPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema    collections.Schema
	AllowList collections.KeySet[string]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))

	return Keeper{
		AllowList: collections.NewKeySet(sb, AllowListPrefix, "allow_list", collections.StringKey),
	}
}

Let's analyse the shared arguments, what they do, and why we need them.

SchemaBuilder

The first argument passed is the SchemaBuilder

SchemaBuilder is a structure that keeps track of all the state of a module, it is not required by the collections to deal with state but it offers a dynamic and reflective way for clients to explore a module's state.

We instantiate a SchemaBuilder by passing it a function that given the modules store key returns the module's specific store.

We then need to pass the schema builder to every collection type we instantiate in our keeper, in our case the AllowList.

Prefix

The second argument passed to our KeySet is a collections.Prefix, a prefix represents a partition of the module's KVStore where all the state of a specific collection will be saved.

Since a module can have multiple collections, the following is expected:

  • module params will become a collections.Item
  • the AllowList is a collections.KeySet

We don't want a collection to write over the state of the other collection so we pass it a prefix, which defines a storage partition owned by the collection.

If you already built modules, the prefix translates to the items you were creating in your types/keys.go file, example: https://github.com/cosmos/cosmos-sdk/blob/main/x/feegrant/key.go#L27

your old:

var (
	// FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data
	// - 0x00<allowance_key_bytes>: allowance
	FeeAllowanceKeyPrefix = []byte{0x00}

	// FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data
	// - 0x01<allowance_prefix_queue_key_bytes>: <empty value>
	FeeAllowanceQueueKeyPrefix = []byte{0x01}
)

becomes:

var (
	// FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data
	// - 0x00<allowance_key_bytes>: allowance
	FeeAllowanceKeyPrefix = collections.NewPrefix(0)

	// FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data
	// - 0x01<allowance_prefix_queue_key_bytes>: <empty value>
	FeeAllowanceQueueKeyPrefix = collections.NewPrefix(1)
)
Rules

collections.NewPrefix accepts either uint8, string or []bytes it's good practice to use an always increasing uint8for disk space efficiency.

A collection MUST NOT share the same prefix as another collection in the same module, and a collection prefix MUST NEVER start with the same prefix as another, examples:

prefix1 := collections.NewPrefix("prefix")
prefix2 := collections.NewPrefix("prefix") // THIS IS BAD!

prefix1 := collections.NewPrefix("a")
prefix2 := collections.NewPrefix("aa") // prefix2 starts with the same as prefix1: BAD!!!
Human-Readable Name

The third parameter we pass to a collection is a string, which is a human-readable name. It is needed to make the role of a collection understandable by clients who have no clue about what a module is storing in state.

Rules

Each collection in a module MUST have a unique humanised name.

Key and Value Codecs

A collection is generic over the type you can use as keys or values. This makes collections dumb, but also means that hypothetically we can store everything that can be a go type into a collection. We are not bounded to any type of encoding (be it proto, json or whatever)

So a collection needs to be given a way to understand how to convert your keys and values to bytes. This is achieved through KeyCodec and ValueCodec, which are arguments that you pass to your collections when you're instantiating them using the collections.NewMap/collections.NewItem/... instantiation functions.

NOTE: Generally speaking you will never be required to implement your own Key/ValueCodec as the SDK and collections libraries already come with default, safe and fast implementation of those. You might need to implement them only if you're migrating to collections and there are state layout incompatibilities.

Let's explore an example:

package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	sdk "github.com/cosmos/cosmos-sdk/types"
)

var IDsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema    collections.Schema
	IDs   collections.Map[string, uint64]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))

	return Keeper{
		IDs: collections.NewMap(sb, IDsPrefix, "ids", collections.StringKey, collections.Uint64Value),
	}
}

We're now instantiating a map where the key is string and the value is uint64. We already know the first three arguments of the NewMap function.

The fourth parameter is our KeyCodec, we know that the Map has string as key so we pass it a KeyCodec that handles strings as keys.

The fifth parameter is our ValueCodec, we know that the Map as a uint64 as value so we pass it a ValueCodec that handles uint64.

Collections already comes with all the required implementations for golang primitive types.

Let's make another example, this falls closer to what we build using cosmos SDK, let's say we want to create a collections.Map that maps account addresses to their base account. So we want to map an sdk.AccAddress to an auth.BaseAccount (which is a proto):

package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema    collections.Schema
	Accounts   collections.Map[sdk.AccAddress, authtypes.BaseAccount]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewMap(sb, AccountsPrefix, "accounts",
			sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc)),
	}
}

As we can see here since our collections.Map maps sdk.AccAddress to authtypes.BaseAccount, we use the sdk.AccAddressKey which is the KeyCodec implementation for AccAddress and we use codec.CollValue to encode our proto type BaseAccount.

Generally speaking you will always find the respective key and value codecs for types in the go.mod path you're using to import that type. If you want to encode proto values refer to the codec codec.CollValue function, which allows you to encode any type implement the proto.Message interface.

Map

We analyse the first and most important collection type, the collections.Map. This is the type that everything else builds on top of.

Use case

A collections.Map is used to map arbitrary keys with arbitrary values.

Example

It's easier to explain a collections.Map capabilities through an example:

package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	"fmt"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema    collections.Schema
	Accounts   collections.Map[sdk.AccAddress, authtypes.BaseAccount]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewMap(sb, AccountsPrefix, "accounts",
			sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc)),
	}
}

func (k Keeper) CreateAccount(ctx sdk.Context, addr sdk.AccAddress, account authtypes.BaseAccount) error {
	has, err := k.Accounts.Has(ctx, addr)
	if err != nil {
		return err
	}
	if has {
		return fmt.Errorf("account already exists: %s", addr)
	}
	
	err = k.Accounts.Set(ctx, addr, account)
	if err != nil {
		return err
	}
	return nil
}

func (k Keeper) GetAccount(ctx sdk.Context, addr sdk.AccAddress) (authtypes.BaseAccount, error) {
	acc, err := k.Accounts.Get(ctx, addr)
	if err != nil {
		return authtypes.BaseAccount{}, err
	}
	
	return acc,	nil
}

func (k Keeper) RemoveAccount(ctx sdk.Context, addr sdk.AccAddress) error {
	err := k.Accounts.Remove(ctx, addr)
	if err != nil {
		return err
	}
	return nil
}
Set method

Set maps with the provided AccAddress (the key) to the auth.BaseAccount (the value).

Under the hood the collections.Map will convert the key and value to bytes using the key and value codec. It will prepend to our bytes key the prefix and store it in the KVStore of the module.

Has method

The has method reports if the provided key exists in the store.

Get method

The get method accepts the AccAddress and returns the associated auth.BaseAccount if it exists, otherwise it errors.

Remove method

The remove method accepts the AccAddress and removes it from the store. It won't report errors if it does not exist, to check for existence before removal use the Has method.

Iteration

Iteration has a separate section.

KeySet

The second type of collection is collections.KeySet, as the word suggests it maintains only a set of keys without values.

Implementation curiosity

A collections.KeySet is just a collections.Map with a key but no value. The value internally is always the same and is represented as an empty byte slice []byte{}.

Example

As always we explore the collection type through an example:

package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	"fmt"
	sdk "github.com/cosmos/cosmos-sdk/types"
)

var ValidatorsSetPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema        collections.Schema
	ValidatorsSet collections.KeySet[sdk.ValAddress]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		ValidatorsSet: collections.NewKeySet(sb, ValidatorsSetPrefix, "validators_set", sdk.ValAddressKey),
	}
}

func (k Keeper) AddValidator(ctx sdk.Context, validator sdk.ValAddress) error {
	has, err := k.ValidatorsSet.Has(ctx, validator)
	if err != nil {
		return err
	}
	if has {
		return fmt.Errorf("validator already in set: %s", validator)
	}
	
	err = k.ValidatorsSet.Set(ctx, validator)
	if err != nil {
		return err
	}
	
	return nil
}

func (k Keeper) RemoveValidator(ctx sdk.Context, validator sdk.ValAddress) error {
	err := k.ValidatorsSet.Remove(ctx, validator)
	if err != nil {
		return err
	}
	return nil
}

The first difference we notice is that KeySet needs use to specify only one type parameter: the key (sdk.ValAddress in this case). The second difference we notice is that KeySet in its NewKeySet function does not require us to specify a ValueCodec but only a KeyCodec. This is because a KeySet only saves keys and not values.

Let's explore the methods.

Has method

Has allows us to understand if a key is present in the collections.KeySet or not, functions in the same way as collections.Map.Has

Set method

Set inserts the provided key in the KeySet.

Remove method

Remove removes the provided key from the KeySet, it does not error if the key does not exist, if existence check before removal is required it needs to be coupled with the Has method.

Item

The third type of collection is the collections.Item. It stores only one single item, it's useful for example for parameters, there's only one instance of parameters in state always.

implementation curiosity

A collections.Item is just a collections.Map with no key but just a value. The key is the prefix of the collection!

Example
package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types"
)

var ParamsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema        collections.Schema
	Params collections.Item[stakingtypes.Params]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Params: collections.NewItem(sb, ParamsPrefix, "params", codec.CollValue[stakingtypes.Params](cdc)),
	}
}

func (k Keeper) UpdateParams(ctx sdk.Context, params stakingtypes.Params) error {
	err := k.Params.Set(ctx, params)
	if err != nil {
		return err
	}
	return nil
}

func (k Keeper) GetParams(ctx sdk.Context) (stakingtypes.Params, error) {
	return k.Params.Get(ctx)
}

The first key difference we notice is that we specify only one type parameter, which is the value we're storing. The second key difference is that we don't specify the KeyCodec, since we store only one item we already know the key and the fact that it is constant.

Iteration

One of the key features of the KVStore is iterating over keys.

Collections which deal with keys (so Map, KeySet and IndexedMap) allow you to iterate over keys in a safe and typed way. They all share the same API, the only difference being that KeySet returns a different type of Iterator because KeySet only deals with keys.

:::note

Every collection shares the same Iterator semantics.

:::

Let's have a look at the Map.Iterate method:

func (m Map[K, V]) Iterate(ctx context.Context, ranger Ranger[K]) (Iterator[K, V], error)

It accepts a collections.Ranger[K], which is an API that instructs map on how to iterate over keys. As always we don't need to implement anything here as collections already provides some generic Ranger implementers that expose all you need to work with ranges.

Example

We have a collections.Map that maps accounts using uint64 IDs.

package collections

import (
	"cosmossdk.io/collections"
	storetypes "cosmossdk.io/store/types"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema   collections.Schema
	Accounts collections.Map[uint64, authtypes.BaseAccount]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewMap(sb, AccountsPrefix, "accounts", collections.Uint64Key, codec.CollValue[authtypes.BaseAccount](cdc)),
	}
}

func (k Keeper) GetAllAccounts(ctx sdk.Context) ([]authtypes.BaseAccount, error) {
	// passing a nil Ranger equals to: iterate over every possible key
	iter, err := k.Accounts.Iterate(ctx, nil)
	if err != nil {
		return nil, err
	}
	accounts, err := iter.Values()
	if err != nil {
		return nil, err
	}

	return accounts, err
}

func (k Keeper) IterateAccountsBetween(ctx sdk.Context, start, end uint64) ([]authtypes.BaseAccount, error) {
	// The collections.Range API offers a lot of capabilities
	// like defining where the iteration starts or ends.
	rng := new(collections.Range[uint64]).
		StartInclusive(start).
		EndExclusive(end).
		Descending()

	iter, err := k.Accounts.Iterate(ctx, rng)
	if err != nil {
		return nil, err
	}
	accounts, err := iter.Values()
	if err != nil {
		return nil, err
	}

	return accounts, nil
}

func (k Keeper) IterateAccounts(ctx sdk.Context, do func(id uint64, acc authtypes.BaseAccount) (stop bool)) error {
	iter, err := k.Accounts.Iterate(ctx, nil)
	if err != nil {
		return err
	}
	defer iter.Close()

	for ; iter.Valid(); iter.Next() {
		kv, err := iter.KeyValue()
		if err != nil {
			return err
		}

		if do(kv.Key, kv.Value) {
			break
		}
	}
	return nil
}

Let's analyse each method in the example and how it makes use of the Iterate and the returned Iterator API.

GetAllAccounts

In GetAllAccounts we pass to our Iterate a nil Ranger. This means that the returned Iterator will include all the existing keys within the collection.

Then we use some the Values method from the returned Iterator API to collect all the values into a slice.

Iterator offers other methods such as Keys() to collect only the keys and not the values and KeyValues to collect all the keys and values.

IterateAccountsBetween

Here we make use of the collections.Range helper to specialise our range. We make it start in a point through StartInclusive and end in the other with EndExclusive, then we instruct it to report us results in reverse order through Descending

Then we pass the range instruction to Iterate and get an Iterator, which will contain only the results we specified in the range.

Then we use again th Values method of the Iterator to collect all the results.

collections.Range also offers a Prefix API which is not appliable to all keys types, for example uint64 cannot be prefix because it is of constant size, but a string key can be prefixed.

IterateAccounts

Here we showcase how to lazily collect values from an Iterator.

:::note

Keys/Values/KeyValues fully consume and close the Iterator, here we need to explicitly do a defer iterator.Close() call.

:::

Iterator also exposes a Value and Key method to collect only the current value or key, if collecting both is not needed.

:::note

For this callback pattern, collections expose a Walk API.

:::

Composite keys

So far we've worked only with simple keys, like uint64, the account address, etc. There are some more complex cases in, which we need to deal with composite keys.

A key is composite when it is composed of multiple keys, for example bank balances as stored as the composite key (AccAddress, string) where the first part is the address holding the coins and the second part is the denom.

Example, let's say address BOB holds 10atom,15osmo, this is how it is stored in state:

(bob, atom) => 10
(bob, osmos) => 15

Now this allows to efficiently get a specific denom balance of an address, by simply getting (address, denom), or getting all the balances of an address by prefixing over (address).

Let's see now how we can work with composite keys using collections.

Example

In our example we will show-case how we can use collections when we are dealing with balances, similar to bank, a balance is a mapping between (address, denom) => math.Int the composite key in our case is (address, denom).

Instantiation of a composite key collection

package collections

import (
	"cosmossdk.io/collections"
	"cosmossdk.io/math"
	storetypes "cosmossdk.io/store/types"
	sdk "github.com/cosmos/cosmos-sdk/types"
)


var BalancesPrefix = collections.NewPrefix(1)

type Keeper struct {
	Schema   collections.Schema
	Balances collections.Map[collections.Pair[sdk.AccAddress, string], math.Int]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Balances: collections.NewMap(
			sb, BalancesPrefix, "balances",
			collections.PairKeyCodec(sdk.AccAddressKey, collections.StringKey),
			math.IntValue,
		),
	}
}
The Map Key definition

First of all we can see that in order to define a composite key of two elements we use the collections.Pair type:

collections.Map[collections.Pair[sdk.AccAddress, string], math.Int]

collections.Pair defines a key composed of two other keys, in our case the first part is sdk.AccAddress, the second part is string.

The Key Codec instantiation

The arguments to instantiate are always the same, the only thing that changes is how we instantiate the KeyCodec, since this key is composed of two keys we use collections.PairKeyCodec, which generates a KeyCodec composed of two key codecs. The first one will encode the first part of the key, the second one will encode the second part of the key.

Working with composite key collections

Let's expand on the example we used before:

var BalancesPrefix = collections.NewPrefix(1)

type Keeper struct {
	Schema   collections.Schema
	Balances collections.Map[collections.Pair[sdk.AccAddress, string], math.Int]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Balances: collections.NewMap(
			sb, BalancesPrefix, "balances",
			collections.PairKeyCodec(sdk.AccAddressKey, collections.StringKey),
			math.IntValue,
		),
	}
}

func (k Keeper) SetBalance(ctx sdk.Context, address sdk.AccAddress, denom string, amount math.Int) error {
	key := collections.Join(address, denom)
	return k.Balances.Set(ctx, key, amount)
}

func (k Keeper) GetBalance(ctx sdk.Context, address sdk.AccAddress, denom string) (math.Int, error) {
	return k.Balances.Get(ctx, collections.Join(address, denom))
}

func (k Keeper) GetAllAddressBalances(ctx sdk.Context, address sdk.AccAddress) (sdk.Coins, error) {
	balances := sdk.NewCoins()

	rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address)

	iter, err := k.Balances.Iterate(ctx, rng)
	if err != nil {
		return nil, err
	}

	kvs, err := iter.KeyValues()
	if err != nil {
		return nil, err
	}

	for _, kv := range kvs {
		balances = balances.Add(sdk.NewCoin(kv.Key.K2(), kv.Value))
	}
	return balances, nil
}

func (k Keeper) GetAllAddressBalancesBetween(ctx sdk.Context, address sdk.AccAddress, startDenom, endDenom string) (sdk.Coins, error) {
    rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address).
        StartInclusive(startDenom).
        EndInclusive(endDenom)

    iter, err := k.Balances.Iterate(ctx, rng)
    if err != nil {
        return nil, err
	}
    ...
}
SetBalance

As we can see here we're setting the balance of an address for a specific denom. We use the collections.Join function to generate the composite key. collections.Join returns a collections.Pair (which is the key of our collections.Map)

collections.Pair contains the two keys we have joined, it also exposes two methods: K1 to fetch the 1st part of the key and K2 to fetch the second part.

As always, we use the collections.Map.Set method to map the composite key to our value (math.Intin this case)

GetBalance

To get a value in composite key collection, we simply use collections.Join to compose the key.

GetAllAddressBalances

We use collections.PrefixedPairRange to iterate over all the keys starting with the provided address. Concretely the iteration will report all the balances belonging to the provided address.

The first part is that we instantiate a PrefixedPairRange, which is a Ranger implementer aimed to help in Pair keys iterations.

	rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address)

As we can see here we're passing the type parameters of the collections.Pair because golang type inference with respect to generics is not as permissive as other languages, so we need to explitly say what are the types of the pair key.

GetAllAddressesBalancesBetween

This showcases how we can further specialise our range to limit the results further, by specifying the range between the second part of the key (in our case the denoms, which are strings).

IndexedMap

collections.IndexedMap is a collection that uses under the hood a collections.Map, and has a struct, which contains the indexes that we need to define.

Example

Let's say we have an auth.BaseAccount struct which looks like the following:

type BaseAccount struct {
	AccountNumber uint64     `protobuf:"varint,3,opt,name=account_number,json=accountNumber,proto3" json:"account_number,omitempty"`
	Sequence      uint64     `protobuf:"varint,4,opt,name=sequence,proto3" json:"sequence,omitempty"`
}

First of all, when we save our accounts in state we map them using a primary key sdk.AccAddress. If it were to be a collections.Map it would be collections.Map[sdk.AccAddres, authtypes.BaseAccount].

Then we also want to be able to get an account not only by its sdk.AccAddress, but also by its AccountNumber.

So we can say we want to create an Index that maps our BaseAccount to its AccountNumber.

We also know that this Index is unique. Unique means that there can only be one BaseAccount that maps to a specific AccountNumber.

First of all, we start by defining the object that contains our index:

var AccountsNumberIndexPrefix = collections.NewPrefix(1)

type AccountsIndexes struct {
	Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount]
}

func (a AccountsIndexes) IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] {
	return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{a.Number}
}

func NewAccountIndexes(sb *collections.SchemaBuilder) AccountsIndexes {
	return AccountsIndexes{
		Number: indexes.NewUnique(
			sb, AccountsNumberIndexPrefix, "accounts_by_number",
			collections.Uint64Key, sdk.AccAddressKey,
			func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) {
				return v.AccountNumber, nil
			},
		),
	}
}

We create an AccountIndexes struct which contains a field: Number. This field represents our AccountNumber index. AccountNumber is a field of authtypes.BaseAccount and it's a uint64.

Then we can see in our AccountIndexes struct the Number field is defined as:

*indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount]

Where the first type parameter is uint64, which is the field type of our index. The second type parameter is the primary key sdk.AccAddress And the third type parameter is the actual object we're storing authtypes.BaseAccount.

Then we implement a function called IndexesList on our AccountIndexes struct, this will be used by the IndexedMap to keep the underlying map in sync with the indexes, in our case Number. This function just needs to return the slice of indexes contained in the struct.

Then we create a NewAccountIndexes function that instantiates and returns the AccountsIndexes struct.

The function takes a SchemaBuilder. Then we instantiate our indexes.Unique, let's analyse the arguments we pass to indexes.NewUnique.

Instantiating a indexes.Unique

The first three arguments, we already know them, they are: SchemaBuilder, Prefix which is our index prefix (the partition where index keys relationship for the Number index will be maintained), and the human name for the Number index.

The second argument is a collections.Uint64Key which is a key codec to deal with uint64 keys, we pass that because the key we're trying to index is a uint64 key (the account number), and then we pass as fifth argument the primary key codec, which in our case is sdk.AccAddress (remember: we're mapping sdk.AccAddress => BaseAccount).

Then as last parameter we pass a function that: given the BaseAccount returns its AccountNumber.

After this we can proceed instantiating our IndexedMap.

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema   collections.Schema
	Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewIndexedMap(
			sb, AccountsPrefix, "accounts",
			sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc),
			NewAccountIndexes(sb),
		),
	}
}

As we can see here what we do, for now, is the same thing as we did for collections.Map. We pass it the SchemaBuilder, the Prefix where we plan to store the mapping between sdk.AccAddress and authtypes.BaseAccount, the human name and the respective sdk.AccAddress key codec and authtypes.BaseAccount value codec.

Then we pass the instantiation of our AccountIndexes through NewAccountIndexes.

Full example:

package docs

import (
	"cosmossdk.io/collections"
	"cosmossdk.io/collections/indexes"
	storetypes "cosmossdk.io/store/types"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsNumberIndexPrefix = collections.NewPrefix(1)

type AccountsIndexes struct {
	Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount]
}

func (a AccountsIndexes) IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] {
	return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{a.Number}
}

func NewAccountIndexes(sb *collections.SchemaBuilder) AccountsIndexes {
	return AccountsIndexes{
		Number: indexes.NewUnique(
			sb, AccountsNumberIndexPrefix, "accounts_by_number",
			collections.Uint64Key, sdk.AccAddressKey,
			func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) {
				return v.AccountNumber, nil
			},
		),
	}
}

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema   collections.Schema
	Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewIndexedMap(
			sb, AccountsPrefix, "accounts",
			sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc),
			NewAccountIndexes(sb),
		),
	}
}
Working with IndexedMaps

Whilst instantiating collections.IndexedMap is tedious, working with them is extremely smooth.

Let's take the full example, and expand it with some use-cases.

package docs

import (
	"cosmossdk.io/collections"
	"cosmossdk.io/collections/indexes"
	storetypes "cosmossdk.io/store/types"
	"github.com/cosmos/cosmos-sdk/codec"
	sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsNumberIndexPrefix = collections.NewPrefix(1)

type AccountsIndexes struct {
	Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount]
}

func (a AccountsIndexes) IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] {
	return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{a.Number}
}

func NewAccountIndexes(sb *collections.SchemaBuilder) AccountsIndexes {
	return AccountsIndexes{
		Number: indexes.NewUnique(
			sb, AccountsNumberIndexPrefix, "accounts_by_number",
			collections.Uint64Key, sdk.AccAddressKey,
			func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) {
				return v.AccountNumber, nil
			},
		),
	}
}

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
	Schema   collections.Schema
	Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes]
}

func NewKeeper(storeKey *storetypes.KVStoreKey, cdc codec.BinaryCodec) Keeper {
	sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
	return Keeper{
		Accounts: collections.NewIndexedMap(
			sb, AccountsPrefix, "accounts",
			sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc),
			NewAccountIndexes(sb),
		),
	}
}

func (k Keeper) CreateAccount(ctx sdk.Context, addr sdk.AccAddress) error {
	nextAccountNumber := k.getNextAccountNumber()
	
	newAcc := authtypes.BaseAccount{
		AccountNumber: nextAccountNumber,
		Sequence:      0,
	}
	
	return k.Accounts.Set(ctx, addr, newAcc)
}

func (k Keeper) RemoveAccount(ctx sdk.Context, addr sdk.AccAddress) error {
	return k.Accounts.Remove(ctx, addr)
} 

func (k Keeper) GetAccountByNumber(ctx sdk.Context, accNumber uint64) (sdk.AccAddress, authtypes.BaseAccount, error) {
	accAddress, err := k.Accounts.Indexes.Number.MatchExact(ctx, accNumber)
	if err != nil {
		return nil, authtypes.BaseAccount{}, err
	}
	
	acc, err := k.Accounts.Get(ctx, accAddress)
	return accAddress, acc, nil
}

func (k Keeper) GetAccountsByNumber(ctx sdk.Context, startAccNum, endAccNum uint64) ([]authtypes.BaseAccount, error) {
	rng := new(collections.Range[uint64]).
		StartInclusive(startAccNum).
		EndInclusive(endAccNum)
	
	iter, err := k.Accounts.Indexes.Number.Iterate(ctx, rng)
	if err != nil {
		return nil, err
	}
	
	return indexes.CollectValues(ctx, k.Accounts, iter)
}


func (k Keeper) getNextAccountNumber() uint64 {
	return 0
}

Collections with interfaces as values

Although cosmos-sdk is shifting away from the usage of interface registry, there are still some places where it is used. In order to support old code, we have to support collections with interface values.

The generic codec.CollValue is not able to handle interface values, so we need to use a special type codec.CollValueInterface. codec.CollValueInterface takes a codec.BinaryCodec as an argument, and uses it to marshal and unmarshal values as interfaces. The codec.CollValueInterface lives in the codec package, whose import path is github.com/cosmos/cosmos-sdk/codec.

Instantiating Collections with interface values

In order to instantiate a collection with interface values, we need to use codec.CollValueInterface instead of codec.CollValue.

package example

import (
    "cosmossdk.io/collections"
    storetypes "cosmossdk.io/store/types"
    "github.com/cosmos/cosmos-sdk/codec"
    sdk "github.com/cosmos/cosmos-sdk/types"
	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
)

var AccountsPrefix = collections.NewPrefix(0)

type Keeper struct {
    Schema   collections.Schema
    Accounts *collections.Map[sdk.AccAddress, sdk.AccountI]
}

func NewKeeper(cdc codec.BinaryCodec, storeKey *storetypes.KVStoreKey) Keeper {
    sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
    return Keeper{
        Accounts: collections.NewMap(
            sb, AccountsPrefix, "accounts",
            sdk.AccAddressKey, codec.CollInterfaceValue[sdk.AccountI](cdc),
        ),
    }
}

func (k Keeper) SaveBaseAccount(ctx sdk.Context, account authtypes.BaseAccount) error {
    return k.Accounts.Set(ctx, account.GetAddress(), account)
}

func (k Keeper) SaveModuleAccount(ctx sdk.Context, account authtypes.ModuleAccount) error {
    return k.Accounts.Set(ctx, account.GetAddress(), account)
}

func (k Keeper) GetAccount(ctx sdk.context, addr sdk.AccAddress) (sdk.AccountI, error) {
    return k.Accounts.Get(ctx, addr)
}

Triple key

The collections.Triple is a special type of key composed of three keys, it's identical to collections.Pair.

Let's see an example.

package example

import (
 "context"

 "cosmossdk.io/collections"
 storetypes "cosmossdk.io/store/types"
 "github.com/cosmos/cosmos-sdk/codec"
)

type AccAddress = string
type ValAddress = string

type Keeper struct {
 // let's simulate we have redelegations which are stored as a triple key composed of
 // the delegator, the source validator and the destination validator.
 Redelegations collections.KeySet[collections.Triple[AccAddress, ValAddress, ValAddress]]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
 sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
 return Keeper{
  Redelegations: collections.NewKeySet(sb, collections.NewPrefix(0), "redelegations", collections.TripleKeyCodec(collections.StringKey, collections.StringKey, collections.StringKey)
 }
}

// RedelegationsByDelegator iterates over all the redelegations of a given delegator and calls onResult providing
// each redelegation from source validator towards the destination validator.
func (k Keeper) RedelegationsByDelegator(ctx context.Context, delegator AccAddress, onResult func(src, dst ValAddress) (stop bool, err error)) error {
 rng := collections.NewPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator)
 return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) {
  return onResult(key.K2(), key.K3())
 })
}

// RedelegationsByDelegatorAndValidator iterates over all the redelegations of a given delegator and its source validator and calls onResult for each
// destination validator.
func (k Keeper) RedelegationsByDelegatorAndValidator(ctx context.Context, delegator AccAddress, validator ValAddress, onResult func(dst ValAddress) (stop bool, err error)) error {
 rng := collections.NewSuperPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator, validator)
 return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) {
  return onResult(key.K3())
 })
}

Advanced Usages

Alternative Value Codec

The codec.AltValueCodec allows a collection to decode values using a different codec than the one used to encode them. Basically it enables to decode two different byte representations of the same concrete value. It can be used to lazily migrate values from one bytes representation to another, as long as the new representation is not able to decode the old one.

A concrete example can be found in x/bank where the balance was initially stored as Coin and then migrated to Int.


var BankBalanceValueCodec = codec.NewAltValueCodec(sdk.IntValue, func(b []byte) (sdk.Int, error) {
    coin := sdk.Coin{}
    err := coin.Unmarshal(b)
    if err != nil {
        return sdk.Int{}, err
    }
    return coin.Amount, nil
})

The above example shows how to create an AltValueCodec that can decode both sdk.Int and sdk.Coin values. The provided decoder function will be used as a fallback in case the default decoder fails. When the value will be encoded back into state it will use the default encoder. This allows to lazily migrate values to a new bytes representation.

Documentation

Index

Constants

View Source 
const DefaultSequenceStart uint64 = 0

DefaultSequenceStart defines the default starting number of a sequence.

View Source 
const NameRegex = "[A-Za-z][A-Za-z0-9_]*"

NameRegex is the regular expression that all valid collection names must match.

Variables

View Source 
var (
	// ErrNotFound is returned when the provided key is not present in the StorageProvider.
	ErrNotFound = errors.New("collections: not found")
	// ErrEncoding is returned when something fails during key or value encoding/decoding.
	ErrEncoding = codec.ErrEncoding
	// ErrConflict is returned when there are conflicts, for example in UniqueIndex.
	ErrConflict = errors.New("collections: conflict")
)
View Source 
var (
	// Uint16Key can be used to encode uint16 keys. Encoding is big endian to retain ordering.
	Uint16Key = codec.NewUint16Key[uint16]()
	// Uint32Key can be used to encode uint32 keys. Encoding is big endian to retain ordering.
	Uint32Key = codec.NewUint32Key[uint32]()
	// Uint64Key can be used to encode uint64 keys. Encoding is big endian to retain ordering.
	Uint64Key = codec.NewUint64Key[uint64]()
	// Int32Key can be used to encode int32 keys. Encoding retains ordering by toggling the MSB.
	Int32Key = codec.NewInt32Key[int32]()
	// Int64Key can be used to encode int64. Encoding retains ordering by toggling the MSB.
	Int64Key = codec.NewInt64Key[int64]()
	// StringKey can be used to encode string keys. The encoding just converts the string
	// to bytes.
	// Non-terminality in multipart keys is handled by appending the StringDelimiter,
	// this means that a string key when used as the non final part of a multipart key cannot
	// contain the StringDelimiter.
	// Lexicographical ordering is retained both in non and multipart keys.
	StringKey = codec.NewStringKeyCodec[string]()
	// BytesKey can be used to encode bytes keys. The encoding will just use
	// the provided bytes.
	// When used as the non-terminal part of a multipart key, we prefix the bytes key
	// with a single byte representing the length of the key. This means two things:
	// 1. When used in multipart keys the length can be at maximum 255 (max number that
	// can be represented with a single byte).
	// 2. When used in multipart keys the lexicographical ordering is lost due to the
	// length prefixing.
	// JSON encoding represents a bytes key as a hex encoded string.
	BytesKey = codec.NewBytesKey[[]byte]()
	// BoolKey can be used to encode booleans. It uses a single byte to represent the boolean.
	// 0x0 is used to represent false, and 0x1 is used to represent true.
	BoolKey = codec.NewBoolKey[bool]()
)
View Source 
var (
	// BoolValue implements a ValueCodec for bool.
	BoolValue = codec.KeyToValueCodec(BoolKey)
	// Uint16Value implements a ValueCodec for uint16.
	Uint16Value = codec.KeyToValueCodec(Uint16Key)
	// Uint32Value implements a ValueCodec for uint32.
	Uint32Value = codec.KeyToValueCodec(Uint32Key)
	// Uint64Value implements a ValueCodec for uint64.
	Uint64Value = codec.KeyToValueCodec(Uint64Key)
	// Int32Value implements a ValueCodec for int32.
	Int32Value = codec.KeyToValueCodec(Int32Key)
	// Int64Value implements a ValueCodec for int64.
	Int64Value = codec.KeyToValueCodec(Int64Key)
	// StringValue implements a ValueCodec for string.
	StringValue = codec.KeyToValueCodec(StringKey)
	// BytesValue implements a ValueCodec for bytes.
	BytesValue = codec.KeyToValueCodec(BytesKey)
)
View Source 
var ErrInvalidIterator = errors.New("collections: invalid iterator")

ErrInvalidIterator is returned when an Iterate call resulted in an invalid iterator.

Functions

func EncodeKeyWithPrefix added in v0.2.0 

func EncodeKeyWithPrefix[K any](prefix []byte, kc codec.KeyCodec[K], key K) ([]byte, error)

EncodeKeyWithPrefix returns how the collection would store the key in storage given prefix, key codec and the concrete key.

func PairKeyCodec 

func PairKeyCodec[K1, K2 any](keyCodec1 codec.KeyCodec[K1], keyCodec2 codec.KeyCodec[K2]) codec.KeyCodec[Pair[K1, K2]]

PairKeyCodec instantiates a new KeyCodec instance that can encode the Pair, given the KeyCodec of the first part of the key and the KeyCodec of the second part of the key.

func TripleKeyCodec added in v0.4.0 

func TripleKeyCodec[K1, K2, K3 any](keyCodec1 codec.KeyCodec[K1], keyCodec2 codec.KeyCodec[K2], keyCodec3 codec.KeyCodec[K3]) codec.KeyCodec[Triple[K1, K2, K3]]

TripleKeyCodec instantiates a new KeyCodec instance that can encode the Triple, given the KeyCodecs of the three parts of the key, in order.

func WithKeySetUncheckedValue added in v0.3.0 

func WithKeySetUncheckedValue() func(opt *keySetOptions)

WithKeySetUncheckedValue changes the behavior of the KeySet when it encounters a value different from '[]byte{}', by default the KeySet errors when this happens. This option allows to ignore the value and continue with the operation, in turn the value will be cleared out and set to '[]byte{}'. You should never use this option if you're creating a new state object from scratch. This should be used only to behave nicely in case you have used values different from '[]byte{}' in your storage before migrating to collections.

Types

type Collection added in v0.2.0 

type Collection interface {
	// GetName is the unique name of the collection within a schema. It must
	// match format specified by NameRegex.
	GetName() string

	// GetPrefix is the unique prefix of the collection within a schema.
	GetPrefix() []byte

	// ValueCodec returns the codec used to encode/decode values of the collection.
	ValueCodec() codec.UntypedValueCodec
	// contains filtered or unexported methods
}

Collection is the interface that all collections implement. It will eventually include methods for importing/exporting genesis data and schema reflection for clients. NOTE: Unstable.

type Index 

type Index[PrimaryKey, Value any] interface {
	// Reference creates a reference between the provided primary key and value.
	// It provides a lazyOldValue function that if called will attempt to fetch
	// the previous old value, returns ErrNotFound if no value existed.
	Reference(ctx context.Context, pk PrimaryKey, newValue Value, lazyOldValue func() (Value, error)) error
	// Unreference removes the reference between the primary key and value.
	// If error is ErrNotFound then it means that the value did not exist before.
	Unreference(ctx context.Context, pk PrimaryKey, lazyOldValue func() (Value, error)) error
}

Index represents an index of the Value indexed using the type PrimaryKey.

type IndexedMap 

type IndexedMap[PrimaryKey, Value any, Idx Indexes[PrimaryKey, Value]] struct {
	Indexes Idx
	// contains filtered or unexported fields
}

IndexedMap works like a Map but creates references between fields of Value and its PrimaryKey. These relationships are expressed and maintained using the Indexes type. Internally IndexedMap can be seen as a partitioned collection, one partition is a Map[PrimaryKey, Value], that maintains the object, the second are the Indexes.

func NewIndexedMap 

func NewIndexedMap[PrimaryKey, Value any, Idx Indexes[PrimaryKey, Value]](
	schema *SchemaBuilder,
	prefix Prefix,
	name string,
	pkCodec codec.KeyCodec[PrimaryKey],
	valueCodec codec.ValueCodec[Value],
	indexes Idx,
) *IndexedMap[PrimaryKey, Value, Idx]

NewIndexedMap instantiates a new IndexedMap. Accepts a SchemaBuilder, a Prefix, a humanized name that defines the name of the collection, the primary key codec which is basically what IndexedMap uses to encode the primary key to bytes, the value codec which is what the IndexedMap uses to encode the value. Then it expects the initialized indexes.

func (*IndexedMap[PrimaryKey, Value, Idx]) Get 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Get(ctx context.Context, pk PrimaryKey) (Value, error)

Get gets the object given its primary key.

func (*IndexedMap[PrimaryKey, Value, Idx]) Has 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Has(ctx context.Context, pk PrimaryKey) (bool, error)

Has reports if exists a value with the provided primary key.

func (*IndexedMap[PrimaryKey, Value, Idx]) Iterate 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Iterate(ctx context.Context, ranger Ranger[PrimaryKey]) (Iterator[PrimaryKey, Value], error)

Iterate allows to iterate over the objects given a Ranger of the primary key.

func (*IndexedMap[PrimaryKey, Value, Idx]) IterateRaw 

func (m *IndexedMap[PrimaryKey, Value, Idx]) IterateRaw(ctx context.Context, start, end []byte, order Order) (Iterator[PrimaryKey, Value], error)

IterateRaw iterates the IndexedMap using raw bytes keys. Follows the same semantics as Map.IterateRaw

func (*IndexedMap[PrimaryKey, Value, Idx]) KeyCodec 

func (m *IndexedMap[PrimaryKey, Value, Idx]) KeyCodec() codec.KeyCodec[PrimaryKey]

func (*IndexedMap[PrimaryKey, Value, Idx]) Remove 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Remove(ctx context.Context, pk PrimaryKey) error

Remove removes the value associated with the primary key from the map. Then it iterates over all the indexes and instructs them to remove all the references associated with the removed value.

func (*IndexedMap[PrimaryKey, Value, Idx]) Set 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Set(ctx context.Context, pk PrimaryKey, value Value) error

Set maps the value using the primary key. It will also iterate every index and instruct them to add or update the indexes.

func (*IndexedMap[PrimaryKey, Value, Idx]) ValueCodec 

func (m *IndexedMap[PrimaryKey, Value, Idx]) ValueCodec() codec.ValueCodec[Value]

func (*IndexedMap[PrimaryKey, Value, Idx]) Walk 

func (m *IndexedMap[PrimaryKey, Value, Idx]) Walk(ctx context.Context, ranger Ranger[PrimaryKey], walkFunc func(key PrimaryKey, value Value) (stop bool, err error)) error

Walk applies the same semantics as Map.Walk.

type Indexes 

type Indexes[PrimaryKey, Value any] interface {
	// IndexesList is implemented by the Indexes type
	// and returns all the grouped Index of Value.
	IndexesList() []Index[PrimaryKey, Value]
}

Indexes represents a type which groups multiple Index of one Value saved with the provided PrimaryKey. Indexes is just meant to be a struct containing all the indexes to maintain relationship for.

type Item 

type Item[V any] Map[noKey, V]

Item is a type declaration based on Map with a non-existent key.

func NewItem 

func NewItem[V any](
	schema *SchemaBuilder,
	prefix Prefix,
	name string,
	valueCodec codec.ValueCodec[V],
) Item[V]

NewItem instantiates a new Item instance, given the value encoder of the item V. Name and prefix must be unique within the schema and name must match the format specified by NameRegex, or else this method will panic.

func (Item[V]) Get 

func (i Item[V]) Get(ctx context.Context) (V, error)

Get gets the item, if it is not set it returns an ErrNotFound error. If value decoding fails then an ErrEncoding is returned.

func (Item[V]) Has 

func (i Item[V]) Has(ctx context.Context) (bool, error)

Has reports whether the item exists in the store or not. Returns an error in case

func (Item[V]) Remove 

func (i Item[V]) Remove(ctx context.Context) error

Remove removes the item in the store.

func (Item[V]) Set 

func (i Item[V]) Set(ctx context.Context, value V) error

Set sets the item in the store. If Value encoding fails then an ErrEncoding is returned.

type Iterator 

type Iterator[K, V any] struct {
	// contains filtered or unexported fields
}

Iterator defines a generic wrapper around an storetypes.Iterator. This iterator provides automatic key and value encoding, it assumes all the keys and values contained within the storetypes.Iterator range are the same.

func (Iterator[K, V]) Close 

func (i Iterator[K, V]) Close() error

func (Iterator[K, V]) Key 

func (i Iterator[K, V]) Key() (K, error)

Key returns the current storetypes.Iterator decoded key.

func (Iterator[K, V]) KeyValue 

func (i Iterator[K, V]) KeyValue() (kv KeyValue[K, V], err error)

KeyValue returns the current key and value decoded.

func (Iterator[K, V]) KeyValues 

func (i Iterator[K, V]) KeyValues() ([]KeyValue[K, V], error)

KeyValues fully consumes the iterator and returns the list of key and values within the iterator range.

func (Iterator[K, V]) Keys 

func (i Iterator[K, V]) Keys() ([]K, error)

Keys fully consumes the iterator and returns all the decoded keys contained within the range.

func (Iterator[K, V]) Next 

func (i Iterator[K, V]) Next()

func (Iterator[K, V]) Valid 

func (i Iterator[K, V]) Valid() bool

func (Iterator[K, V]) Value 

func (i Iterator[K, V]) Value() (V, error)

Value returns the current iterator value bytes decoded.

func (Iterator[K, V]) Values 

func (i Iterator[K, V]) Values() ([]V, error)

Values fully consumes the iterator and returns all the decoded values contained within the range.

type KeySet 

type KeySet[K any] Map[K, NoValue]

KeySet builds on top of a Map and represents a collection retaining only a set of keys and no value. It can be used, for example, in an allow list.

func NewKeySet 

func NewKeySet[K any](
	schema *SchemaBuilder,
	prefix Prefix,
	name string,
	keyCodec codec.KeyCodec[K],
	options ...func(opt *keySetOptions),
) KeySet[K]

NewKeySet returns a KeySet given a Schema, Prefix a human name for the collection and a KeyCodec for the key K.

func (KeySet[K]) Clear added in v0.3.0 

func (k KeySet[K]) Clear(ctx context.Context, ranger Ranger[K]) error

Clear clears the KeySet using the provided Ranger. Refer to Map.Clear for behavioral documentation.

func (KeySet[K]) Has 

func (k KeySet[K]) Has(ctx context.Context, key K) (bool, error)

Has returns if the key is present in the KeySet. An error is returned only in case of encoding problems.

func (KeySet[K]) Iterate 

func (k KeySet[K]) Iterate(ctx context.Context, ranger Ranger[K]) (KeySetIterator[K], error)

Iterate iterates over the keys given the provided Ranger. If ranger is nil, the KeySetIterator will include all the existing keys within the KeySet.

func (KeySet[K]) IterateRaw 

func (k KeySet[K]) IterateRaw(ctx context.Context, start, end []byte, order Order) (Iterator[K, NoValue], error)

func (KeySet[K]) KeyCodec 

func (k KeySet[K]) KeyCodec() codec.KeyCodec[K]

func (KeySet[K]) Remove 

func (k KeySet[K]) Remove(ctx context.Context, key K) error

Remove removes the key for the KeySet. An error is returned in case of encoding error, it won't report through the error if the key was removed or not.

func (KeySet[K]) Set 

func (k KeySet[K]) Set(ctx context.Context, key K) error

Set adds the key to the KeySet. Errors on encoding problems.

func (KeySet[K]) ValueCodec 

func (k KeySet[K]) ValueCodec() codec.ValueCodec[NoValue]

func (KeySet[K]) Walk 

func (k KeySet[K]) Walk(ctx context.Context, ranger Ranger[K], walkFunc func(key K) (stop bool, err error)) error

Walk provides the same functionality as Map.Walk, but callbacks the walk function only with the key.

type KeySetIterator 

type KeySetIterator[K any] Iterator[K, NoValue]

KeySetIterator works like an Iterator, but it does not expose any API to deal with values.

func (KeySetIterator[K]) Close 

func (i KeySetIterator[K]) Close() error

func (KeySetIterator[K]) Key 

func (i KeySetIterator[K]) Key() (K, error)

func (KeySetIterator[K]) Keys 

func (i KeySetIterator[K]) Keys() ([]K, error)

func (KeySetIterator[K]) Next 

func (i KeySetIterator[K]) Next()

func (KeySetIterator[K]) Valid 

func (i KeySetIterator[K]) Valid() bool

type KeyValue 

type KeyValue[K, V any] struct {
	Key   K
	Value V
}

KeyValue represent a Key and Value pair of an iteration.

type Map 

type Map[K, V any] struct {
	// contains filtered or unexported fields
}

Map represents the basic collections object. It is used to map arbitrary keys to arbitrary objects.

func NewMap 

func NewMap[K, V any](
	schemaBuilder *SchemaBuilder,
	prefix Prefix,
	name string,
	keyCodec codec.KeyCodec[K],
	valueCodec codec.ValueCodec[V],
) Map[K, V]

NewMap returns a Map given a StoreKey, a Prefix, human-readable name and the relative value and key encoders. Name and prefix must be unique within the schema and name must match the format specified by NameRegex, or else this method will panic.

func (Map[K, V]) Clear added in v0.3.0 

func (m Map[K, V]) Clear(ctx context.Context, ranger Ranger[K]) error

Clear clears the collection contained within the provided key range. A nil ranger equals to clearing the whole collection. NOTE: this API needs to be used with care, considering that as of today cosmos-sdk stores the deletion records to be committed in a memory cache, clearing a lot of data might make the node go OOM.

func (Map[K, V]) Get 

func (m Map[K, V]) Get(ctx context.Context, key K) (v V, err error)

Get returns the value associated with the provided key, errors with ErrNotFound if the key does not exist, or with ErrEncoding if the key or value decoding fails.

func (Map[K, V]) GetName added in v0.2.0 

func (m Map[K, V]) GetName() string

func (Map[K, V]) GetPrefix added in v0.2.0 

func (m Map[K, V]) GetPrefix() []byte

func (Map[K, V]) Has 

func (m Map[K, V]) Has(ctx context.Context, key K) (bool, error)

Has reports whether the key is present in storage or not. Errors with ErrEncoding if key encoding fails.

func (Map[K, V]) Iterate 

func (m Map[K, V]) Iterate(ctx context.Context, ranger Ranger[K]) (Iterator[K, V], error)

Iterate provides an Iterator over K and V. It accepts a Ranger interface. A nil ranger equals to iterate over all the keys in ascending order.

func (Map[K, V]) IterateRaw 

func (m Map[K, V]) IterateRaw(ctx context.Context, start, end []byte, order Order) (Iterator[K, V], error)

IterateRaw iterates over the collection. The iteration range is untyped, it uses raw bytes. The resulting Iterator is typed. A nil start iterates from the first key contained in the collection. A nil end iterates up to the last key contained in the collection. A nil start and a nil end iterates over every key contained in the collection. TODO(tip): simplify after https://github.com/cosmos/cosmos-sdk/pull/14310 is merged

func (Map[K, V]) KeyCodec 

func (m Map[K, V]) KeyCodec() codec.KeyCodec[K]

KeyCodec returns the Map's KeyCodec.

func (Map[K, V]) Remove 

func (m Map[K, V]) Remove(ctx context.Context, key K) error

Remove removes the key from the storage. Errors with ErrEncoding if key encoding fails. If the key does not exist then this is a no-op.

func (Map[K, V]) Set 

func (m Map[K, V]) Set(ctx context.Context, key K, value V) error

Set maps the provided value to the provided key in the store. Errors with ErrEncoding if key or value encoding fails.

func (Map[K, V]) ValueCodec 

func (m Map[K, V]) ValueCodec() codec.ValueCodec[V]

ValueCodec returns the Map's ValueCodec.

func (Map[K, V]) Walk 

func (m Map[K, V]) Walk(ctx context.Context, ranger Ranger[K], walkFunc func(key K, value V) (stop bool, err error)) error

Walk iterates over the Map with the provided range, calls the provided walk function with the decoded key and value. If the callback function returns true then the walking is stopped. A nil ranger equals to walking over the entire key and value set.

type NoValue 

type NoValue struct{}

NoValue is a type that can be used to represent a non-existing value.

func (NoValue) Decode 

func (NoValue) Decode(b []byte) (NoValue, error)

func (NoValue) DecodeJSON 

func (n NoValue) DecodeJSON(b []byte) (NoValue, error)

func (NoValue) Encode 

func (NoValue) Encode(_ NoValue) ([]byte, error)

func (NoValue) EncodeJSON 

func (n NoValue) EncodeJSON(_ NoValue) ([]byte, error)

func (NoValue) Stringify 

func (NoValue) Stringify(_ NoValue) string

func (NoValue) ValueType 

func (n NoValue) ValueType() string

type Order 

type Order uint8

Order defines the key order. 

const (
	// OrderAscending instructs the Iterator to provide keys from the smallest to the greatest.
	OrderAscending Order = 0
	// OrderDescending instructs the Iterator to provide keys from the greatest to the smallest.
	OrderDescending Order = 1
)

type Pair 

type Pair[K1, K2 any] struct {
	// contains filtered or unexported fields
}

Pair defines a key composed of two keys.

func Join 

func Join[K1, K2 any](key1 K1, key2 K2) Pair[K1, K2]

Join creates a new Pair instance composed of the two provided keys, in order.

func PairPrefix 

func PairPrefix[K1, K2 any](key K1) Pair[K1, K2]

PairPrefix creates a new Pair instance composed only of the first part of the key.

func (Pair[K1, K2]) K1 

func (p Pair[K1, K2]) K1() (k1 K1)

K1 returns the first part of the key. If not present the zero value is returned.

func (Pair[K1, K2]) K2 

func (p Pair[K1, K2]) K2() (k2 K2)

K2 returns the second part of the key. If not present the zero value is returned.

type PairRange 

type PairRange[K1, K2 any] struct {
	// contains filtered or unexported fields
}

PairRange is an API that facilitates working with Pair iteration. It implements the Ranger API. Unstable: API and methods are currently unstable.

func NewPrefixUntilPairRange added in v0.2.0 

func NewPrefixUntilPairRange[K1, K2 any](prefix K1) *PairRange[K1, K2]

NewPrefixUntilPairRange defines a collection query which ranges until the provided Pair prefix. Unstable: this API might change in the future.

func NewPrefixedPairRange 

func NewPrefixedPairRange[K1, K2 any](prefix K1) *PairRange[K1, K2]

NewPrefixedPairRange creates a new PairRange which will prefix over all the keys starting with the provided prefix.

func (*PairRange[K1, K2]) Descending 

func (p *PairRange[K1, K2]) Descending() *PairRange[K1, K2]

func (*PairRange[K1, K2]) EndExclusive 

func (p *PairRange[K1, K2]) EndExclusive(k2 K2) *PairRange[K1, K2]

func (*PairRange[K1, K2]) EndInclusive 

func (p *PairRange[K1, K2]) EndInclusive(k2 K2) *PairRange[K1, K2]

func (*PairRange[K1, K2]) RangeValues 

func (p *PairRange[K1, K2]) RangeValues() (start, end *RangeKey[Pair[K1, K2]], order Order, err error)

func (*PairRange[K1, K2]) StartExclusive 

func (p *PairRange[K1, K2]) StartExclusive(k2 K2) *PairRange[K1, K2]

func (*PairRange[K1, K2]) StartInclusive 

func (p *PairRange[K1, K2]) StartInclusive(k2 K2) *PairRange[K1, K2]

type Prefix 

type Prefix []byte

Prefix defines a segregation bytes namespace for specific collections objects.

func NewPrefix 

func NewPrefix[T interface{ int | string | []byte }](identifier T) Prefix

NewPrefix returns a Prefix given the provided namespace identifier. In the same module, no prefixes should share the same starting bytes meaning that having two namespaces whose bytes representation is: p1 := []byte("prefix") p2 := []byte("prefix1") yields to iterations of p1 overlapping over p2. If a numeric prefix is provided, it must be between 0 and 255 (uint8). If out of bounds this function will panic. Reason for which this function is constrained to `int` instead of `uint8` is for API ergonomics, golang's type inference will infer int properly but not uint8 meaning that developers would need to write NewPrefix(uint8(number)) for numeric prefixes.

func (Prefix) Bytes 

func (n Prefix) Bytes() []byte

Bytes returns the raw Prefix bytes.

type Range 

type Range[K any] struct {
	// contains filtered or unexported fields
}

Range is a Ranger implementer.

func (*Range[K]) Descending 

func (r *Range[K]) Descending() *Range[K]

func (*Range[K]) EndExclusive 

func (r *Range[K]) EndExclusive(end K) *Range[K]

EndExclusive makes the range contain only keys which are smaller to the provided end K.

func (*Range[K]) EndInclusive 

func (r *Range[K]) EndInclusive(end K) *Range[K]

EndInclusive makes the range contain only keys which are smaller or equal to the provided end K.

func (*Range[K]) Prefix 

func (r *Range[K]) Prefix(key K) *Range[K]

Prefix sets a fixed prefix for the key range.

func (*Range[K]) RangeValues 

func (r *Range[K]) RangeValues() (start, end *RangeKey[K], order Order, err error)

func (*Range[K]) StartExclusive 

func (r *Range[K]) StartExclusive(start K) *Range[K]

StartExclusive makes the range contain only keys which are bigger to the provided start K.

func (*Range[K]) StartInclusive 

func (r *Range[K]) StartInclusive(start K) *Range[K]

StartInclusive makes the range contain only keys which are bigger or equal to the provided start K.

type RangeKey 

type RangeKey[K any] struct {
	// contains filtered or unexported fields
}

RangeKey wraps a generic range key K, acts as an enum which defines different ways to encode the wrapped key to bytes when it's being used in an iteration.

func RangeKeyExact 

func RangeKeyExact[K any](key K) *RangeKey[K]

RangeKeyExact instantiates a RangeKey that applies no modifications to the key K. So its bytes representation will not be altered.

func RangeKeyNext 

func RangeKeyNext[K any](key K) *RangeKey[K]

RangeKeyNext instantiates a RangeKey that when encoded to bytes identifies the next key after the provided key K. Example: given a string key "ABCD" the next key is bytes("ABCD\0") It's useful when defining inclusivity or exclusivity of a key in store iteration. Specifically: to make an Iterator start exclude key K I would return a RangeKeyNext(key) in the Ranger start.

func RangeKeyPrefixEnd 

func RangeKeyPrefixEnd[K any](key K) *RangeKey[K]

RangeKeyPrefixEnd instantiates a RangeKey that when encoded to bytes identifies the key that would end the prefix of the key K. Example: if the string key "ABCD" is provided, it would be encoded as bytes("ABCE").

type Ranger 

type Ranger[K any] interface {
	// RangeValues is defined by Ranger implementers.
	// The implementer can optionally return a start and an end.
	// If start is nil and end is not, the iteration will include all the keys
	// in the collection up until the provided end.
	// If start is defined and end is nil, the iteration will include all the keys
	// in the collection starting from the provided start.
	// If both are nil then the iteration will include all the possible keys in the
	// collection.
	// Order defines the order of the iteration, if order is OrderAscending then the
	// iteration will yield keys from the smallest to the biggest, if order
	// is OrderDescending then the iteration will yield keys from the biggest to the smallest.
	// Ordering is defined by the keys bytes representation, which is dependent on the KeyCodec used.
	RangeValues() (start, end *RangeKey[K], order Order, err error)
}

Ranger defines a generic interface that provides a range of keys.

func NewPrefixUntilTripleRange added in v0.4.0 

func NewPrefixUntilTripleRange[K1, K2, K3 any](k1 K1) Ranger[Triple[K1, K2, K3]]

NewPrefixUntilTripleRange defines a collection query which ranges until the provided Pair prefix. Unstable: this API might change in the future.

func NewPrefixedTripleRange added in v0.4.0 

func NewPrefixedTripleRange[K1, K2, K3 any](k1 K1) Ranger[Triple[K1, K2, K3]]

NewPrefixedTripleRange provides a Range for all keys prefixed with the given first part of the Triple key.

func NewSuperPrefixedTripleRange added in v0.4.0 

func NewSuperPrefixedTripleRange[K1, K2, K3 any](k1 K1, k2 K2) Ranger[Triple[K1, K2, K3]]

NewSuperPrefixedTripleRange provides a Range for all keys prefixed with the given first and second parts of the Triple key.

type Schema 

type Schema struct {
	// contains filtered or unexported fields
}

Schema specifies a group of collections stored within the storage specified by a single store key. All the collections within the schema must have a unique binary prefix and human-readable name. Schema will eventually include methods for importing/exporting genesis data and for schema reflection for clients.

func NewMemoryStoreSchema 

func NewMemoryStoreSchema(service store.MemoryStoreService) Schema

NewMemoryStoreSchema creates a new schema for the provided MemoryStoreService.

func NewSchema 

func NewSchema(service store.KVStoreService) Schema

NewSchema creates a new schema for the provided KVStoreService.

func NewSchemaFromAccessor 

func NewSchemaFromAccessor(accessor func(context.Context) store.KVStore) Schema
NewSchemaFromAccessor(func(ctx context.Context) store.KVStore {
		return sdk.UnwrapSDKContext(ctx).KVStore(kvStoreKey)
}

func (Schema) DefaultGenesis 

func (s Schema) DefaultGenesis(target appmodule.GenesisTarget) error

DefaultGenesis implements the appmodule.HasGenesis.DefaultGenesis method.

func (Schema) ExportGenesis 

func (s Schema) ExportGenesis(ctx context.Context, target appmodule.GenesisTarget) error

ExportGenesis implements the appmodule.HasGenesis.ExportGenesis method.

func (Schema) InitGenesis 

func (s Schema) InitGenesis(ctx context.Context, source appmodule.GenesisSource) error

InitGenesis implements the appmodule.HasGenesis.InitGenesis method.

func (Schema) IsAppModule added in v0.4.0 

func (s Schema) IsAppModule()

IsAppModule implements the appmodule.AppModule interface.

func (Schema) IsOnePerModuleType added in v0.4.0 

func (s Schema) IsOnePerModuleType()

IsOnePerModuleType implements the depinject.OnePerModuleType interface.

func (Schema) ListCollections added in v0.2.0 

func (s Schema) ListCollections() []Collection

func (Schema) ValidateGenesis 

func (s Schema) ValidateGenesis(source appmodule.GenesisSource) error

ValidateGenesis implements the appmodule.HasGenesis.ValidateGenesis method.

type SchemaBuilder 

type SchemaBuilder struct {
	// contains filtered or unexported fields
}

SchemaBuilder is used for building schemas. The Build method should always be called after all collections have been initialized. Initializing new collections with the builder after initialization will result in panics.

func NewSchemaBuilder 

func NewSchemaBuilder(service store.KVStoreService) *SchemaBuilder

NewSchemaBuilder creates a new schema builder from the provided store key. Callers should always call the SchemaBuilder.Build method when they are done adding collections to the schema.

func NewSchemaBuilderFromAccessor 

func NewSchemaBuilderFromAccessor(accessorFunc func(ctx context.Context) store.KVStore) *SchemaBuilder

NewSchemaBuilderFromAccessor creates a new schema builder from the provided store accessor function.

func (*SchemaBuilder) Build 

func (s *SchemaBuilder) Build() (Schema, error)

Build should be called after all collections that are part of the schema have been initialized in order to get a reference to the Schema. It is important to check the returned error for any initialization errors. The SchemaBuilder CANNOT be used after Build is called - doing so will result in panics.

type Sequence 

type Sequence Item[uint64]

Sequence builds on top of an Item, and represents a monotonically increasing number.

func NewSequence 

func NewSequence(schema *SchemaBuilder, prefix Prefix, name string) Sequence

NewSequence instantiates a new sequence given a Schema, a Prefix and humanized name for the sequence.

func (Sequence) Next 

func (s Sequence) Next(ctx context.Context) (uint64, error)

Next returns the next sequence number, and sets the next expected sequence. Errors on encoding issues.

func (Sequence) Peek 

func (s Sequence) Peek(ctx context.Context) (uint64, error)

Peek returns the current sequence value, if no number is set then the DefaultSequenceStart is returned. Errors on encoding issues.

func (Sequence) Set 

func (s Sequence) Set(ctx context.Context, value uint64) error

Set hard resets the sequence to the provided value. Errors on encoding issues.

type Triple added in v0.4.0 

type Triple[K1, K2, K3 any] struct {
	// contains filtered or unexported fields
}

Triple defines a multipart key composed of three keys.

func Join3 added in v0.4.0 

func Join3[K1, K2, K3 any](k1 K1, k2 K2, k3 K3) Triple[K1, K2, K3]

Join3 instantiates a new Triple instance composed of the three provided keys, in order.

func TriplePrefix added in v0.4.0 

func TriplePrefix[K1, K2, K3 any](k1 K1) Triple[K1, K2, K3]

TriplePrefix creates a new Triple instance composed only of the first part of the key.

func TripleSuperPrefix added in v0.4.0 

func TripleSuperPrefix[K1, K2, K3 any](k1 K1, k2 K2) Triple[K1, K2, K3]

TripleSuperPrefix creates a new Triple instance composed only of the first two parts of the key.

func (Triple[K1, K2, K3]) K1 added in v0.4.0 

func (t Triple[K1, K2, K3]) K1() (x K1)

K1 returns the first part of the key. If nil, the zero value is returned.

func (Triple[K1, K2, K3]) K2 added in v0.4.0 

func (t Triple[K1, K2, K3]) K2() (x K2)

K2 returns the second part of the key. If nil, the zero value is returned.

func (Triple[K1, K2, K3]) K3 added in v0.4.0 

func (t Triple[K1, K2, K3]) K3() (x K3)

K3 returns the third part of the key. If nil, the zero value is returned.

Source Files

View all Source files

Directories

Path Synopsis
Package codec defines how collections transform keys and values into and from bytes.
 Package codec defines how collections transform keys and values into and from bytes.
Package indexes contains the most common indexes types to be used with a collections.IndexedMap.
 Package indexes contains the most common indexes types to be used with a collections.IndexedMap.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.