var ( // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data // - 0x00<allowance_key_bytes>: allowance FeeAllowanceKeyPrefix = []byte{0x00} // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data // - 0x01<allowance_prefix_queue_key_bytes>: <empty value> FeeAllowanceQueueKeyPrefix = []byte{0x01})
现在会变成：
var ( // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data // - 0x00<allowance_key_bytes>: allowance FeeAllowanceKeyPrefix = collections.NewPrefix(0) // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data // - 0x01<allowance_prefix_queue_key_bytes>: <empty value> FeeAllowanceQueueKeyPrefix = collections.NewPrefix(1))
上面的示例展示了如何创建一个 AltValueCodec，使其既能解码 sdk.Int 值，也能解码 sdk.Coin 值。所提供的
解码函数会在默认解码器失败时作为回退使用。当该值再次被编码回状态中时，
它会使用默认编码器。这样就可以将值惰性迁移到新的字节表示形式。Collections is a library meant to simplify the experience with respect to module state handling.Cosmos SDK modules handle their state using the KVStore interface. The problem with working with
KVStore is that it forces you to think of state as a bytes KV pairings when in reality the majority of
state comes from complex concrete golang objects (strings, ints, structs, etc.).Collections allows you to work with state as if they were normal golang objects and removes the need
for you to think of your state as raw bytes in your code.It also allows you to migrate your existing state without causing any state breakage that forces you into
tedious and complex chain state migrations.
Before exploring the different collections types and their capability it is necessary to introduce
the three components that every collection shares. In fact when instantiating a collection type by doing, for example,
collections.NewMap/collections.NewItem/... you will find yourself having to pass them some common arguments.For example, in code:
The first argument passed is the SchemaBuilderSchemaBuilder is a structure that keeps track of all the state of a module, it is not required by the collections
to deal with state but it offers a dynamic and reflective way for clients to explore a module’s state.We instantiate a SchemaBuilder by passing it a store.KVStoreService, which is the module’s store service obtained via dependency injection or runtime.NewKVStoreService.We then need to pass the schema builder to every collection type we instantiate in our keeper, in our case the AllowList.After creating all collections, call sb.Build() to validate prefix uniqueness and finalize the schema. Store the returned collections.Schema in the keeper’s Schema field:
k := Keeper{ AllowList: collections.NewKeySet(sb, AllowListPrefix, "allow_list", collections.StringKey),}schema, err := sb.Build()if err != nil { panic(err)}k.Schema = schemareturn k
The code examples in this document show the collection instantiation patterns but omit the sb.Build() call for brevity. In production code, sb.Build() is required.
The second argument passed to our KeySet is a collections.Prefix, a prefix represents a partition of the module’s KVStore
where all the state of a specific collection will be saved.Since a module can have multiple collections, the following is expected:
module params will become a collections.Item
the AllowList is a collections.KeySet
We don’t want a collection to write over the state of the other collection so we pass it a prefix, which defines a storage
partition owned by the collection.If you already built modules, the prefix translates to the items you were creating in your types/keys.go file, example: Linkyour old:
var ( // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data // - 0x00<allowance_key_bytes>: allowance FeeAllowanceKeyPrefix = []byte{0x00} // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data // - 0x01<allowance_prefix_queue_key_bytes>: <empty value> FeeAllowanceQueueKeyPrefix = []byte{0x01})
becomes:
var ( // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data // - 0x00<allowance_key_bytes>: allowance FeeAllowanceKeyPrefix = collections.NewPrefix(0) // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data // - 0x01<allowance_prefix_queue_key_bytes>: <empty value> FeeAllowanceQueueKeyPrefix = collections.NewPrefix(1))
collections.NewPrefix accepts either int, string or []byte. It is good practice to use a monotonically increasing int (values 0–255) for disk space efficiency.A collection MUST NOT share the same prefix as another collection in the same module, and a collection prefix MUST NEVER start with the same prefix as another, examples:
prefix1 := collections.NewPrefix("prefix")prefix2 := collections.NewPrefix("prefix") // THIS IS BAD!
prefix1 := collections.NewPrefix("a")prefix2 := collections.NewPrefix("aa") // prefix2 starts with the same as prefix1: BAD!!!
The third parameter we pass to a collection is a string, which is a human-readable name.
It is needed to make the role of a collection understandable by clients who have no clue about
what a module is storing in state.
A collection is generic over the type you can use as keys or values.
This makes collections dumb, but also means that hypothetically we can store everything
that can be a go type into a collection. We are not bounded to any type of encoding (be it proto, json or whatever)So a collection needs to be given a way to understand how to convert your keys and values to bytes.
This is achieved through KeyCodec and ValueCodec, which are arguments that you pass to your
collections when you’re instantiating them using the collections.NewMap/collections.NewItem/...
instantiation functions.NOTE: Generally speaking you will never be required to implement your own Key/ValueCodec as
the SDK and collections libraries already come with default, safe and fast implementation of those.
You might need to implement them only if you’re migrating to collections and there are state layout incompatibilities.Let’s explore an example:
We’re now instantiating a map where the key is string and the value is uint64.
We already know the first three arguments of the NewMap function.The fourth parameter is our KeyCodec, we know that the Map has string as key so we pass it a KeyCodec that handles strings as keys.The fifth parameter is our ValueCodec, we know that the Map has a uint64 as value so we pass it a ValueCodec that handles uint64.Collections already comes with all the required implementations for golang primitive types.Let’s make another example, this falls closer to what we build using cosmos SDK, let’s say we want
to create a collections.Map that maps account addresses to their base account. So we want to map an sdk.AccAddress to an auth.BaseAccount (which is a proto):
As we can see here since our collections.Map maps sdk.AccAddress to authtypes.BaseAccount,
we use the sdk.AccAddressKey which is the KeyCodec implementation for AccAddress and we use codec.CollValue to
encode our proto type BaseAccount.Generally speaking you will always find the respective key and value codecs for types in the go.mod path you’re using
to import that type. If you want to encode proto values refer to the codec codec.CollValue function, which allows you
to encode any type implement the proto.Message interface.
Set maps with the provided AccAddress (the key) to the auth.BaseAccount (the value).Under the hood the collections.Map will convert the key and value to bytes using the key and value codec.
It will prepend to our bytes key the prefix and store it in the KVStore of the module.
The remove method accepts the AccAddress and removes it from the store. It won’t report errors
if it does not exist, to check for existence before removal use the Has method.
A collections.KeySet is just a collections.Map with a key but no value.
The value internally is always the same and is represented as an empty byte slice []byte{}.
The first difference we notice is that KeySet needs use to specify only one type parameter: the key (sdk.ValAddress in this case).
The second difference we notice is that KeySet in its NewKeySet function does not require
us to specify a ValueCodec but only a KeyCodec. This is because a KeySet only saves keys and not values.Let’s explore the methods.
Remove removes the provided key from the KeySet, it does not error if the key does not exist,
if existence check before removal is required it needs to be coupled with the Has method.
The third type of collection is the collections.Item.
It stores only one single item, it’s useful for example for parameters, there’s only one instance
of parameters in state always.
The first key difference we notice is that we specify only one type parameter, which is the value we’re storing.
The second key difference is that we don’t specify the KeyCodec, since we store only one item we already know the key
and the fact that it is constant.
One of the key features of the KVStore is iterating over keys.Collections which deal with keys (so Map, KeySet and IndexedMap) allow you to iterate
over keys in a safe and typed way. They all share the same API, the only difference being
that KeySet returns a different type of Iterator because KeySet only deals with keys.
Every collection shares the same Iterator semantics.
It accepts a collections.Ranger[K], which is an API that instructs map on how to iterate over keys.
As always we don’t need to implement anything here as collections already provides some generic Ranger implementers
that expose all you need to work with ranges.
In GetAllAccounts we pass to our Iterate a nil Ranger. This means that the returned Iterator will include
all the existing keys within the collection.Then we use the Values method from the returned Iterator API to collect all the values into a slice.Iterator offers other methods such as Keys() to collect only the keys and not the values and KeyValues to collect
all the keys and values.
Here we make use of the collections.Range helper to specialize our range.
We make it start in a point through StartInclusive and end in the other with EndExclusive, then
we instruct it to report us results in reverse order through DescendingThen we pass the range instruction to Iterate and get an Iterator, which will contain only the results
we specified in the range.Then we use again the Values method of the Iterator to collect all the results.collections.Range also offers a Prefix API which is not applicable to all keys types,
for example uint64 cannot be prefix because it is of constant size, but a string key
can be prefixed.
So far we’ve worked only with simple keys, like uint64, the account address, etc.
There are some more complex cases in, which we need to deal with composite keys.A key is composite when it is composed of multiple keys, for example bank balances as stored as the composite key
(AccAddress, string) where the first part is the address holding the coins and the second part is the denom.Example, let’s say address BOB holds 10atom,15osmo, this is how it is stored in state:
(bob, atom) => 10(bob, osmos) => 15
Now this allows to efficiently get a specific denom balance of an address, by simply getting(address, denom), or getting all the balances
of an address by prefixing over (address).Let’s see now how we can work with composite keys using collections.
In our example we will showcase how we can use collections when we are dealing with balances, similar to bank,
a balance is a mapping between (address, denom) => math.Int the composite key in our case is (address, denom).
The arguments to instantiate are always the same, the only thing that changes is how we instantiate
the KeyCodec, since this key is composed of two keys we use collections.PairKeyCodec, which generates
a KeyCodec composed of two key codecs. The first one will encode the first part of the key, the second one will
encode the second part of the key.
As we can see here we’re setting the balance of an address for a specific denom.
We use the collections.Join function to generate the composite key.
collections.Join returns a collections.Pair (which is the key of our collections.Map)collections.Pair contains the two keys we have joined, it also exposes two methods: K1 to fetch the 1st part of the
key and K2 to fetch the second part.As always, we use the collections.Map.Set method to map the composite key to our value (math.Int in this case)
We use collections.PrefixedPairRange to iterate over all the keys starting with the provided address.
Concretely the iteration will report all the balances belonging to the provided address.The first part is that we instantiate a PrefixedPairRange, which is a Ranger implementer aimed to help
in Pair keys iterations.
As we can see here we’re passing the type parameters of the collections.Pair because golang type inference
with respect to generics is not as permissive as other languages, so we need to explicitly say what are the types of the pair key.
This showcases how we can further specialize our range to limit the results further, by specifying
the range between the second part of the key (in our case the denoms, which are strings).
collections.IndexedMap is a collection that uses under the hood a collections.Map, and has a struct, which contains the indexes that we need to define.
First of all, when we save our accounts in state we map them using a primary key sdk.AccAddress.
If it were to be a collections.Map it would be collections.Map[sdk.AccAddress, authtypes.BaseAccount].Then we also want to be able to get an account not only by its sdk.AccAddress, but also by its AccountNumber.So we can say we want to create an Index that maps our BaseAccount to its AccountNumber.We also know that this Index is unique. Unique means that there can only be one BaseAccount that maps to a specific
AccountNumber.First of all, we start by defining the object that contains our index:
We create an AccountIndexes struct which contains a field: Number. This field represents our AccountNumber index.
AccountNumber is a field of authtypes.BaseAccount and it’s a uint64.Then we can see in our AccountIndexes struct the Number field is defined as:
Where the first type parameter is uint64, which is the field type of our index.
The second type parameter is the primary key sdk.AccAddress.
And the third type parameter is the actual object we’re storing authtypes.BaseAccount.Then we create a NewAccountIndexes function that instantiates and returns the AccountsIndexes struct.The function takes a SchemaBuilder. Then we instantiate our indexes.Unique, let’s analyze the arguments we pass to
indexes.NewUnique.
The AccountsIndexes struct contains the indexes, the NewIndexedMap function will infer the indexes form that struct
using reflection, this happens only at init and is not computationally expensive. In case you want to explicitly declare
indexes: implement the Indexes interface in the AccountsIndexes struct:
func (a AccountsIndexes)IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] { return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{ a.Number}}
The first three arguments, we already know them, they are: SchemaBuilder, Prefix which is our index prefix (the partition
where index keys relationship for the Number index will be maintained), and the human name for the Number index.The second argument is a collections.Uint64Key which is a key codec to deal with uint64 keys, we pass that because
the key we’re trying to index is a uint64 key (the account number), and then we pass as fifth argument the primary key codec,
which in our case is sdk.AccAddress (remember: we’re mapping sdk.AccAddress => BaseAccount).Then as last parameter we pass a function that: given the BaseAccount returns its AccountNumber.After this we can proceed instantiating our IndexedMap.
As we can see here what we do, for now, is the same thing as we did for collections.Map.
We pass it the SchemaBuilder, the Prefix where we plan to store the mapping between sdk.AccAddress and authtypes.BaseAccount,
the human name and the respective sdk.AccAddress key codec and authtypes.BaseAccount value codec.Then we pass the instantiation of our AccountIndexes through NewAccountIndexes.Full example:
While instantiating collections.IndexedMap is tedious, working with them is extremely smooth.Let’s take the full example, and expand it with some use-cases.
Although cosmos-sdk is shifting away from the usage of interface registry, there are still some places where it is used.
In order to support old code, we have to support collections with interface values.The generic codec.CollValue is not able to handle interface values, so we need to use a special type codec.CollInterfaceValue.
codec.CollInterfaceValue takes a codec.BinaryCodec as an argument, and uses it to marshal and unmarshal values as interfaces.
The codec.CollInterfaceValue lives in the codec package, whose import path is github.com/cosmos/cosmos-sdk/codec.
The codec.AltValueCodec allows a collection to decode values using a different codec than the one used to encode them.
Basically it enables to decode two different byte representations of the same concrete value.
It can be used to lazily migrate values from one bytes representation to another, as long as the new representation is
not able to decode the old one.A concrete example can be found in x/bank where the balance was initially stored as Coin and then migrated to Int.
The above example shows how to create an AltValueCodec that can decode both sdk.Int and sdk.Coin values. The provided
decoder function will be used as a fallback in case the default decoder fails. When the value will be encoded back into state
it will use the default encoder. This allows to lazily migrate values to a new bytes representation.
