store 包定义了 Cosmos SDK 模块在 Cosmos SDK 应用中,对默克尔化状态进行读取和写入时所使用的接口、类型和抽象。store 包提供了许多基础原语,供开发者同时处理状态存储与状态承诺。下面将介绍这些不同的抽象。
类型
Store
大多数 store 接口定义在这里。其中最基础的原语接口是 Store 类型,其他接口都建立在它之上。Store 接口定义了识别具体 store 实现类型的能力,以及通过 CacheWrapper 接口进行缓存包装的能力。
CacheWrapper 与 CacheWrap
store 最重要的能力之一是执行缓存包装。所谓缓存包装,本质上是底层 store 将自身包装到另一种 store 类型中,由后者为读写都提供缓存能力,并可通过 Write() 刷新写入。
KVStore 与 CacheKVStore
KVStore 是开发者和模块都会直接交互的核心接口之一,也是大多数状态存储与状态承诺操作的基础。KVStore 接口提供基础的 CRUD 能力,以及基于前缀的迭代能力,包括反向迭代。
通常,每个模块都有自己专属的 KVStore 实例,并可通过 sdk.Context 与基于指针的命名 key KVStoreKey 获取访问权限。KVStoreKey 提供一种类似伪 OCAP 的能力。KVStoreKey 究竟如何映射到某个 KVStore,将在下文通过 CommitMultiStore 进行说明。
需要注意,KVStore 本身不能直接提交状态。相反,KVStore 可以被 CacheKVStore 包装。CacheKVStore 扩展了 KVStore,并允许调用方执行 Write(),将待写入内容刷新到内存中的父级 KVStore。但这并不会真正将写入刷到磁盘,因为在 CommitMultiStore 调用 Commit() 之前,写入始终保存在内存中。
CommitMultiStore
CommitMultiStore 接口暴露了顶层接口,SDK 应用通过它管理状态承诺与存储,并对多个模块使用的多个 KVStore 概念进行了抽象。具体来说,它支持以下高级原语:
- 允许调用方通过提供
KVStoreKey获取一个KVStore。 - 提供裁剪机制,用于删除过去某个特定高度或版本上被保留的状态。
- 允许加载过去某个特定高度或版本的状态存储,以支持当前头部查询和历史查询。
- 提供将状态回滚到先前高度或版本的能力。
- 提供在加载特定高度或版本的状态存储时同时执行 store 升级的能力,这通常用于在线硬分叉期间的应用状态迁移。
- 提供将当前累计的全部状态提交到磁盘并执行默克尔承诺的能力。
实现细节
虽然store 包提供了很多接口,但 Cosmos SDK 通常会为模块和开发者实际交互的每个主要接口提供一个核心实现。
iavl.Store
iavl.Store 通过实现以下接口,提供了状态存储与状态承诺的核心实现:
KVStoreCommitStoreCommitKVStoreQueryableStoreWithInitialVersion
iavl.Store 还提供了从状态承诺层移除历史状态的能力。
IAVL 实现概览可见这里。需要特别注意的是,IAVL store 同时提供状态承诺和逻辑存储操作,这也带来了一些缺点:上述各种操作会产生不同程度的性能影响,其中部分影响可能非常明显。
在处理模块和客户端中的状态管理时,Cosmos SDK 提供了多层抽象,也就是“store wrapping”,其中 iavl.Store 位于最底层。当模块请求某个 store 执行读写时,典型的抽象层顺序如下:
IAVL store 的并发使用
iavl.Store 底层的树结构并不是并发安全的。调用方有责任确保不会对该 store 执行并发访问。
并发使用的主要问题在于:当数据正在被迭代时,同时又有写入发生。这样会导致不可恢复的致命错误,因为内部 map 会发生并发读写。
虽然不推荐,但你可以通过禁用 “FastNode” 的方式,在写入的同时进行迭代,不过不能保证迭代过程中一定能返回这些新写入的值(如果你确实需要这种行为,可能需要重新审视应用的设计)。具体做法是在配置 TOML 文件中将 iavl-disable-fastnode 设置为 true。
cachekv.Store
cachekv.Store 会包装底层 KVStore,通常是一个 iavl.Store,并在内存中维护一个缓存,用于保存待写入到底层 KVStore 的内容。Set 和 Delete 调用都作用于内存缓存。Has 会先检查缓存,只有缓存未命中时才会继续访问底层 KVStore。
cachekv.Store 最重要的方法之一是 Write()。它会先对 key 排序,从而确保键值对以确定性且有序的方式写入到底层 KVStore。该 store 会跟踪“dirty” key,并据此决定需要排序哪些 key。删除操作会被表示为零值(nil)条目;Write() 会识别这些条目,并对每个条目在底层 KVStore 上调用 Delete。
cachekv.Store 还支持正向迭代和反向迭代。迭代通过 cacheMergeIterator 类型完成,并同时利用脏缓存和底层 KVStore 对键值对进行遍历。
需要注意,针对 cachekv.Store 的所有 CRUD 和迭代操作调用都是线程安全的。
gaskv.Store
gaskv.Store 为 KVStore 提供了一个简单实现。更具体地说,它只是包装了一个已有的 KVStore,例如经过缓存包装的 iavl.Store,并在构造时接收一个 GasMeter,通过调用 ConsumeGas() 为 CRUD 操作计入可配置的 gas 成本,然后将底层 CRUD 调用代理到被包装的 store。
cachemulti.Store 与 rootmulti.Store
rootmulti.Store 是围绕一组 store 的抽象。具体来说,它实现了 CommitMultiStore 和 Queryable 接口。SDK 模块通过 rootmulti.Store 持有唯一的 KVStoreKey,从而请求访问某个 KVStore,执行状态 CRUD 操作和查询。
rootmulti.Store 会确保这些查询和状态操作通过前文所述的、经过缓存包装的 cachekv.Store 实例执行。rootmulti.Store 的实现还负责将各个 KVStore 中累计的全部状态提交到磁盘,并返回应用状态的默克尔根。
查询不仅可以返回状态数据,还可以返回与之关联的状态承诺证明,既支持过去的高度或版本,也支持当前状态根。查询会根据 store 名称(也就是模块)以及 SDK RequestQuery 类型中定义的其他参数进行路由。
rootmulti.Store 还提供了在给定高度或版本上对状态存储数据进行裁剪的原语。当某个高度被提交时,rootmulti.Store 会根据操作员在 PruningOptions 中定义的裁剪设置,判断是否应将更早的高度纳入删除范围。PruningOptions 定义了磁盘上需要保留多少最近版本,以及以什么间隔将“暂存”的待裁剪高度从磁盘中移除。在每个间隔点,这些暂存高度都会从每个 KVStore 中移除。需要注意,裁剪究竟如何执行,取决于底层 KVStore 的具体实现。PruningOptions 定义如下:
default、everything、nothing 和 custom。
需要特别注意,rootmulti.Store 将每个 KVStore 视为彼此独立的逻辑 store。换言之,它们不会共享同一棵默克尔树或类似的数据结构。这意味着当通过 rootmulti.Store 提交状态时,每个 store 都会按顺序依次提交,因此整个过程不是原子的。
从 store 的构造与装配角度看,每个 Cosmos SDK 应用都包含一个 BaseApp 实例,它内部持有一个 CommitMultiStore 引用,而该接口由 rootmulti.Store 实现。随后,应用会注册一个或多个 KVStoreKey,每个 key 对应一个唯一模块,也因此对应一个 KVStore。通过 sdk.Context 与 KVStoreKey,每个模块都可以直接访问各自对应的 KVStore 实例。
示例:
rootmulti.Store 本身也可以被缓存包装,返回一个 cachemulti.Store 实例。对于每个区块,BaseApp 都会确保在 CommitMultiStore 之上创建正确的抽象层,也就是确保 rootmulti.Store 被缓存包装,并将得到的 cachemulti.Store 设置到 sdk.Context 上,随后用于区块和交易执行。因此,所有因区块和交易执行产生的状态变更,实际上都只是暂时保存在内存中,直到 ABCI 客户端调用 Commit() 为止。这个概念在每笔交易执行 AnteHandler 时会进一步体现,以确保那些 CheckTx 失败的交易不会提交状态。
The store package defines the interfaces, types and abstractions for Cosmos SDK modules to read and write to Merkleized state within a Cosmos SDK application. The store package provides many primitives for developers to use in order to work with both state storage and state commitment. Below we describe the various abstractions.
Types
Store
The bulk of the store interfaces are defined here,
where the base primitive interface, for which other interfaces build off of, is
the Store type. The Store interface defines the ability to tell the type of
the implementing store and the ability to cache wrap via the CacheWrapper interface.
CacheWrapper & CacheWrap
One of the most important features a store has the ability to perform is the
ability to cache wrap. Cache wrapping is essentially the underlying store wrapping
itself within another store type that performs caching for both reads and writes
with the ability to flush writes via Write().
KVStore & CacheKVStore
One of the most important interfaces that both developers and modules interface
with, which also provides the basis of most state storage and commitment operations,
is the KVStore. The KVStore interface provides basic CRUD abilities and
prefix-based iteration, including reverse iteration.
Typically, each module has its own dedicated KVStore instance, which it can
get access to via the sdk.Context and the use of a pointer-based named key —
KVStoreKey. The KVStoreKey provides pseudo-OCAP. How exactly a KVStoreKey
maps to a KVStore will be illustrated below through the CommitMultiStore.
Note, a KVStore cannot directly commit state. Instead, a KVStore can be wrapped
by a CacheKVStore which extends a KVStore and provides the ability for the
caller to execute Write() which flushes pending writes to the parent KVStore in memory.
Note, this doesn’t actually flush writes to disk as writes are held in memory
until Commit() is called on the CommitMultiStore.
CommitMultiStore
The CommitMultiStore interface exposes the top-level interface that is used
to manage state commitment and storage by an SDK application and abstracts the
concept of multiple KVStores which are used by multiple modules. Specifically,
it supports the following high-level primitives:
- Allows for a caller to retrieve a
KVStoreby providing aKVStoreKey. - Exposes pruning mechanisms to remove state pinned against a specific height/version in the past.
- Allows for loading state storage at a particular height/version in the past to provide current head and historical queries.
- Provides the ability to rollback state to a previous height/version.
- Provides the ability to load state storage at a particular height/version while also performing store upgrades, which are used during live hard-fork application state migrations.
- Provides the ability to commit all current accumulated state to disk and performs Merkle commitment.
Implementation Details
While there are many interfaces that thestore package provides, there is
typically a core implementation for each main interface that modules and
developers interact with that are defined in the Cosmos SDK.
iavl.Store
The iavl.Store provides the core implementation for state storage and commitment
by implementing the following interfaces:
KVStoreCommitStoreCommitKVStoreQueryableStoreWithInitialVersion
iavl.Store also provides the ability to remove
historical state from the state commitment layer.
An overview of the IAVL implementation can be found here.
It is important to note that the IAVL store provides both state commitment and
logical storage operations, which comes with drawbacks as there are various
performance impacts, some of which are very drastic, when it comes to the
operations mentioned above.
When dealing with state management in modules and clients, the Cosmos SDK provides
various layers of abstractions or “store wrapping”, where the iavl.Store is the
bottom most layer. When requesting a store to perform reads or writes in a module,
the typical abstraction layer in order is defined as follows:
Concurrent use of IAVL store
The tree underiavl.Store is not safe for concurrent use. It is the
responsibility of the caller to ensure that concurrent access to the store is
not performed.
The main issue with concurrent use is when data is written at the same time as
it’s being iterated over. Doing so will cause an irrecoverable fatal error because
of concurrent reads and writes to an internal map.
Although it’s not recommended, you can iterate through values while writing to
it by disabling “FastNode” without guarantees that the values being written will
be returned during the iteration (if you need this, you might want to reconsider
the design of your application). This is done by setting iavl-disable-fastnode
to true in the config TOML file.
cachekv.Store
The cachekv.Store store wraps an underlying KVStore, typically a iavl.Store
and contains an in-memory cache for storing pending writes to underlying KVStore.
Set and Delete calls are executed on the in-memory cache. Has checks the cache first, falling through to the underlying KVStore only on a cache miss.
One of the most important calls to a cachekv.Store is Write(), which ensures
that key-value pairs are written to the underlying KVStore in a deterministic
and ordered manner by sorting the keys first. The store keeps track of “dirty”
keys and uses these to determine what keys to sort. Deletions are represented as zero-value (nil) entries; Write() detects these and calls Delete on the underlying KVStore for each one.
The cachekv.Store also provides the ability to perform iteration and reverse
iteration. Iteration is performed through the cacheMergeIterator type and uses
both the dirty cache and underlying KVStore to iterate over key-value pairs.
Note, all calls to CRUD and iteration operations on a cachekv.Store are thread-safe.
gaskv.Store
The gaskv.Store store provides a simple implementation of a KVStore.
Specifically, it just wraps an existing KVStore, such as a cache-wrapped
iavl.Store, and incurs configurable gas costs for CRUD operations via
ConsumeGas() calls on a GasMeter passed at construction time, then proxies the underlying CRUD call to the wrapped store.
cachemulti.Store & rootmulti.Store
The rootmulti.Store acts as an abstraction around a series of stores. Namely,
it implements the CommitMultiStore an Queryable interfaces. Through the
rootmulti.Store, an SDK module can request access to a KVStore to perform
state CRUD operations and queries by holding access to a unique KVStoreKey.
The rootmulti.Store ensures these queries and state operations are performed
through cached-wrapped instances of cachekv.Store which is described above. The
rootmulti.Store implementation is also responsible for committing all accumulated
state from each KVStore to disk and returning an application state Merkle root.
Queries can be performed to return state data along with associated state
commitment proofs for both previous heights/versions and the current state root.
Queries are routed based on store name, i.e. a module, along with other parameters defined in the SDK’s RequestQuery type.
The rootmulti.Store also provides primitives for pruning data at a given
height/version from state storage. When a height is committed, the rootmulti.Store
will determine if other previous heights should be considered for removal based
on the operator’s pruning settings defined by PruningOptions, which defines
how many recent versions to keep on disk and the interval at which to remove
“staged” pruned heights from disk. During each interval, the staged heights are
removed from each KVStore. Note, it is up to the underlying KVStore
implementation to determine how pruning is actually performed. The PruningOptions
are defined as follows:
default, everything, nothing, and custom.
It is important to note that the rootmulti.Store considers each KVStore as a
separate logical store. In other words, they do not share a Merkle tree or
comparable data structure. This means that when state is committed via
rootmulti.Store, each store is committed in sequence and thus is not atomic.
In terms of store construction and wiring, each Cosmos SDK application contains
a BaseApp instance which internally has a reference to a CommitMultiStore
that is implemented by a rootmulti.Store. The application then registers one or
more KVStoreKey that pertain to a unique module and thus a KVStore. Through
the use of an sdk.Context and a KVStoreKey, each module can get direct access
to it’s respective KVStore instance.
Example:
rootmulti.Store itself can be cache-wrapped which returns an instance of a
cachemulti.Store. For each block, BaseApp ensures that the proper abstractions
are created on the CommitMultiStore, i.e. ensuring that the rootmulti.Store
is cached-wrapped and uses the resulting cachemulti.Store to be set on the
sdk.Context which is then used for block and transaction execution. As a result,
all state mutations due to block and transaction execution are actually held
ephemerally until Commit() is called by the ABCI client. This concept is further
expanded upon when the AnteHandler is executed per transaction to ensure state
is not committed for transactions that failed CheckTx.