​
变更记录

  • 2022-08-10: 初始草案 (@alexanderbez, @tac0turtle)
  • 2022 年 11 月 12 日：根据初始实现 PR 更新 PrepareProposal 和 ProcessProposal 的语义 (@alexanderbez)

​
状态

已接受

​
摘要

本 ADR 描述了在 Cosmos SDK 中对 ABCI 1.0 的初始采用。 ABCI 1.0 是 ABCI 的下一次演进，旨在为应用开发者提供更高的灵活性，以及对应用与共识语义更强的控制力，例如应用内 mempool、进程内预言机，以及订单簿式撮合引擎。

​
背景

Tendermint 将发布 ABCI 1.0。特别是，在撰写本文时，Tendermint 正在发布 v0.37.0，其中将包含 PrepareProposal 和 ProcessProposal。 PrepareProposal 这个 ABCI 方法关注的是区块提议者请求应用评估一组待纳入下一个区块的交易，这组交易被定义为 TxRecord 对象切片。应用可以接受、拒绝，或完全忽略其中部分或全部交易。这一点非常重要，因为应用本质上可以定义并控制自己的 mempool，从而定义复杂的交易优先级与过滤机制；它甚至可以完全忽略 Tendermint 发送过来的 TxRecords，转而优先使用自己的交易。这实际上意味着 Tendermint 的 mempool 更像是一种 gossip 数据结构。 第二个 ABCI 方法 ProcessProposal 用于处理由 PrepareProposal 定义的区块提议。关于 ProcessProposal，需要注意以下几点：
  • ProcessProposal 的执行必须是确定性的。
  • PrepareProposal 与 ProcessProposal 之间必须保持一致性。换句话说，对于任意两个正确进程 p 和 q，如果 q 的 Tendermint 对 up 调用 RequestProcessProposal，那么 q 的应用必须在 ResponseProcessProposal 中返回 ACCEPT。
需要注意的是，在 ABCI 1.0 集成中，应用不负责锁定语义，这仍然由 Tendermint 负责。不过在未来，应用将负责锁定，从而为并行执行提供可能性。

​
决策

我们将在 Cosmos SDK 的下一个主版本中集成 ABCI 1.0，而它将随 Tendermint v0.37.0 引入。我们会在 BaseApp 类型上集成 ABCI 1.0 方法。下面分别描述这两个方法的实现。 在介绍这两个新方法的实现之前，需要说明，现有的 ABCI 方法，如 CheckTx、DeliverTx 等，仍然存在，并继续承担它们当前的职责。

​
PrepareProposal

在评估如何实现 PrepareProposal 之前，需要先说明：CheckTx 仍然会执行，并像现在一样负责评估交易有效性，但有一个非常重要的增量性区别。 在 CheckTx 中执行交易时，应用现在会将有效交易，也就是通过 AnteHandler 的交易，加入其自身的 mempool 数据结构。为了提供一种灵活的方式来满足不同应用开发者的需求，我们将定义一个 mempool 接口以及一个利用 Golang 泛型的数据结构，使开发者只需要关注交易排序。对于需要绝对完全控制的开发者，则可以实现自定义的 mempool。 我们将一般性的 mempool 接口定义如下（后续可能调整）： 
type Mempool interface {
	// Insert attempts to insert a Tx into the app-side mempool returning
	// an error upon failure.
	Insert(sdk.Context, sdk.Tx)

error

	// Select returns an Iterator over the app-side mempool. If txs are specified,
	// then they shall be incorporated into the Iterator. The Iterator must
	// closed by the caller.
	Select(sdk.Context, [][]byte)

Iterator

	// CountTx returns the number of transactions currently in the mempool.
	CountTx()

int

	// Remove attempts to remove a transaction from the mempool, returning an error
	// upon failure.
	Remove(sdk.Tx)

error
}

// Iterator defines an app-side mempool iterator interface that is as minimal as
// possible. The order of iteration is determined by the app-side mempool
// implementation.
type Iterator interface {
	// Next returns the next transaction from the mempool. If there are no more
	// transactions, it returns nil.
	Next()

Iterator

	// Tx returns the transaction at the current position of the iterator.
	Tx()

sdk.Tx
}
我们将定义一个 Mempool 的实现，名为 nonceMempool，它将覆盖大多数基础应用场景。具体来说，它会按交易发送者对交易进行优先级排序，并允许同一发送者提交多笔交易。 默认的应用侧 mempool 实现 nonceMempool 将基于单个跳表数据结构运行。具体来说，全局 nonce 更低的交易优先级更高；若 nonce 相同，则按发送者地址排序。 
type nonceMempool struct {
    txQueue *huandu.SkipList
}
此前的讨论1 已达成一致：Tendermint 将通过 RequestPrepareProposal 向应用发起请求，请求中带有从 Tendermint 本地 mempool 收割出的若干交易。具体收割多少交易，将由本地操作员配置决定。这在讨论中被称为“one-shot approach”。 当 Tendermint 从本地 mempool 收割交易并通过 RequestPrepareProposal 发送给应用时，应用需要对这些交易进行评估。具体来说，它需要告知 Tendermint 对每笔交易是拒绝还是纳入。注意，应用甚至可以用其他交易替换这些交易。 在评估 RequestPrepareProposal 中的交易时，应用将忽略请求中发送给它的所有交易，转而从自己的 mempool 中收割最多 RequestPrepareProposal.max_tx_bytes 的交易。 由于应用在 CheckTx 执行期间，理论上可以在 Insert 时插入或注入交易，因此建议应用在 PrepareProposal 收割交易时确保这些交易的有效性。不过，“有效性”具体意味着什么，完全由应用自行决定。 Cosmos SDK 将提供一个默认的 PrepareProposal 实现，它只会选择最多 MaxBytes 的有效交易。 不过，应用也可以使用自己的实现覆盖这一默认行为，并通过 SetPrepareProposal 将其设置到 BaseApp 上。

​
ProcessProposal

ProcessProposal 这个 ABCI 方法相对直接。它负责确保提议区块的有效性，而该区块包含的是在 PrepareProposal 步骤中选出的交易。不过，应用如何判定一个提议区块是否有效，取决于应用本身及其具体用例。对于大多数应用，直接调用 AnteHandler 链就足够了；但也完全可能存在需要对提议区块校验过程拥有更强控制力的应用，例如确保交易按某种顺序排列，或者确保某些交易必须被包含。理论上，这可以通过自定义 AnteHandler 实现来完成，但这既不是最清晰的用户体验，也不是最高效的方案。 因此，我们将在现有 Application 接口上定义一个额外的 ABCI 接口方法，类似于现有的 BeginBlock 或 EndBlock 等 ABCI 方法。这个新接口方法定义如下： 
ProcessProposal(sdk.Context, abci.RequestProcessProposal)

error {
}
注意，我们必须在 Context 参数上使用一个新的内部分支状态来调用 ProcessProposal，因为此时不能直接复用已有的 checkState，因为 BaseApp 在此时已经拥有一个被修改过的 checkState。因此，在执行 ProcessProposal 时，我们会基于 deliverState 创建一个类似的分支状态 processProposalState。需要注意的是，processProposalState 永远不会被提交，并会在 ProcessProposal 执行结束后被完全丢弃。 Cosmos SDK 将提供 ProcessProposal 的默认实现，其中所有交易都会通过 CheckTx 流程，也就是 AnteHandler，进行校验；除非某笔交易无法解码，否则始终返回 ACCEPT。

​
DeliverTx

由于在 PrepareProposal 期间，交易并不会真正从应用侧 mempool 中移除，因为 ProcessProposal 可能失败或经历多轮处理，而我们不希望丢失交易，所以我们需要在 DeliverTx 阶段最终将交易从应用侧 mempool 中移除，因为在这一阶段，这些交易正在被纳入提议区块。 另一种方式是在 PrepareProposal 的收割阶段就真正移除这些交易，并在 ProcessProposal 失败时再将它们重新加入应用侧 mempool。

​
影响

​
向后兼容性

ABCI 1.0 与 Cosmos SDK 和 Tendermint 的早期版本天然不向后兼容。例如，一个发起 RequestPrepareProposal 请求的应用，如果对接到一个不支持 ABCI 1.0 的同类应用，自然会失败。 不过，在集成的第一阶段中，我们今天所熟知的现有 ABCI 方法仍将保留，并继续按当前方式运行。

​
正面影响

  • 应用现在可以完全控制交易排序和优先级。
  • 为 ABCI 1.0 的完整集成打下基础，这将为围绕区块构建以及与 Tendermint 共识引擎集成的应用侧用例解锁更多可能性。

​
负面影响

  • 要求作为通用数据结构、用于收集和存储未提交交易的 “mempool”，在 Tendermint 和 Cosmos SDK 两侧都各自保留一份副本。
  • 在区块执行上下文中，Tendermint 与 Cosmos SDK 之间会增加额外请求。不过，这部分开销应当可以忽略不计。
  • 与 Tendermint 和 Cosmos SDK 的旧版本不向后兼容。

​
后续讨论

应用侧 Mempool[T MempoolTx] 的实现可以采用许多不同的设计方式、数据结构和实现方案，它们各自都有不同的权衡。当前提议的方案保持了简单性，并覆盖了大多数基础应用所需要的场景。与此同时，也仍然可以通过不同权衡来提升所提供 mempool 实现的收割和插入性能。

​
参考资料

​
Changelog

  • 2022-08-10: Initial Draft (@alexanderbez, @tac0turtle)
  • Nov 12, 2022: Update PrepareProposal and ProcessProposal semantics per the initial implementation PR (@alexanderbez)

​
Status

ACCEPTED

​
Abstract

This ADR describes the initial adoption of ABCI 1.0, the next evolution of ABCI, within the Cosmos SDK. ABCI 1.0 aims to provide application developers with more flexibility and control over application and consensus semantics, e.g. in-application mempools, in-process oracles, and order-book style matching engines.

​
Context

Tendermint will release ABCI 1.0. Notably, at the time of this writing, Tendermint is releasing v0.37.0 which will include PrepareProposal and ProcessProposal. The PrepareProposal ABCI method is concerned with a block proposer requesting the application to evaluate a series of transactions to be included in the next block, defined as a slice of TxRecord objects. The application can either accept, reject, or completely ignore some or all of these transactions. This is an important consideration to make as the application can essentially define and control its own mempool allowing it to define sophisticated transaction priority and filtering mechanisms, by completely ignoring the TxRecords Tendermint sends it, favoring its own transactions. This essentially means that the Tendermint mempool acts more like a gossip data structure. The second ABCI method, ProcessProposal, is used to process the block proposer’s proposal as defined by PrepareProposal. It is important to note the following with respect to ProcessProposal:
  • Execution of ProcessProposal must be deterministic.
  • There must be coherence between PrepareProposal and ProcessProposal. In other words, for any two correct processes p and q, if q’s Tendermint calls RequestProcessProposal on up, q’s Application returns ACCEPT in ResponseProcessProposal.
It is important to note that in ABCI 1.0 integration, the application is NOT responsible for locking semantics — Tendermint will still be responsible for that. In the future, however, the application will be responsible for locking, which allows for parallel execution possibilities.

​
Decision

We will integrate ABCI 1.0, which will be introduced in Tendermint v0.37.0, in the next major release of the Cosmos SDK. We will integrate ABCI 1.0 methods on the BaseApp type. We describe the implementations of the two methods individually below. Prior to describing the implementation of the two new methods, it is important to note that the existing ABCI methods, CheckTx, DeliverTx, etc, still exist and serve the same functions as they do now.

​
PrepareProposal

Prior to evaluating the decision for how to implement PrepareProposal, it is important to note that CheckTx will still be executed and will be responsible for evaluating transaction validity as it does now, with one very important additive distinction. When executing transactions in CheckTx, the application will now add valid transactions, i.e. passing the AnteHandler, to its own mempool data structure. In order to provide a flexible approach to meet the varying needs of application developers, we will define both a mempool interface and a data structure utilizing Golang generics, allowing developers to focus only on transaction ordering. Developers requiring absolute full control can implement their own custom mempool implementation. We define the general mempool interface as follows (subject to change): 
type Mempool interface {
	// Insert attempts to insert a Tx into the app-side mempool returning
	// an error upon failure.
	Insert(sdk.Context, sdk.Tx)

error

	// Select returns an Iterator over the app-side mempool. If txs are specified,
	// then they shall be incorporated into the Iterator. The Iterator must
	// closed by the caller.
	Select(sdk.Context, [][]byte)

Iterator

	// CountTx returns the number of transactions currently in the mempool.
	CountTx()

int

	// Remove attempts to remove a transaction from the mempool, returning an error
	// upon failure.
	Remove(sdk.Tx)

error
}

// Iterator defines an app-side mempool iterator interface that is as minimal as
// possible. The order of iteration is determined by the app-side mempool
// implementation.
type Iterator interface {
	// Next returns the next transaction from the mempool. If there are no more
	// transactions, it returns nil.
	Next()

Iterator

	// Tx returns the transaction at the current position of the iterator.
	Tx()

sdk.Tx
}
We will define an implementation of Mempool, defined by nonceMempool, that will cover most basic application use-cases. Namely, it will prioritize transactions by transaction sender, allowing for multiple transactions from the same sender. The default app-side mempool implementation, nonceMempool, will operate on a single skip list data structure. Specifically, transactions with the lowest nonce globally are prioritized. Transactions with the same nonce are prioritized by sender address. 
type nonceMempool struct {
    txQueue *huandu.SkipList
}
Previous discussions1 have come to the agreement that Tendermint will perform a request to the application, via RequestPrepareProposal, with a certain amount of transactions reaped from Tendermint’s local mempool. The exact amount of transactions reaped will be determined by a local operator configuration. This is referred to as the “one-shot approach” seen in discussions. When Tendermint reaps transactions from the local mempool and sends them to the application via RequestPrepareProposal, the application will have to evaluate the transactions. Specifically, it will need to inform Tendermint if it should reject and or include each transaction. Note, the application can even replace transactions entirely with other transactions. When evaluating transactions from RequestPrepareProposal, the application will ignore ALL transactions sent to it in the request and instead reap up to RequestPrepareProposal.max_tx_bytes from its own mempool. Since an application can technically insert or inject transactions on Insert during CheckTx execution, it is recommended that applications ensure transaction validity when reaping transactions during PrepareProposal. However, what validity exactly means is entirely determined by the application. The Cosmos SDK will provide a default PrepareProposal implementation that simply select up to MaxBytes valid transactions. However, applications can override this default implementation with their own implementation and set that on BaseApp via SetPrepareProposal.

​
ProcessProposal

The ProcessProposal ABCI method is relatively straightforward. It is responsible for ensuring validity of the proposed block containing transactions that were selected from the PrepareProposal step. However, how an application determines validity of a proposed block depends on the application and its varying use cases. For most applications, simply calling the AnteHandler chain would suffice, but there could easily be other applications that need more control over the validation process of the proposed block, such as ensuring txs are in a certain order or that certain transactions are included. While this theoretically could be achieved with a custom AnteHandler implementation, it’s not the cleanest UX or the most efficient solution. Instead, we will define an additional ABCI interface method on the existing Application interface, similar to the existing ABCI methods such as BeginBlock or EndBlock. This new interface method will be defined as follows: 
ProcessProposal(sdk.Context, abci.RequestProcessProposal)

error {
}
Note, we must call ProcessProposal with a new internal branched state on the Context argument as we cannot simply just use the existing checkState because BaseApp already has a modified checkState at this point. So when executing ProcessProposal, we create a similar branched state, processProposalState, off of deliverState. Note, the processProposalState is never committed and is completely discarded after ProcessProposal finishes execution. The Cosmos SDK will provide a default implementation of ProcessProposal in which all transactions are validated using the CheckTx flow, i.e. the AnteHandler, and will always return ACCEPT unless any transaction cannot be decoded.

​
DeliverTx

Since transactions are not truly removed from the app-side mempool during PrepareProposal, since ProcessProposal can fail or take multiple rounds and we do not want to lose transactions, we need to finally remove the transaction from the app-side mempool during DeliverTx since during this phase, the transactions are being included in the proposed block. Alternatively, we can keep the transactions as truly being removed during the reaping phase in PrepareProposal and add them back to the app-side mempool in case ProcessProposal fails.

​
Consequences

​
Backwards Compatibility

ABCI 1.0 is naturally not backwards compatible with prior versions of the Cosmos SDK and Tendermint. For example, an application that requests RequestPrepareProposal to the same application that does not speak ABCI 1.0 will naturally fail. However, in the first phase of the integration, the existing ABCI methods as we know them today will still exist and function as they currently do.

​
Positive

  • Applications now have full control over transaction ordering and priority.
  • Lays the groundwork for the full integration of ABCI 1.0, which will unlock more app-side use cases around block construction and integration with the Tendermint consensus engine.

​
Negative

  • Requires that the “mempool”, as a general data structure that collects and stores uncommitted transactions will be duplicated between both Tendermint and the Cosmos SDK.
  • Additional requests between Tendermint and the Cosmos SDK in the context of block execution. Albeit, the overhead should be negligible.
  • Not backwards compatible with previous versions of Tendermint and the Cosmos SDK.

​
Further Discussions

It is possible to design the app-side implementation of the Mempool[T MempoolTx] in many different ways using different data structures and implementations. All of which have different tradeoffs. The proposed solution keeps things simple and covers cases that would be required for most basic applications. There are tradeoffs that can be made to improve performance of reaping and inserting into the provided mempool implementation.

​
References