指南前提
本指南面向希望从零开始构建 CometBFT 应用的初学者。 默认你此前没有任何 CometBFT 使用经验。 CometBFT 是一个为状态机复制提供拜占庭容错共识引擎的服务。 这个被复制的状态机,也就是“应用”,可以使用任何支持在客户端-服务器模型中发送和接收 protocol buffer 消息的语言来编写。 使用 Go 编写的应用还可以将 CometBFT 作为库来使用,并让该服务与应用运行在同一个进程中。 跟随本教程,你将创建一个名为 kvstore 的 CometBFT 应用, 它是一个(非常)简单的分布式 BFT 键值存储。 该应用将使用 Go 编写,因此默认你对 Go 编程语言有一定了解。 如果你从未写过 Go,建议先阅读 用 Y 分钟学会 X:这里的 X=Go,先熟悉一下语法。 注意:请使用本指南和 CometBFT 的最新正式发布版本。 我们强烈不建议在开发中使用未发布的提交版本。内置应用与外部应用
一方面,如果你的应用是用 Go 编写的,那么为了获得最高性能, 你可以让应用与 CometBFT 运行在同一个进程中。 Cosmos SDK 就是以这种方式编写的。 本教程采用的就是这种方式。 另一方面,将应用独立为单独进程可能会带来更好的安全性保障, 因为两个进程之间通过既定的二进制协议通信。 CometBFT 将无法直接访问应用状态。 如果你希望采用这种方式,请改为参考 用 Go 创建应用 指南,而不是本文。1.1 安装 Go
请确认你已经安装了最新版 Go(参见 Go 官方安装指南):1.2 创建新的 Go 项目
我们先从创建一个新的 Go 项目开始。main.go 文件,内容如下:
"Hello, CometBFT"。
v0.38.0 作为依赖。
go.mod 和 go.sum。
go.mod 文件应当类似如下:
v0.38.0 使用了一个稍显过时的 gogoproto 库,
在较新的 Go 版本下可能编译失败。为了避免任何编译错误,
请手动升级 gogoproto:
1.3 编写 CometBFT 应用
CometBFT 通过应用区块链接口(ABCI)与应用通信。 通过该接口交换的消息定义在 ABCI 的 protobuf 文件 中。 我们先为 ABCI 应用搭建基础骨架: 创建一个新的类型KVStoreApplication,它实现 abcitypes.Application 接口中定义的方法。
创建一个名为 app.go 的文件,内容如下:
go get 时已经作为依赖加入项目。
如果你的 IDE 没有识别这些类型,再执行一次下面的命令即可。
main.go,修改 main 函数如下,
其中会创建一个 KVStoreApplication 类型的实例。
go get 和 go build 来重新编译并运行应用,但它暂时还不会做任何事情。
因此接下来我们继续完善代码,补上实现最小键值存储所需的逻辑,
并让它与 CometBFT 服务一起启动。
1.3.1 添加持久化数据存储
我们的应用需要将状态写入持久化存储, 这样它停止后再启动时不会丢失全部数据。 在本教程中,我们将使用 BadgerDB, 这是一个速度很快的嵌入式键值存储。 首先,使用go get 命令将 Badger 添加为 go module 的依赖:
go get github.com/dgraph-io/badger/v3
接下来,更新应用及其构造函数,使其接收一个数据库句柄,如下所示:
onGoingBlock 用于跟踪在一个区块完成时更新应用状态的 Badger 事务。
现在先不用担心它;后面会详细说明。
接下来,更新顶部的 import 段,加入 Badger 库:
main.go 文件,调用更新后的构造函数:
1.3.2 CheckTx
当 CometBFT 从客户端或其他全节点收到一笔新交易时, 它会通过CheckTx 方法向应用确认该交易是否可接受。
无效交易不会被传播给其他节点,也不会进入任何区块,因此也不会被应用执行。
在我们的应用中,一笔交易是一个形如 key=value 的字符串,
表示向存储中写入一个键和值。
我们可以做的最基本校验,就是检查交易是否符合 key=value 这个模式。
为此,在 app.go 中加入如下辅助方法:
CheckTx 方法来使用这个辅助函数:
CheckTx 很简单,只验证交易格式是否正确,
但在实际应用中,CheckTx 非常常见的一种用法是更复杂地利用应用状态。
例如,你可能会拒绝覆盖已有值,或者为键值对关联版本号,
并允许调用方指定版本来执行条件更新。
根据检查逻辑及违反的条件不同,函数可以返回不同的值,
但只要返回的 code 非零,CometBFT 就会认为该交易无效。
当交易通过校验时,我们的 CheckTx 会向 CometBFT 返回 0。
这个 code 的具体数值对 CometBFT 本身没有特殊含义。
非零 code 会被 CometBFT 记录到日志中,因此应用可以借此提供更具体的拒绝原因。
注意,CheckTx 并不会执行交易;它只验证这笔交易是否“可以被执行”。
此时我们仍然不知道网络中的其他节点是否已经同意将这笔交易纳入某个区块。
最后,记得把 bytes 包也加入 app.go 顶部的 import 段:
1.3.3 FinalizeBlock
当 CometBFT 共识引擎已经对区块达成一致后, 该区块会通过FinalizeBlock 传递给应用。
FinalizeBlock 是 CometBFT v0.38.0 中引入的 ABCI 方法。
它替代了此前(v0.38.0 之前)由 BeginBlock、DeliverTx 和 EndBlock 三个 ABCI 方法组合提供的功能。
FinalizeBlock 的参数是对 BeginBlock、DeliverTx 和 EndBlock 参数的聚合。
这个方法负责执行区块,并向共识引擎返回响应。
使用单一的 FinalizeBlock 方法来表示区块最终落定,简化了 ABCI 接口,也提高了执行流水线的灵活性。
FinalizeBlock 方法会执行区块,包括任何必要的交易处理和状态更新,
并返回一个 ResponseFinalizeBlock 对象,其中包含关于已执行区块的必要信息。
注意: FinalizeBlock 只是在准备要进行的更新,并不会真正修改应用状态。
状态变更会在后续阶段真正提交,也就是 commit 阶段。
注意,为了在应用中实现这些调用,我们会使用 Badger 的事务机制。
为了避免与 CometBFT 交付区块中包含的交易混淆,
我们始终将前者称为 Badger 事务,而后者称为 应用交易。
首先,在 FinalizeBlock 期间创建一个新的 Badger 事务。
当前区块中的所有应用交易都会在这个 Badger 事务中执行。
接着,修改 FinalizeBlock,使得应用每次从 RequestFinalizeBlock 接收到一笔新应用交易时,
都把对应的 key 和 value 加入这个 Badger 事务。
注意,我们会在 FinalizeBlock 中再次检查交易的有效性。
CheckTx 与稍后在 FinalizeBlock 中真正交付交易之间,
应用状态可能已经发生变化,导致这笔交易不再有效。
注意,FinalizeBlock 此时还不能提交我们在区块执行期间构建的 Badger 事务。
Query 等其他方法依赖于应用状态的一致视图;
因此应用应当只在完整区块交付结束并调用 Commit 方法时,
通过提交 Badger 事务来更新状态。
Commit 方法会通知应用将应用交易的影响永久写入。
让我们更新这个方法,终结挂起的 Badger 事务,
并持久化最终状态:
log 库也加入 import 段:
FinalizeBlock 或 Commit 方法中从 Badger 数据库收到意外错误,
我们编写的这个应用会直接崩溃。
这并不是意外设计。
如果应用从数据库收到了错误,
它就没有确定性的方式继续安全推进,因此唯一安全的选择就是终止。
当应用重启后,该执行失败区块中的交易会重新执行;
如果 Badger 错误只是暂时性的,那么这些交易应当能够成功执行。
1.3.4 Query
当客户端尝试从kvstore 读取信息时,请求会由 Query 方法处理。
为此,我们将 app.go 中的 Query 方法重写如下:
1.3.5 PrepareProposal 和 ProcessProposal
PrepareProposal 和 ProcessProposal 是 CometBFT v0.37.0 中引入的方法,
用于让应用在交易区块的构建与处理过程中拥有更多控制权。
当 CometBFT 发现存在可被纳入区块的有效交易(已通过 CheckTx 验证)时,
它会先将其中一部分交易分组,然后通过调用 PrepareProposal 给应用一个修改该分组的机会。
应用可以在返回前自由修改这个分组,前提是最终得到的交易集合
占用字节数不能超过 RequestPrepareProposal.max_tx_bytes。
例如,应用可以对交易重新排序、添加交易,甚至移除交易,
以便在区块被接受后提升执行效果。
下面的代码中,应用只是原样返回未修改的交易分组:
1.4 在同一进程中启动应用和 CometBFT 实例
现在,我们已经具备了应用的基础功能,接下来把这些内容整合到main.go 文件中。
将你的 main.go 文件内容改为如下所示。
FilePV,它是一个私有验证者(也就是用于签署共识消息的组件)。
通常情况下,你会使用 SignerRemote 来连接外部 HSM。
nodeKey 用于在 p2p 网络中标识节点。
1.5 初始化并运行
我们的应用已经几乎可以运行了,但首先还需要填充 CometBFT 的配置文件。 下面的命令会在你的项目中创建一个cometbft-home 目录,并在 cometbft-home/config/ 中加入一组基础配置文件。
关于这些文件具体包含什么内容,参见 配置文档。
在项目根目录执行:
num_valid_txs=0 这一部分可以看出,这些区块目前还是空的,不过我们接下来就来解决这个问题。
1.6 使用应用
让我们尝试向这个新应用提交一笔交易。 打开另一个终端窗口,运行下面的 curl 命令:json 对象,其中包含 key 和 value 字段。
key 和 value。
这里发生了什么?
响应中包含的是我们提交数据的 base64 编码形式。
要从这些数据中还原出原始值,可以使用 base64 命令行工具:
结语
希望你已经顺利跑通全部流程。如果你在实践本教程时遇到任何问题,可以通过 Discord 联系我们,或者在 GitHub 上新建一个 issue。Guide Assumptions
This guide is designed for beginners who want to get started with a CometBFT application from scratch. It does not assume that you have any prior experience with CometBFT. CometBFT is a service that provides a Byzantine Fault Tolerant consensus engine for state-machine replication. The replicated state machine, or “application”, can be written in any language that can send and receive protocol buffer messages in a client-server model. Applications written in Go can also use CometBFT as a library and run the service in the same process as the application. By following along with this tutorial, you will create a CometBFT application called kvstore, a (very) simple distributed BFT key-value store. The application will be written in Go, and some understanding of the Go programming language is expected. If you have never written Go, you may want to go through Learn X in Y minutes Where X=Go first to familiarize yourself with the syntax. Note: Please use the latest released version of this guide and of CometBFT. We strongly advise against using unreleased commits for your development.Built-in app vs external app
On the one hand, to get maximum performance, you can run your application in the same process as CometBFT, as long as your application is written in Go. Cosmos SDK is written this way. This is the approach followed in this tutorial. On the other hand, having a separate application might give you better security guarantees, as two processes would be communicating via an established binary protocol. CometBFT will not have access to the application’s state. If that is the way you wish to proceed, use the Creating an application in Go guide instead of this one.1.1 Installing Go
Verify that you have the latest version of Go installed (refer to the official guide for installing Go):1.2 Creating a new Go project
We’ll start by creating a new Go project.main.go file with the following content:
v0.38.0 in this example.
go.mod and go.sum.
The go.mod file should look similar to:
v0.38.0 uses a slightly outdated gogoproto library, which
may fail to compile with newer Go versions. To avoid any compilation errors,
upgrade gogoproto manually:
1.3 Writing a CometBFT application
CometBFT communicates with the application through the Application BlockChain Interface (ABCI). The messages exchanged through the interface are defined in the ABCI protobuf file. We begin by creating the basic scaffolding for an ABCI application by creating a new type,KVStoreApplication, which implements the
methods defined by the abcitypes.Application interface.
Create a file called app.go with the following contents:
go get. If your IDE is not recognizing the types, go ahead and run the command again.
main.go and modify the main function so it matches the following,
where an instance of the KVStoreApplication type is created.
go get and go build, but it does
not do anything.
So let’s revisit the code, adding the logic needed to implement our minimal key-value store
and to start it along with the CometBFT service.
1.3.1 Add a persistent data store
Our application will need to write its state out to persistent storage so that it can stop and start without losing all of its data. For this tutorial, we will use BadgerDB, a fast embedded key-value store. First, add Badger as a dependency of your go module using thego get command:
go get github.com/dgraph-io/badger/v3
Next, let’s update the application and its constructor to receive a handle to the database, as follows:
onGoingBlock keeps track of the Badger transaction that will update the application’s state when a block
is completed. Don’t worry about it for now; we’ll get to that later.
Next, update the import stanza at the top to include the Badger library:
main.go file to invoke the updated constructor:
1.3.2 CheckTx
When CometBFT receives a new transaction from a client or from another full node, CometBFT asks the application if the transaction is acceptable using theCheckTx method.
Invalid transactions will not be shared with other nodes and will not become part of any blocks and, therefore, will not be executed by the application.
In our application, a transaction is a string with the form key=value, indicating a key and value to write to the store.
The most basic validation check we can perform is to check if the transaction conforms to the key=value pattern.
For that, let’s add the following helper method to app.go:
CheckTx method to use the helper function:
CheckTx is simple and only validates that the transaction is well-formed,
it is very common for CheckTx to make more complex use of the state of an application.
For example, you may refuse to overwrite an existing value, or you can associate
versions to the key-value pairs and allow the caller to specify a version to
perform a conditional update.
Depending on the checks and on the conditions violated, the function may return
different values, but any response with a non-zero code will be considered invalid
by CometBFT. Our CheckTx logic returns 0 to CometBFT when a transaction passes
its validation checks. The specific value of the code is meaningless to CometBFT.
Non-zero codes are logged by CometBFT, so applications can provide more specific
information on why the transaction was rejected.
Note that CheckTx does not execute the transaction; it only verifies that the transaction could be executed. We do not know yet if the rest of the network has agreed to accept this transaction into a block.
Finally, make sure to add the bytes package to the import stanza at the top of app.go:
1.3.3 FinalizeBlock
When the CometBFT consensus engine has decided on the block, the block is transferred to the application viaFinalizeBlock.
FinalizeBlock is an ABCI method introduced in CometBFT v0.38.0. This replaces the functionality provided previously (pre-v0.38.0) by the combination of ABCI methods BeginBlock, DeliverTx, and EndBlock. FinalizeBlock’s parameters are an aggregation of those in BeginBlock, DeliverTx, and EndBlock.
This method is responsible for executing the block and returning a response to the consensus engine.
Providing a single FinalizeBlock method to signal the finalization of a block simplifies the ABCI interface and increases flexibility in the execution pipeline.
The FinalizeBlock method executes the block, including any necessary transaction processing and state updates, and returns a ResponseFinalizeBlock object, which contains any necessary information about the executed block.
Note: FinalizeBlock only prepares the update to be made and does not change the state of the application. The state change is actually committed in a later stage, i.e., in the commit phase.
Note that to implement these calls in our application, we’re going to make use of Badger’s transaction mechanism. We will always refer to these as Badger transactions, not to confuse them with the transactions included in the blocks delivered by CometBFT, the application transactions.
First, let’s create a new Badger transaction during FinalizeBlock. All application transactions in the current block will be executed within this Badger transaction.
Next, let’s modify FinalizeBlock to add the key and value to the Badger transaction every time our application processes a new application transaction from the list received through RequestFinalizeBlock.
Note that we check the validity of the transaction again during FinalizeBlock.
CheckTx and the transaction delivery in FinalizeBlock in a way that rendered the transaction no longer valid.
Note that FinalizeBlock cannot yet commit the Badger transaction we were building during the block execution.
Other methods, such as Query, rely on a consistent view of the application’s state; the application should only update its state by committing the Badger transactions when the full block has been delivered and the Commit method is invoked.
The Commit method tells the application to make permanent the effects of
the application transactions.
Let’s update the method to terminate the pending Badger transaction and
persist the resulting state:
import stanza as well:
FinalizeBlock or Commit methods.
This is not an accident. If the application received an error from the database, there
is no deterministic way for it to make progress, so the only safe option is to terminate.
Once the application is restarted, the transactions in the block that failed execution will
be re-executed and should succeed if the Badger error was transient.
1.3.4 Query
When a client tries to read some information from thekvstore, the request will be
handled in the Query method. To do this, let’s rewrite the Query method in app.go:
1.3.5 PrepareProposal and ProcessProposal
PrepareProposal and ProcessProposal are methods introduced in CometBFT v0.37.0
to give the application more control over the construction and processing of transaction blocks.
When CometBFT sees that valid transactions (validated through CheckTx) are available to be
included in blocks, it groups some of these transactions and then gives the application a chance
to modify the group by invoking PrepareProposal.
The application is free to modify the group before returning from the call, as long as the resulting set
does not use more bytes than RequestPrepareProposal.max_tx_bytes.
For example, the application may reorder, add, or even remove transactions from the group to improve the
execution of the block once accepted.
In the following code, the application simply returns the unmodified group of transactions:
1.4 Starting an application and a CometBFT instance in the same process
Now that we have the basic functionality of our application in place, let’s put it all together inside of ourmain.go file.
Change the contents of your main.go file to the following.
FilePV, which is a private validator (i.e., a thing which signs consensus
messages). Normally, you would use SignerRemote to connect to an external
HSM.
nodeKey is needed to identify the node in a p2p network.
1.5 Initializing and Running
Our application is almost ready to run, but first we’ll need to populate the CometBFT configuration files. The following command will create acometbft-home directory in your project and add a basic set of configuration files in cometbft-home/config/.
For more information on what these files contain, see the configuration documentation.
From the root of your project, run:
num_valid_txs=0 part, are empty, but let’s remedy that next.
1.6 Using the application
Let’s try submitting a transaction to our new application. Open another terminal window and run the following curl command:json object with a key and value field set.
key and value we sent to CometBFT.
What’s going on here?
The response contains a base64 encoded representation of the data we submitted.
To get the original value out of this data, we can use the base64 command line utility: