什么是 Protobuf?
Protocol Buffers(protobuf)是由 Google 开发的一种与语言无关的二进制序列化格式。你可以使用一种模式语言在.proto 文件中定义数据结构,然后基于该模式为目标语言生成代码。生成的代码负责序列化(将结构化数据转换为字节)和反序列化(将字节还原为结构化数据)。
一个简单的 protobuf 消息如下所示:
为什么 Cosmos SDK 使用 protobuf
Cosmos SDK 使用 protobuf 的根本原因是:共识要求确定性。 网络中的每个验证者都会独立执行每个区块。执行完成后,每个验证者都会计算应用哈希(app hash),也就是应用状态的密码学哈希。要让验证者在 app hash 上达成一致,他们写入的每一份状态数据都必须产出完全相同的字节。 仅靠 protobuf 本身并不能保证这一点。Cosmos SDK 使用 protobuf 时,叠加了额外的确定性编码规则,这些规则在 ADR-027(确定性 Protobuf 序列化)中被正式定义。ADR-027 规定了诸如字段必须按字段编号升序出现、varint 编码必须尽可能短等约束。SDK 会在处理交易之前,先依据这些规则校验传入交易,因此非确定性编码的交易会被直接拒绝,而不是产生分叉状态。所有验证者在这些规则下对同一份数据进行编码时,都会得到完全一致的字节序列。 除了确定性之外,protobuf 还提供了:- 紧凑编码:二进制线格式比 JSON 或 XML 更小,这对交易吞吐量和区块大小很重要。
- 模式演进:可以新增或废弃字段而不破坏现有客户端,这对链升级至关重要。
- 代码生成:
.proto文件可以自动生成 Go 结构体、gRPC 服务桩以及 REST 网关处理器。 - 跨语言支持:任意语言的客户端都可以基于同一组
.proto文件生成代码并与链交互。
二进制编码与 JSON 编码
Cosmos SDK 在两种编码模式下使用 protobuf: 二进制编码是所有参与共识内容的默认编码方式:写入区块的交易、存储在 KV store 中的状态,以及创世数据。二进制编码紧凑且具有确定性。当交易被广播到网络时,它以 protobuf 二进制形式传输。当模块写入状态时,它会先将值序列化为 protobuf 二进制,再对 store 调用Set。
JSON 编码用于人类可读的输出:CLI、gRPC-gateway REST 端点以及链下工具。Cosmos SDK 使用 protobuf 的 JSON 编码(ProtoMarshalJSON),而不是标准 Go JSON;这样可以保留 .proto 模式中的字段名,并正确处理 Any 等特殊类型。
需要牢记的是,二进制编码对共识至关重要。两个验证者对于相同数据必须产出完全一致的二进制字节。JSON 只用于人类或外部客户端需要读取数据的场景;它绝不会影响 AppHash。
genesis.json 的 JSON 形式分发,但在链初始化期间,InitGenesis 会将该 JSON 反序列化为 protobuf 结构体,并以二进制形式写入 KV store。KV store(以及因此得到的 AppHash)中只会包含二进制形式。
交易编码
交易是定义在cosmos.tx.v1beta1 中的 protobuf 消息。一笔交易由三部分组成:
- TxBody 包含待执行的消息,以
repeated google.protobuf.Any messages的形式序列化。 - AuthInfo 包含签名者信息(包括每个签名者各自的 sequence 编号)以及手续费。
- signatures 包含密码学签名,每个签名者各有一份。
google.protobuf.Any 的形式存储,这样单笔交易就可以同时包含来自不同模块的多种消息类型。
当用户提交交易时,SDK 会将其编码为 TxRaw,也就是一种扁平结构,其中 TxBody 字节、AuthInfo 字节和签名都已经完成序列化。随后它会将这一二进制表示广播到网络中。
交易签名与 SignDoc
交易并不是直接被签名的。相反,SDK 会构造一个名为 SignDoc 的确定性结构,用来精确定义签名者承诺的是哪些字节:
SignDoc 会被序列化为 protobuf 二进制,然后再用用户的私钥签名:
SignDoc 是以确定性方式序列化的,因此所有验证者在检查交易签名时验证的都是完全相同的字节。每个签名者的 sequence 编号位于 AuthInfo.SignerInfo.sequence 中,并被包含在 auth_info_bytes 里,而后者又是 SignDoc 的一部分,这正是防止重放攻击的机制。
签名模式
签名模式决定了签名者在签署交易时实际承诺的是哪些字节。SDK 支持多种签名模式,以适配不同的客户端和硬件:-
SIGN_MODE_DIRECT(默认):签名者对上文所述、以 protobuf 二进制序列化的SignDoc进行签名。这种方式紧凑、确定性强,并且是所有新开发的正确选择。 -
SIGN_MODE_LEGACY_AMINO_JSON:签名者签署的是经过 Amino JSON 编码的StdSignDoc,而不是 protobuf 的SignDoc。该模式是为与硬件钱包(例如较旧的 Ledger 固件)和早于 protobuf 出现的客户端工具保持向后兼容而保留的。新的模块和链不应依赖它。 -
SIGN_MODE_TEXTUAL:签名者对交易的人类可读 CBOR 编码表示进行签名,这种表示被设计为可以在硬件钱包屏幕上清晰展示(在 v0.50 中引入,见 ADR-050)。这是 SDK 在硬件钱包人类可读签名方向上的较新方案,目标是逐步替代SIGN_MODE_LEGACY_AMINO_JSON。它的规范带有版本管理,并且随着 SDK 版本演进而发生过变化。 -
SIGN_MODE_DIRECT_AUX:允许多签交易中的 N-1 个签名者只对TxBody和各自的SignerInfo进行签名,而无需指定手续费。被指定的手续费支付者最后使用SIGN_MODE_DIRECT签名。这简化了多签交互体验。
signing.proto 中。
对于模块开发者:
SIGN_MODE_DIRECT 不需要任何额外工作。如果你希望模块中的消息能够在使用 SIGN_MODE_LEGACY_AMINO_JSON 的 Ledger 硬件钱包上完成签名,请在模块的 codec.go 中通过 RegisterLegacyAminoCodec 将你的消息类型注册到 Amino codec。消息签名者
每个交易消息都必须声明哪些地址有权对其签名。在 v0.50+ 中,这是通过cosmos.msg.v1.signer protobuf 注解完成的。SDK 会在启动时读取该注解,并自动从对应字段中提取签名者地址。完整的注解说明请参阅Protobuf 注解。
对于无法使用该注解的消息,例如具有非标准签名逻辑的消息(如兼容 EVM 的交易),你可以使用 signing.CustomGetSigner 注册自定义签名者函数:
TxConfig 时,对传给 authtx.NewTxConfigWithOptions 的 txsigning.Options 调用 signingOptions.DefineCustomGetSigners(msgType, fn)。
protobuf 在模块中的使用方式
现代 SDK 模块中,大多数公开且需要持久化的数据类型都定义在.proto 文件中,并使用 protobuf 进行序列化。这覆盖了核心 API 表面:交易消息、查询请求/响应类型、存储的状态值,以及创世状态。
消息与交易
每个模块都会在tx.proto 文件中定义自己的交易消息。上面的 MsgSend 定义就是一个示例。当用户提交交易时,SDK 会先使用 protobuf 将交易体(包括其中的消息)序列化为二进制,再进行广播。
如需动手示例,请参阅“构建模块”教程中的 tx.proto。
查询
模块会在query.proto 中定义查询服务。请求类型和响应类型都是 protobuf 消息。SDK 使用 gRPC 进行查询,而 gRPC 按定义就是使用 protobuf 作为序列化格式。
如需动手示例,请参阅“构建模块”教程中的 query.proto。
状态类型
存储在 KV store 中的数据是经过 protobuf 编码的。模块在存储自定义结构体时,会先使用 codec 将其编码为字节,再把这些字节写入 store。读取时,则会把字节反序列化回结构体。注意,只有值会进行 protobuf 编码;键是手工构造的字节序列,而不是 protobuf。键布局在状态、存储与创世一节中有说明。创世
创世状态定义在genesis.proto 中。InitGenesis 和 ExportGenesis 使用 protobuf 对 genesis.json 中的创世状态进行反序列化和重新序列化。
下面这个具体示例展示了模块如何将带类型的状态以字节形式读写:
k.cdc)就是下一节所介绍的 protobuf codec。
编解码器与接口注册表
Cosmos SDK 将 protobuf 封装在一个模块用于编组与反编组的 codec 中。其主要实现是ProtoCodec,底层会调用 protobuf 的 Marshal 与 Unmarshal。
接口类型与 Any
Protobuf 是强类型的。你不能在 protobuf 消息中直接把某个字段存成“某个接口的某种实现”。Cosmos SDK 使用 protobuf 的 google.protobuf.Any 解决这个问题,它会包装一个任意消息类型,并附带一个用于标识内部具体类型的 URL。
凡是 SDK 需要序列化一个在编译期无法确定具体类型的值时,都会使用 Any。最常见的例子是公钥。一个账户可能使用 secp256k1 密钥、ed25519 密钥或多重签名密钥。BaseAccount 会将公钥存为 Any:
Any 字段保存序列化后的公钥字节,以及类似 /cosmos.crypto.secp256k1.PubKey 这样的类型 URL。当 SDK 读取账户时,会使用该类型 URL 查找具体的 Go 类型,然后将字节反编组到该类型中。
交易中的消息
交易消息是 SDK 中Any 最常见的使用场景。一笔交易可以在同一个 TxBody 中携带来自不同模块的多种消息类型(bank.MsgSend、staking.MsgDelegate、gov.MsgVote)。由于 protobuf 要求字段层面必须是具体类型,因此每条消息在放入交易前都会先被打包进一个 Any:
type_url,在接口注册表中查找对应的具体类型,并将字节反编组成正确的消息结构体。这也是为什么每个 sdk.Msg 实现都必须在应用启动前通过 RegisterInterfaces 完成注册。
这个查找过程由 interface registry 处理。
接口注册表
InterfaceRegistry 是一个在运行时将类型 URL 映射到 Go 类型的表。当 SDK 遇到一个 Any 值时,会用该类型 URL 查询注册表以找到对应的具体 Go 类型,然后使用 protobuf 对字节进行反编组。
Any 值。这也是为什么类型必须先显式注册,之后才能被反序列化。
注册接口实现
由于接口注册表是一个运行时查找表,所有实现了 SDK 接口的具体类型都必须在应用启动前完成注册。这通过RegisterInterfaces 完成:
PubKey 接口可以是 secp256k1.PubKey 或 ed25519.PubKey。”如果某个类型在应用的任意位置被用于 Any 字段中,但没有完成注册,那么 codec 在对其执行反编组时就会失败并返回错误。
每个模块都会在应用初始化期间调用 RegisterInterfaces,而 app.go 会在构建应用时通过模块管理器调用这些注册函数。实现了 SDK 接口的自定义类型也必须遵循同样的模式。
codec.go
按照约定,模块会将所有 codec 注册集中放在单个文件中:x/mymodule/types/codec.go。这个文件通常包含两个函数:
RegisterInterfaces 都是必需的。没有它,SDK 就无法从交易中解码这些消息。RegisterLegacyAminoCodec 是可选的,只有在你需要通过 SIGN_MODE_LEGACY_AMINO_JSON 支持 Ledger 硬件钱包时才需要。
关于工作模块中的接口注册示例,参见 Build a Module 教程中的 Interface Registration。
Proto 到代码的生成流程
编写.proto 文件后,会通过代码生成步骤产出 .pb.go 文件。生成的 Go 代码包含结构体定义、marshal/unmarshal 方法以及 gRPC 服务桩。你不应该直接编辑这些生成文件。
流程如下:
1. 编写 .proto 文件
模块的 proto 文件位于仓库根目录下的 proto/ 目录:
.proto 文件运行 buf(或带插件的 protoc),并在模块的 types/ 目录下生成 Go 代码。Cosmos SDK 的完整生成 API 参考发布在 buf.build/cosmos/cosmos-sdk/docs/main。
proto.Message,因此可以直接传给 codec 做编组,注册到接口注册表中,并用于 keeper 方法和消息处理器:
Legacy Amino 编码
在 protobuf 之前,Cosmos SDK 使用一种名为 Amino 的自定义序列化格式来处理交易编码、JSON 签名文档和接口序列化。现在 protobuf 已经在这些角色上取代了它。LegacyAmino codec 仍然存在以保持向后兼容,但它不用于共识关键路径。
仍有一些遗留组件会引用它:
LegacyAmino仍保留在 codec 包中以支持向后兼容LegacyAminoPubKey(多重签名)会与 protobuf 公钥类型一起注册- 一些较老的链、硬件钱包和客户端工具仍依赖 Amino JSON 签名
编码在整体流程中的位置
Cosmos SDK 的每一层都依赖编码:sdk.Context、gas 计量以及事件。
As described in State, Storage, and Genesis, modules write structured state values into the KV store as raw bytes. Encoding defines how those structured values are serialized into bytes, and why every validator must produce exactly the same bytes. This page explains how that encoding works, why the Cosmos SDK chose Protocol Buffers, and what that means for module development.
What is Protobuf?
Protocol Buffers (protobuf) is a language-neutral, binary serialization format developed by Google. You define your data structures in.proto files using a schema language, then generate code in your target language from that schema. The generated code handles serialization (converting structured data into bytes) and deserialization (converting bytes back into structured data).
A simple protobuf message looks like this:
Why the Cosmos SDK uses protobuf
The Cosmos SDK uses protobuf for a fundamental reason: consensus requires determinism. Every validator in the network independently executes each block. After execution, each validator computes the app hash, a cryptographic hash of the application state. For validators to agree on the app hash, they must all produce exactly the same bytes for every piece of state they write. Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf with additional deterministic encoding rules formalized in ADR-027 (Deterministic Protobuf Serialization). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. Beyond determinism, protobuf provides:- Compact encoding: binary wire format is smaller than JSON or XML, which matters for transaction throughput and block size.
- Schema evolution: fields can be added or deprecated without breaking existing clients, which is critical for chain upgrades.
- Code generation:
.protofiles generate Go structs, gRPC service stubs, and REST gateway handlers automatically. - Cross-language support: clients in any language can interact with the chain by generating code from the same
.protofiles.
Binary and JSON encoding
The Cosmos SDK uses protobuf in two encoding modes: Binary encoding is the default for everything that participates in consensus: transactions written to blocks, state stored in KV stores, and genesis data. Binary encoding is compact and deterministic. When a transaction is broadcast to the network, it travels as protobuf binary. When a module writes state, it serializes values to protobuf binary before callingSet on the store.
JSON encoding is used for human-readable output: the CLI, gRPC-gateway REST endpoints, and off-chain tooling. The Cosmos SDK uses protobuf’s JSON encoding (ProtoMarshalJSON) rather than standard Go JSON, which preserves field names from the .proto schema and handles special types like Any correctly.
It is important to keep in mind that binary encoding is consensus-critical. Two validators must produce identical binary bytes for identical data. JSON is only used where humans or external clients need to read the data; it never influences the AppHash.
genesis.json, but during chain initialization InitGenesis deserializes that JSON into protobuf structs and writes them to the KV store as binary. The KV store (and therefore the AppHash) only ever contains the binary form.
Transaction encoding
Transactions are protobuf messages defined incosmos.tx.v1beta1. A transaction is composed of three parts:
- TxBody contains the messages to execute, serialized as
repeated google.protobuf.Any messages. - AuthInfo contains signer information (including the per-signer sequence number) and fee.
- signatures contains the cryptographic signatures, one per signer.
google.protobuf.Any values so that a single transaction can contain multiple message types from different modules.
When a user submits a transaction, the SDK encodes it as a TxRaw—a flat structure with the TxBody bytes, AuthInfo bytes, and signatures already serialized. It then broadcasts that binary representation over the network.
Transaction signing and SignDoc
Transactions are not signed directly. Instead, the SDK constructs a deterministic structure called a SignDoc, which defines exactly what bytes the signer commits to:
SignDoc is serialized to protobuf binary and then signed with the user’s private key:
SignDoc is serialized deterministically, all validators verify the exact same bytes when checking transaction signatures. The per-signer sequence number lives in AuthInfo.SignerInfo.sequence and is included in auth_info_bytes, which is part of SignDoc—this is what prevents replay attacks.
Sign modes
A sign mode determines what bytes a signer commits to when signing a transaction. The SDK supports multiple sign modes to accommodate different clients and hardware:-
SIGN_MODE_DIRECT(default): the signer signs over the protobuf-binary-serializedSignDocdescribed above. This is compact, deterministic, and the correct choice for all new development. -
SIGN_MODE_LEGACY_AMINO_JSON: the signer signs over an Amino JSON-encodedStdSignDocinstead of the protobufSignDoc. This exists for backward compatibility with hardware wallets (e.g., older Ledger firmware) and client tooling that predates protobuf. New modules and chains should not depend on it. -
SIGN_MODE_TEXTUAL: the signer signs over a human-readable CBOR-encoded representation of the transaction, designed to display legibly on hardware wallet screens (introduced in v0.50, see ADR-050). This is the SDK’s newer direction for human-readable signing on hardware wallets, intended to replaceSIGN_MODE_LEGACY_AMINO_JSONover time. Its specification is versioned and has evolved across SDK releases. -
SIGN_MODE_DIRECT_AUX: allows N-1 signers in a multi-signer transaction to sign over onlyTxBodyand their ownSignerInfo, without specifying fees. The designated fee payer signs last usingSIGN_MODE_DIRECT. This simplifies multi-signature UX.
signing.proto.
For module developers:
SIGN_MODE_DIRECT requires no extra work. If you want your module’s messages to be signable on Ledger hardware wallets using SIGN_MODE_LEGACY_AMINO_JSON, register your message types with the Amino codec via RegisterLegacyAminoCodec in your module’s codec.go.Message signers
Every transaction message must declare which addresses are authorized to sign it. In v0.50+, this is done via thecosmos.msg.v1.signer protobuf annotation — the SDK reads the annotation at startup and automatically extracts signer addresses from that field. See Protobuf Annotations for the full annotation reference.
For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using signing.CustomGetSigner:
signingOptions.DefineCustomGetSigners(msgType, fn) on the txsigning.Options you pass to authtx.NewTxConfigWithOptions when building your app’s TxConfig.
How protobuf is used in modules
Most public and persisted data types in modern SDK modules are defined in.proto files and serialized with protobuf. This covers the core API surface: transaction messages, query request/response types, stored state values, and genesis state.
Messages and transactions
Each module defines its transaction messages in atx.proto file. The MsgSend definition above is an example. When a user submits a transaction, the SDK serializes the transaction body (including its messages) to binary using protobuf before broadcasting it.
For a hands-on example, see tx.proto in the Build a Module tutorial.
Queries
Modules define their query services inquery.proto. Request and response types are protobuf messages. The SDK uses gRPC for queries, and gRPC uses protobuf as its serialization format by definition.
For a hands-on example, see query.proto in the Build a Module tutorial.
State types
Data stored in the KV store is protobuf-encoded. A module that stores a custom struct first marshals it to bytes using the codec, then writes those bytes to the store. When reading, it unmarshals the bytes back into the struct. Note that only values are protobuf-encoded; keys are manually constructed byte sequences, not protobuf. Key layout is covered in the State, Storage, and Genesis section.Genesis
Genesis state is defined ingenesis.proto. InitGenesis and ExportGenesis use protobuf to deserialize genesis state from genesis.json and serialize it back.
A concrete example shows how a module reads and writes typed state as bytes:
k.cdc) is the protobuf codec described in the next section.
The codec and interface registry
The Cosmos SDK wraps protobuf in a codec that modules use for marshaling and unmarshaling. The primary implementation isProtoCodec, which calls protobuf’s Marshal and Unmarshal under the hood.
Interface types and Any
Protobuf is strongly typed. You cannot store a field as “some implementation of an interface” directly in a protobuf message. The Cosmos SDK solves this using protobuf’s google.protobuf.Any, which wraps an arbitrary message type alongside a URL that identifies what type it contains.
Any is used anywhere the SDK needs to serialize a value whose concrete type is not known at compile time. The most common example is public keys. An account might use a secp256k1 key, an ed25519 key, or a multisig key. The BaseAccount stores the public key as Any:
Any field holds the serialized public key bytes plus a type URL like /cosmos.crypto.secp256k1.PubKey. When the SDK reads the account, it uses the type URL to look up the concrete Go type, then unmarshals the bytes into that type.
Messages inside transactions
Transaction messages are the most common use ofAny in the SDK. A transaction can carry multiple message types from different modules (bank.MsgSend, staking.MsgDelegate, gov.MsgVote) in a single TxBody. Because protobuf requires concrete types at the field level, each message is packed into an Any before being placed inside the transaction:
type_url, looks up the concrete type in the interface registry, and unmarshals the bytes into the correct message struct. This is why every sdk.Msg implementation must be registered with RegisterInterfaces before the application starts.
This lookup is handled by the interface registry.
Interface registry
TheInterfaceRegistry is a runtime map from type URLs to Go types. When the SDK encounters an Any value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes.
Any values. This is why types must be explicitly registered before they can be deserialized.
Registering interface implementations
Because the interface registry is a runtime lookup table, every concrete type that implements an SDK interface must be registered before the application starts. This is done withRegisterInterfaces:
PubKey interface can be a secp256k1.PubKey or an ed25519.PubKey.” If a type is used in an Any field anywhere in the application and is not registered, the codec will fail to unmarshal it and return an error.
Each module calls RegisterInterfaces during app initialization, and app.go calls these registration functions through the module manager when building the app. Custom types that implement SDK interfaces must follow the same pattern.
codec.go
By convention, modules collect all codec registration in a single file: x/mymodule/types/codec.go. This file typically contains two functions:
RegisterInterfaces is required for every module that defines message types. Without it, the SDK cannot decode those messages from transactions. RegisterLegacyAminoCodec is optional and only needed for Ledger hardware wallet support via SIGN_MODE_LEGACY_AMINO_JSON.
For an example of interface registration in a working module, see Interface Registration in the Build a Module tutorial.
Proto-to-code generation workflow
Writing.proto files produces .pb.go files through a code generation step. The generated Go code contains struct definitions, marshal/unmarshal methods, and gRPC service stubs. You never edit these generated files directly.
The workflow is:
1. Write the .proto file
Proto files for a module live in the proto/ directory at the repository root:
buf (or protoc with plugins) against the .proto files and produces Go code under the module’s types/ directory. The full generated API reference for the Cosmos SDK is published at buf.build/cosmos/cosmos-sdk/docs/main.
proto.Message and can be passed directly to the codec for marshaling, registered with the interface registry, and used in keeper methods and message handlers:
Legacy Amino encoding
Before protobuf, the Cosmos SDK used a custom serialization format called Amino for transaction encoding, JSON signing documents, and interface serialization. Protobuf has replaced it in all of those roles. TheLegacyAmino codec still exists for backward compatibility, but is not used in the consensus-critical path.
Some legacy components still reference it:
LegacyAminois still present in the codec package for backward-compatibilityLegacyAminoPubKey(multisig) is registered alongside protobuf public key types- Some older chains, hardware wallets, and client tooling depend on Amino JSON signing
Encoding in context
Every layer of the Cosmos SDK depends on encoding:sdk.Context, gas metering, and events.