ADR 031: Protobuf Msg Services

变更记录

2020-10-05：初始草案
2021-04-21：移除 ServiceMsg，以遵循 Protobuf Any 的规范，见 #9063。

状态

已接受

摘要

我们希望利用 protobuf 的 service 定义来定义 Msg，这将在生成代码以及返回类型现已明确定义这两方面，显著改善开发者体验。

背景

当前，Cosmos SDK 中的 Msg 处理器确实具有返回值，这些返回值会放入响应的 data 字段中。然而，这些返回值除了在 golang 处理器代码中之外，并未在其他地方进行说明。在早期讨论中，曾有人提出使用 protobuf 扩展字段来描述 Msg 返回类型，例如：

package cosmos.gov;

message MsgSubmitProposal
	option (cosmos_proto.msg_return) = “uint64”;
	string delegator_address = 1;
	string validator_address = 2;
	repeated sdk.Coin amount = 3;
}

不过，这一方案从未被采纳。为 Msg 提供定义良好的返回值将改善客户端体验。例如，在 x/gov 中，MsgSubmitProposal 会以大端序 uint64 返回 proposal ID。这一点实际上没有在任何地方得到明确文档说明，客户端需要了解 Cosmos SDK 的内部实现，才能解析该值并将其返回给用户。另外，也可能存在我们希望以编程方式使用这些返回值的场景。例如，Link 提出了一种使用 Msg 路由器执行模块间 Ocaps 的方法。明确定义的返回类型将改善这种方式的开发者体验。此外，Msg 类型的处理器注册往往会在 keeper 之上增加一些样板代码，通常还需要通过手动类型分支来完成。这本身未必是坏事，但确实增加了创建模块的额外开销。

决策

我们决定使用 protobuf 的 service 定义来定义 Msg，并使用其生成的代码来替代 Msg 处理器。下面我们以 x/gov 模块中的 SubmitProposal 消息为例，说明这一形式。我们先从一个 Msg service 定义开始：

package cosmos.gov;

service Msg {
  rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse);
}

// Note that for backwards compatibility this uses MsgSubmitProposal as the request
// type instead of the more canonical MsgSubmitProposalRequest
message MsgSubmitProposal {
  google.protobuf.Any content = 1;
  string proposer = 2;
}

message MsgSubmitProposalResponse {
  uint64 proposal_id;
}

虽然这种方式最常用于 gRPC，但像这样复用 protobuf service 定义并不违背 protobuf 规范的意图，规范中写道：

如果你不想使用 gRPC，也可以将 protocol buffers 与你自己的 RPC 实现一起使用。通过这种方式，我们将获得一个自动生成的 MsgServer 接口：

除了清晰地定义返回类型之外，这样做还有生成客户端和服务端代码的好处。在服务端，这几乎就像一个自动生成的 keeper 方法，并且未来甚至可能可以替代 keeper （见 #7093）：

package gov

type MsgServer interface {
    SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error)
}

在客户端，开发者可以通过创建封装交易逻辑的 RPC 实现来利用这一点。使用异步回调的 Protobuf 库，例如 protobuf.js，甚至可以借此为特定消息注册回调，即使该交易包含多个 Msg 也是如此。每个 Msg 服务方法都应当且只能有一个请求参数：其对应的 Msg 类型。例如，上面的 Msg 服务方法 /cosmos.gov.v1beta1.Msg/SubmitProposal 只有一个请求参数，即 Msg 类型 /cosmos.gov.v1beta1.MsgSubmitProposal。必须让读者清楚理解 Msg 服务（一个 Protobuf service）与 Msg 类型（一个 Protobuf message）在术语上的差异，以及它们完全限定名称之间的区别。之所以选择这一约定，而不是更规范的 Msg...Request 命名，主要是出于向后兼容考虑，同时也因为它在 TxBody.messages 中具有更好的可读性（见下文的编码部分）：包含 /cosmos.gov.MsgSubmitProposal 的交易比包含 /cosmos.gov.v1beta1.MsgSubmitProposalRequest 的交易读起来更自然。这一约定带来的一个结果是，每个 Msg 类型只能作为一个 Msg 服务方法的请求参数。不过，我们认为这种限制有助于增强显式性，是一种好的实践。

编码

使用 Msg 服务生成的交易，其编码方式与 ADR-020 中定义的当前 Protobuf 交易编码没有区别。我们会将 Msg 类型（即 Msg 服务方法的请求参数）编码为 Tx 中的 Any，这涉及将二进制编码后的 Msg 与其类型 URL 一起打包。

解码

由于 Msg 类型被打包进 Any 中，因此对交易消息的解码就是将这些 Any 解包为 Msg 类型。更多信息请参见 ADR-020。

路由

我们提议在 BaseApp 中增加一个 msg_service_router。该路由器是一个键值映射，用于将 Msg 类型的 type_url 映射到对应的 Msg 服务方法处理器。由于 Msg 类型与 Msg 服务方法之间是一一对应关系，因此 msg_service_router 对每个 Msg 服务方法都恰好有一个条目。当 BaseApp 处理一笔交易时（在 CheckTx 或 DeliverTx 中），其 TxBody.messages 会被解码为 Msg。随后，每个 Msg 的 type_url 都会与 msg_service_router 中的条目进行匹配，并调用对应的 Msg 服务方法处理器。为了向后兼容，旧的处理器暂时不会被移除。如果 BaseApp 接收到一个在 msg_service_router 中没有对应条目的旧版 Msg，则会通过其旧的 Route() 方法路由到旧版处理器。

模块配置

在 ADR 021 中，我们向 AppModule 引入了 RegisterQueryService 方法，使模块能够注册 gRPC 查询器。为了注册 Msg 服务，我们尝试采用一种更具扩展性的方法，将 RegisterQueryService 改为更通用的 RegisterServices 方法：

type AppModule interface {
    RegisterServices(Configurator)
  ...
}

type Configurator interface {
    QueryServer()

grpc.Server
  MsgServer()

grpc.Server
}

// example module:
func (am AppModule)

RegisterServices(cfg Configurator) {
    types.RegisterQueryServer(cfg.QueryServer(), keeper)

types.RegisterMsgServer(cfg.MsgServer(), keeper)
}

RegisterServices 方法和 Configurator 接口旨在继续演进，以满足 #7093 和 #7122 中讨论的用例。注册 Msg 服务时，框架应当验证所有 Msg 类型都实现了 sdk.Msg 接口，并在初始化期间抛出错误，而不是等到后续处理交易时才报错。

`Msg` 服务实现

与查询服务类似，Msg 服务方法可以通过 sdk.UnwrapSDKContext 方法，从 context.Context 参数中获取 sdk.Context：

package gov

func (k Keeper)

SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) {
    ctx := sdk.UnwrapSDKContext(goCtx)
    ...
}

BaseApp 的 msg_service_router 应当已经为该 sdk.Context 附加好 EventManager。采用这种方式后，就不再需要单独定义处理器。

影响

这一设计改变了模块功能的暴露和访问方式。它弃用了现有的 Handler 接口和 AppModule.Route，转而采用上文描述的 Protocol Buffer Services 以及服务路由。这极大地简化了代码。我们不再需要创建处理器和 keeper。使用 Protocol Buffer 自动生成的客户端后，模块与模块使用者之间的通信接口被清晰地区分开来。控制逻辑（即处理器和 keeper）将不再暴露。模块接口可以被视为一个通过客户端 API 访问的黑盒。值得注意的是，这些客户端接口同样也是由 Protocol Buffers 生成的。这也使我们能够改变功能测试的方式。我们将不再 mock AppModule 和 Router，而是 mock 一个客户端（服务端仍然保持隐藏）。更具体地说：在 moduleB 中，我们永远不会 mock moduleA.MsgServer，而是 mock moduleA.MsgClient。可以把它理解为在使用外部服务（例如数据库或在线服务等）。我们假设客户端与服务端之间的传输已经由生成的 Protocol Buffers 正确处理。最后，将模块对客户端 API 封闭起来，也打开了 ADR-033 中讨论的理想 OCAP 模式。由于服务端实现和接口都是隐藏的，没有人能够持有“keeper”或服务端实例，开发者将被迫依赖客户端接口，这会推动正确的封装和软件工程实践。

优点

清晰传达返回类型
不再需要手动注册处理器和手动编组返回类型，只需实现接口并完成注册
通信接口会自动生成，开发者现在只需关注状态转换方法；如果我们选择采用该方案，这将改善 #7093 方法（1）的开发体验
生成的客户端代码可能对客户端和测试都很有用
大幅减少并简化代码

缺点

在 gRPC 上下文之外使用 service 定义可能会引起困惑（但并不违反 proto3 规范）

参考资料

Changelog

2020-10-05: Initial Draft
2021-04-21: Remove ServiceMsgs to follow Protobuf Any’s spec, see #9063.

Status

Accepted

Abstract

We want to leverage protobuf service definitions for defining Msgs which will give us significant developer UX improvements in terms of the code that is generated and the fact that return types will now be well defined.

Context

Currently Msg handlers in the Cosmos SDK do have return values that are placed in the data field of the response. These return values, however, are not specified anywhere except in the golang handler code. In early conversations it was proposed that Msg return types be captured using a protobuf extension field, ex:

package cosmos.gov;

message MsgSubmitProposal
	option (cosmos_proto.msg_return) = “uint64”;
	string delegator_address = 1;
	string validator_address = 2;
	repeated sdk.Coin amount = 3;
}

This was never adopted, however. Having a well-specified return value for Msgs would improve client UX. For instance, in x/gov, MsgSubmitProposal returns the proposal ID as a big-endian uint64. This isn’t really documented anywhere and clients would need to know the internals of the Cosmos SDK to parse that value and return it to users. Also, there may be cases where we want to use these return values programatically. For instance, Link proposes a method for doing inter-module Ocaps using the Msg router. A well-defined return type would improve the developer UX for this approach. In addition, handler registration of Msg types tends to add a bit of boilerplate on top of keepers and is usually done through manual type switches. This isn’t necessarily bad, but it does add overhead to creating modules.

Decision

We decide to use protobuf service definitions for defining Msgs as well as the code generated by them as a replacement for Msg handlers. Below we define how this will look for the SubmitProposal message from x/gov module. We start with a Msg service definition:

package cosmos.gov;

service Msg {
  rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse);
}

// Note that for backwards compatibility this uses MsgSubmitProposal as the request
// type instead of the more canonical MsgSubmitProposalRequest
message MsgSubmitProposal {
  google.protobuf.Any content = 1;
  string proposer = 2;
}

message MsgSubmitProposalResponse {
  uint64 proposal_id;
}

While this is most commonly used for gRPC, overloading protobuf service definitions like this does not violate the intent of the protobuf spec which says:

If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation. With this approach, we would get an auto-generated MsgServer interface:

In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server side, this is almost like an automatically generated keeper method and could maybe be used intead of keepers eventually (see #7093):

package gov

type MsgServer interface {
    SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error)
}

On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction logic. Protobuf libraries that use asynchronous callbacks, like protobuf.js could use this to register callbacks for specific messages even for transactions that include multiple Msgs. Each Msg service method should have exactly one request parameter: its corresponding Msg type. For example, the Msg service method /cosmos.gov.v1beta1.Msg/SubmitProposal above has exactly one request parameter, namely the Msg type /cosmos.gov.v1beta1.MsgSubmitProposal. It is important the reader understands clearly the nomenclature difference between a Msg service (a Protobuf service) and a Msg type (a Protobuf message), and the differences in their fully-qualified name. This convention has been decided over the more canonical Msg...Request names mainly for backwards compatibility, but also for better readability in TxBody.messages (see Encoding section below): transactions containing /cosmos.gov.MsgSubmitProposal read better than those containing /cosmos.gov.v1beta1.MsgSubmitProposalRequest. One consequence of this convention is that each Msg type can be the request parameter of only one Msg service method. However, we consider this limitation a good practice in explicitness.

Encoding

Encoding of transactions generated with Msg services do not differ from current Protobuf transaction encoding as defined in ADR-020. We are encoding Msg types (which are exactly Msg service methods’ request parameters) as Any in Txs which involves packing the binary-encoded Msg with its type URL.

Decoding

Since Msg types are packed into Any, decoding transactions messages are done by unpacking Anys into Msg types. For more information, please refer to ADR-020.

Routing

We propose to add a msg_service_router in BaseApp. This router is a key/value map which maps Msg types’ type_urls to their corresponding Msg service method handler. Since there is a 1-to-1 mapping between Msg types and Msg service method, the msg_service_router has exactly one entry per Msg service method. When a transaction is processed by BaseApp (in CheckTx or in DeliverTx), its TxBody.messages are decoded as Msgs. Each Msg’s type_url is matched against an entry in the msg_service_router, and the respective Msg service method handler is called. For backward compatibility, the old handlers are not removed yet. If BaseApp receives a legacy Msg with no corresponding entry in the msg_service_router, it will be routed via its legacy Route() method into the legacy handler.

Module Configuration

In ADR 021, we introduced a method RegisterQueryService to AppModule which allows for modules to register gRPC queriers. To register Msg services, we attempt a more extensible approach by converting RegisterQueryService to a more generic RegisterServices method:

type AppModule interface {
    RegisterServices(Configurator)
  ...
}

type Configurator interface {
    QueryServer()

grpc.Server
  MsgServer()

grpc.Server
}

// example module:
func (am AppModule)

RegisterServices(cfg Configurator) {
    types.RegisterQueryServer(cfg.QueryServer(), keeper)

types.RegisterMsgServer(cfg.MsgServer(), keeper)
}

The RegisterServices method and the Configurator interface are intended to evolve to satisfy the use cases discussed in #7093 and #7122. When Msg services are registered, the framework should verify that all Msg types implement the sdk.Msg interface and throw an error during initialization rather than later when transactions are processed.

`Msg` Service Implementation

Just like query services, Msg service methods can retrieve the sdk.Context from the context.Context parameter method using the sdk.UnwrapSDKContext method:

package gov

func (k Keeper)

SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) {
    ctx := sdk.UnwrapSDKContext(goCtx)
    ...
}

The sdk.Context should have an EventManager already attached by BaseApp’s msg_service_router. Separate handler definition is no longer needed with this approach.

Consequences

This design changes how a module functionality is exposed and accessed. It deprecates the existing Handler interface and AppModule.Route in favor of Protocol Buffer Services and Service Routing described above. This dramatically simplifies the code. We don’t need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It’s worth to note that the client interfaces are also generated by Protocol Buffers. This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock moduleA.MsgServer in moduleB, but rather moduleA.MsgClient. One can think about it as working with external services (eg DBs, or online servers…). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers. Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold “keepers”/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns.

Pros

communicates return type clearly
manual handler registration and return type marshaling is no longer needed, just implement the interface and register it
communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of #7093 approach (1) if we chose to adopt that
generated client code could be useful for clients and tests
dramatically reduces and simplifies the code

Cons

using service definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec)

​变更记录

​状态

​摘要

​背景

​决策

​编码

​解码

​路由

​模块配置

​Msg 服务实现

​影响

​优点

​缺点

​参考资料

​Changelog

​Status

​Abstract

​Context

​Decision

​Encoding

​Decoding

​Routing

​Module Configuration

​Msg Service Implementation

​Consequences

​Pros

​Cons

​References