​
变更记录

  • 2020 年 3 月 27 日：初始草案

​
状态

已接受

​
背景

本 ADR 延续了 ADR 019 和 ADR 020 中确立的动机、设计与背景。也就是说，我们的目标是为 Cosmos SDK 的客户端侧设计 Protocol Buffer 迁移路径。 本 ADR 在 ADD 020 的基础上继续展开，用于规定查询的编码方式。

​
决策

​
自定义查询定义

模块通过 protocol buffers 的 service 定义来自定义查询。这些 service 定义通常与 GRPC 协议相关并被其使用。不过，protocol buffers 规范指出，它们也可以更通用地用于任何采用 protocol buffer 编码的请求/响应协议。因此，我们可以使用 service 定义来描述自定义 ABCI 查询，甚至复用相当大一部分 GRPC 基础设施。 每个带有自定义查询的模块都应定义一个规范命名为 Query 的服务： 
// x/bank/types/types.proto

service Query {
  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { }
  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { }
}

​
接口类型的处理

使用接口类型并需要真正多态性的模块，通常会将 oneof 提升到应用层，由应用层提供该接口所支持的具体实现集合。虽然应用也完全可以对查询采用同样方式，并实现一个应用层查询服务，但更推荐模块通过 google.protobuf.Any 暴露这些接口对应的查询方法。在交易层面，一个顾虑是 Any 的开销过高，不足以支撑其使用价值。不过对于查询而言，这不是问题；提供使用 Any 的通用模块级查询，也不会妨碍应用额外提供返回应用层 oneof 的应用级查询。 对于 gov 模块，一个假设性的示例如下： 
// x/gov/types/types.proto

import "google/protobuf/any.proto";

service Query {
  rpc GetProposal(GetProposalParams) returns (AnyProposal) { }
}

message AnyProposal {
  ProposalBase base = 1;
  google.protobuf.Any content = 2;
}

​
自定义查询实现

为了实现查询服务，我们可以复用现有的 gogo protobuf grpc 插件。对于名为 Query 的服务，它会生成一个名为 QueryServer 的接口，如下所示： 
type QueryServer interface {
    QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error)

QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error)
}
我们模块中的自定义查询可通过实现该接口来完成。 这个生成接口中的第一个参数是通用的 context.Context，而 querier 方法通常需要一个 sdk.Context 实例来从 store 中读取数据。由于可以使用 WithValue 和 Value 方法向 context.Context 附加任意值，因此 Cosmos SDK 应提供一个函数 sdk.UnwrapSDKContext，用于从传入的 context.Context 中取回 sdk.Context。 以上述 bank 模块为例，QueryBalance 的实现大致如下： 
type Querier struct {
    Keeper
}

func (q Querier)

QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) {
    balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom)

return &balance, nil
}

​
自定义查询注册与路由

如上所示的查询服务实现将通过一个新的 RegisterQueryService(grpc.Server) 方法注册到 AppModule 中，其实现可以非常简单，如下： 
// x/bank/module.go
func (am AppModule)

RegisterQueryService(server grpc.Server) {
    types.RegisterQueryServer(server, keeper.Querier{
    am.keeper
})
}
在底层，现有的 baseapp.QueryRouter 将新增一个 RegisterService(sd *grpc.ServiceDesc, handler interface{}) 方法，用于将这些查询加入自定义查询路由表中（其路由方式将在下文说明）。这个方法的签名与 GRPC Server 类型上现有的 RegisterServer 方法一致，其中 handler 就是上文所述的自定义查询服务实现。 类似 GRPC 的请求会根据服务名（例如 cosmos_sdk.x.bank.v1.Query）和方法名（例如 QueryBalance），再用 / 组合成完整方法名（例如 /cosmos_sdk.x.bank.v1.Query/QueryBalance）进行路由。随后它会被转换为一个 ABCI 查询：custom/cosmos_sdk.x.bank.v1.Query/QueryBalance。通过 QueryRouter.RegisterService 注册的服务处理器，就会按这种方式进行路由。 除了方法名之外，类似 GRPC 的请求还会携带一个 protobuf 编码的负载，这与 RequestQuery.Data 可以自然对应，并接收一个 protobuf 编码的响应或错误。因此，类似 GRPC 的 rpc 方法与现有的 sdk.Query 和 QueryRouter 基础设施之间存在非常自然的映射关系。 这一基础规范使我们能够将 protocol buffer 的 service 定义复用于 ABCI 自定义查询，从而大幅减少在查询方法中手动解码和编码的需求。

​
GRPC 协议支持

除了提供一条 ABCI 查询路径之外，我们还可以很容易地提供一个 GRPC 代理服务器，在底层将 GRPC 协议请求路由为 ABCI 查询请求。这样一来，客户端就可以使用其宿主语言中现有的 GRPC 实现，基于这些 service 定义，直接对 Cosmos SDK 应用发起查询。为了让这个服务器能够工作，BaseApp 上的 QueryRouter 需要将通过 QueryRouter.RegisterService 注册的服务处理器暴露给该代理服务器实现。节点可以通过命令行参数，在与 ABCI 应用相同的进程中、使用单独端口启动该代理服务器。

​
REST 查询与 Swagger 生成

grpc-gateway 是一个项目，它通过服务方法上的特殊注解将 REST 调用转换为 GRPC 调用。希望暴露 REST 查询的模块，应当像下面示例那样，在其 rpc 方法上添加 google.api.http 注解。 
// x/bank/types/types.proto

service Query {
  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) {
    option (google.api.http) = {
      get: "/x/bank/v1/balance/{address}/{denom}"
    };
  }
  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) {
    option (google.api.http) = {
      get: "/x/bank/v1/balances/{address}"
    };
  }
}
grpc-gateway 将直接对接上文描述的 GRPC 代理，该代理会在底层把请求转换为 ABCI 查询。grpc-gateway 还可以自动生成 Swagger 定义。 在当前的 REST 查询实现中，每个模块除了 ABCI querier 方法外，还需要手动实现 REST 查询。采用 grpc-gateway 方案后，将不再需要单独生成 REST 查询处理器，只需生成上文所述的查询服务即可，因为 grpc-gateway 会负责将 protobuf 转换为 REST，同时也负责 Swagger 定义的生成。 Cosmos SDK 应为应用提供 CLI 命令，用于在独立进程中或与 ABCI 应用相同的进程中启动 GRPC gateway，同时还应提供生成 grpc-gateway 代理 .proto 文件和 swagger.json 文件的命令。

​
客户端使用

gogo protobuf 的 grpc 插件除了生成服务端接口，也会生成客户端接口。对于上文定义的 Query 服务，我们会得到一个类似如下的 QueryClient 接口： 
type QueryClient interface {
    QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error)

QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error)
}
通过对 gogo protobuf 的一个小补丁（gogo/protobuf#675），我们已经调整了 grpc 代码生成逻辑，使其为生成的客户端结构体使用接口而不是具体类型。这使我们也能够将 GRPC 基础设施复用于 ABCI 客户端查询。 Context 将新增一个 QueryConn 方法，该方法返回一个 ClientConn，用于将调用路由到 ABCI 查询。 随后，客户端（例如 CLI 方法）就可以像下面这样调用查询方法： 
clientCtx := client.NewContext()
    queryClient := types.NewQueryClient(clientCtx.QueryConn())
    params := &types.QueryBalanceParams{
    addr, denom
}

result, err := queryClient.QueryBalance(gocontext.Background(), params)

​
测试

测试可以通过一个 QueryServerTestHelper，直接基于 keeper 和 sdk.Context 引用创建查询客户端，如下所示： 
queryHelper := baseapp.NewQueryServerTestHelper(ctx)

types.RegisterQueryServer(queryHelper, keeper.Querier{
    app.BankKeeper
})
    queryClient := types.NewQueryClient(queryHelper)

​
后续改进

​
影响

​
正面

  • 大幅简化 querier 实现（无需手动编码/解码）
  • 查询客户端生成更加容易（可以使用现有 grpc 和 swagger 工具）
  • 无需再实现 REST 查询
  • 类型安全的查询方法（由 grpc 插件生成）
  • 未来查询方法的破坏性变更会更少，因为 buf 提供了向后兼容保证

​
负面

  • 所有使用现有 ABCI/REST 查询的客户端都需要重构，以适配新的 GRPC/REST 查询路径，以及 protobuf/proto-json 编码的数据；但在 protobuf 重构中，这基本上是不可避免的

​
中性

​
参考

​
Changelog

  • 2020 March 27: Initial Draft

​
Status

Accepted

​
Context

This ADR is a continuation of the motivation, design, and context established in ADR 019 and ADR 020, namely, we aim to design the Protocol Buffer migration path for the client-side of the Cosmos SDK. This ADR continues from ADD 020 to specify the encoding of queries.

​
Decision

​
Custom Query Definition

Modules define custom queries through a protocol buffers service definition. These service definitions are generally associated with and used by the GRPC protocol. However, the protocol buffers specification indicates that they can be used more generically by any request/response protocol that uses protocol buffer encoding. Thus, we can use service definitions for specifying custom ABCI queries and even reuse a substantial amount of the GRPC infrastructure. Each module with custom queries should define a service canonically named Query: 
// x/bank/types/types.proto

service Query {
  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { }
  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { }
}

​
Handling of Interface Types

Modules that use interface types and need true polymorphism generally force a oneof up to the app-level that provides the set of concrete implementations of that interface that the app supports. While app’s are welcome to do the same for queries and implement an app-level query service, it is recommended that modules provide query methods that expose these interfaces via google.protobuf.Any. There is a concern on the transaction level that the overhead of Any is too high to justify its usage. However for queries this is not a concern, and providing generic module-level queries that use Any does not preclude apps from also providing app-level queries that return use the app-level oneofs. A hypothetical example for the gov module would look something like: 
// x/gov/types/types.proto

import "google/protobuf/any.proto";

service Query {
  rpc GetProposal(GetProposalParams) returns (AnyProposal) { }
}

message AnyProposal {
  ProposalBase base = 1;
  google.protobuf.Any content = 2;
}

​
Custom Query Implementation

In order to implement the query service, we can reuse the existing gogo protobuf grpc plugin, which for a service named Query generates an interface named QueryServer as below: 
type QueryServer interface {
    QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error)

QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error)
}
The custom queries for our module are implemented by implementing this interface. The first parameter in this generated interface is a generic context.Context, whereas querier methods generally need an instance of sdk.Context to read from the store. Since arbitrary values can be attached to context.Context using the WithValue and Value methods, the Cosmos SDK should provide a function sdk.UnwrapSDKContext to retrieve the sdk.Context from the provided context.Context. An example implementation of QueryBalance for the bank module as above would look something like: 
type Querier struct {
    Keeper
}

func (q Querier)

QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) {
    balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom)

return &balance, nil
}

​
Custom Query Registration and Routing

Query server implementations as above would be registered with AppModules using a new method RegisterQueryService(grpc.Server) which could be implemented simply as below: 
// x/bank/module.go
func (am AppModule)

RegisterQueryService(server grpc.Server) {
    types.RegisterQueryServer(server, keeper.Querier{
    am.keeper
})
}
Underneath the hood, a new method RegisterService(sd *grpc.ServiceDesc, handler interface{}) will be added to the existing baseapp.QueryRouter to add the queries to the custom query routing table (with the routing method being described below). The signature for this method matches the existing RegisterServer method on the GRPC Server type where handler is the custom query server implementation described above. GRPC-like requests are routed by the service name (ex. cosmos_sdk.x.bank.v1.Query) and method name (ex. QueryBalance) combined with /s to form a full method name (ex. /cosmos_sdk.x.bank.v1.Query/QueryBalance). This gets translated into an ABCI query as custom/cosmos_sdk.x.bank.v1.Query/QueryBalance. Service handlers registered with QueryRouter.RegisterService will be routed this way. Beyond the method name, GRPC requests carry a protobuf encoded payload, which maps naturally to RequestQuery.Data, and receive a protobuf encoded response or error. Thus there is a quite natural mapping of GRPC-like rpc methods to the existing sdk.Query and QueryRouter infrastructure. This basic specification allows us to reuse protocol buffer service definitions for ABCI custom queries substantially reducing the need for manual decoding and encoding in query methods.

​
GRPC Protocol Support

In addition to providing an ABCI query pathway, we can easily provide a GRPC proxy server that routes requests in the GRPC protocol to ABCI query requests under the hood. In this way, clients could use their host languages’ existing GRPC implementations to make direct queries against Cosmos SDK app’s using these service definitions. In order for this server to work, the QueryRouter on BaseApp will need to expose the service handlers registered with QueryRouter.RegisterService to the proxy server implementation. Nodes could launch the proxy server on a separate port in the same process as the ABCI app with a command-line flag.

​
REST Queries and Swagger Generation

grpc-gateway is a project that translates REST calls into GRPC calls using special annotations on service methods. Modules that want to expose REST queries should add google.api.http annotations to their rpc methods as in this example below. 
// x/bank/types/types.proto

service Query {
  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) {
    option (google.api.http) = {
      get: "/x/bank/v1/balance/{address}/{denom}"
    };
  }
  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) {
    option (google.api.http) = {
      get: "/x/bank/v1/balances/{address}"
    };
  }
}
grpc-gateway will work direcly against the GRPC proxy described above which will translate requests to ABCI queries under the hood. grpc-gateway can also generate Swagger definitions automatically. In the current implementation of REST queries, each module needs to implement REST queries manually in addition to ABCI querier methods. Using the grpc-gateway approach, there will be no need to generate separate REST query handlers, just query servers as described above as grpc-gateway handles the translation of protobuf to REST as well as Swagger definitions. The Cosmos SDK should provide CLI commands for apps to start GRPC gateway either in a separate process or the same process as the ABCI app, as well as provide a command for generating grpc-gateway proxy .proto files and the swagger.json file.

​
Client Usage

The gogo protobuf grpc plugin generates client interfaces in addition to server interfaces. For the Query service defined above we would get a QueryClient interface like: 
type QueryClient interface {
    QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error)

QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error)
}
Via a small patch to gogo protobuf (gogo/protobuf#675) we have tweaked the grpc codegen to use an interface rather than concrete type for the generated client struct. This allows us to also reuse the GRPC infrastructure for ABCI client queries. 1Contextwill receive a new methodQueryConnthat returns aClientConn` that routes calls to ABCI queries Clients (such as CLI methods) will then be able to call query methods like this: 
clientCtx := client.NewContext()
    queryClient := types.NewQueryClient(clientCtx.QueryConn())
    params := &types.QueryBalanceParams{
    addr, denom
}

result, err := queryClient.QueryBalance(gocontext.Background(), params)

​
Testing

Tests would be able to create a query client directly from keeper and sdk.Context references using a QueryServerTestHelper as below: 
queryHelper := baseapp.NewQueryServerTestHelper(ctx)

types.RegisterQueryServer(queryHelper, keeper.Querier{
    app.BankKeeper
})
    queryClient := types.NewQueryClient(queryHelper)

​
Future Improvements

​
Consequences

​
Positive

  • greatly simplified querier implementation (no manual encoding/decoding)
  • easy query client generation (can use existing grpc and swagger tools)
  • no need for REST query implementations
  • type safe query methods (generated via grpc plugin)
  • going forward, there will be less breakage of query methods because of the backwards compatibility guarantees provided by buf

​
Negative

  • all clients using the existing ABCI/REST queries will need to be refactored for both the new GRPC/REST query paths as well as protobuf/proto-json encoded data, but this is more or less unavoidable in the protobuf refactoring

​
Neutral

​
References