ADR 033: Protobuf-based Inter-Module Communication

变更记录

2020-10-05：初始草案

状态

提议中

摘要

本 ADR 引入了一套带权限控制的模块间通信系统，利用 ADR 021 和 ADR 031 中定义的 protobuf Query 与 Msg 服务定义，提供：

基于 protobuf 的稳定模块接口，未来有可能取代 keeper 范式
更强的模块间对象能力（OCAP）保证
模块账户与子账户授权

背景

当前 Cosmos SDK 关于对象能力模型的文档中写道：

我们假设，一个繁荣的 Cosmos SDK 模块生态系统若能被轻松组合进区块链应用中，其中将包含存在缺陷或恶意的模块。

目前还没有一个繁荣的 Cosmos SDK 模块生态。我们推测这部分原因在于：

缺少一个稳定的 v1.0 Cosmos SDK 供模块构建。模块接口会在一个点版本到下一个点版本之间发生变化，有时甚至变化很大，尽管这通常有其合理原因，但这并不能形成一个稳定的构建基础。
缺少一个被正确实现的对象能力系统，甚至缺少一个面向对象的封装系统，这使得对模块 keeper 接口的重构不可避免，因为当前接口的约束性太差。

`x/bank` 案例研究

目前，x/bank keeper 几乎向任何引用它的模块开放了不受限制的访问权限。例如， SetBalance 方法允许调用方将任何账户的余额设置为任意值，甚至绕过对供应量的正确跟踪。后来似乎有人尝试通过模块级的铸币、质押和销毁权限，来实现某种 OCAP 雏形。这些权限允许模块以模块自身账户的名义铸造、销毁或委托代币。这些权限实际上以状态中 ModuleAccount 类型上的 []string 数组形式存储。然而，这些权限实际上并没有发挥太大作用。它们控制哪些模块可以在 MintCoins、 BurnCoins 和 DelegateCoins*** 方法中被引用，但首先，控制访问的并不是唯一的对象能力令牌，而只是一个简单的字符串。因此，x/upgrade 模块只需调用 MintCoins("staking")，就可以为 x/staking 模块铸造代币。此外，所有能够访问这些 keeper 方法的模块，同样也能访问 SetBalance，这使其他任何 OCAP 尝试都失去意义，甚至破坏了最基本的面向对象封装。

决策

基于 ADR-021 和 ADR-031，我们引入了用于安全模块授权与 OCAP 的模块间通信框架。在实现后，它也可以作为当前模块之间传递 keeper 这一现有范式的替代方案。这里概述的方法旨在构成 Cosmos SDK v1.0 的基础，为繁荣的模块生态出现提供必要的稳定性与封装保证。特别需要说明的是，此项决策是启用这项能力，是否采用由各模块自行决定。将现有模块迁移到这一新范式的提案，需要作为单独的话题讨论，可能会以对此 ADR 的修订形式处理。

新的 “Keeper” 范式

在 ADR 021 中，引入了使用 protobuf 服务定义来定义查询器的机制；在 ADR 31 中，又增加了使用 protobuf 服务来定义 Msg 的机制。 Protobuf 服务定义会生成两个 Go 接口，分别表示服务的客户端和服务端，再加上一些辅助代码。下面是银行 cosmos.bank.Msg/Send 消息类型的一个最小示例：

package bank

type MsgClient interface {
    Send(context.Context, *MsgSend, opts ...grpc.CallOption) (*MsgSendResponse, error)
}

type MsgServer interface {
    Send(context.Context, *MsgSend) (*MsgSendResponse, error)
}

ADR 021 和 ADR 31 说明了模块如何分别实现生成的 QueryServer 和 MsgServer 接口，以替代旧版查询器和 Msg 处理器。在本 ADR 中，我们解释了模块如何使用生成的 QueryClient 和 MsgClient 接口向其他模块发起查询和发送 Msg，并提议将这一机制作为现有 Keeper 范式的替代方案。需要明确的是，本 ADR 并不要求创建新的 protobuf 定义或服务。相反，它利用的是客户端已经用于模块间通信的同一套基于 proto 的服务接口。与向外部模块暴露 keeper 相比，使用这种 QueryClient/MsgClient 方法有以下关键优势：

Protobuf 类型会通过 buf 检查破坏性变更，并且由于 protobuf 的设计方式，这将为我们提供强有力的向后兼容保证，同时允许向前演进。
客户端与服务端接口的分离，使我们能够在两者之间插入权限检查代码，用于检查某个模块是否有权向另一个模块发送指定的 Msg，从而提供一个合适的对象能力系统（见下文）。
模块间通信的路由器为我们提供了一个便于处理事务回滚的位置，从而实现操作的原子性（目前这是一个问题）。任何模块到模块调用中的失败，都会导致整个事务失败。

这一机制还带来以下额外收益：

通过代码生成减少样板代码，以及
允许使用其他语言编写模块，例如通过 CosmWasm 这样的 VM，或通过使用 gRPC 的子进程

模块间通信

要使用 protobuf 编译器生成的 Client，我们需要一个 grpc.ClientConn 接口实现。为此，我们引入了一种新类型 ModuleKey，它实现了 grpc.ClientConn 接口。ModuleKey 可以被视为与模块账户对应的“private key”，其认证通过一个特殊的 Invoker() 函数来提供，更多细节见下文。区块链用户（外部客户端）使用其账户的私钥来签署包含 Msg 的交易，在这些交易中他们会被列为签名者（每条消息通过 Msg.GetSigner 指定所需签名者）。认证检查由 AnteHandler 执行。在这里，我们扩展了这一过程，允许在 Msg.GetSigners 中标识模块。当一个模块想要触发另一个模块中某个 Msg 的执行时，它的 ModuleKey 会充当发送者（通过我们下文描述的 ClientConn 接口），并被设置为唯一的“签名者”。值得注意的是，在这种情况下我们不会使用任何加密签名。例如，模块 A 可以使用其 A.ModuleKey 为 /cosmos.bank.Msg/Send 交易创建 MsgSend 对象。MsgSend 的校验会确保 from 账户（此处即 A.ModuleKey）就是签名者。下面是一个假设中的模块 foo 与 x/bank 交互的示例：

package foo

type FooMsgServer {
  // ...

  bankQuery bank.QueryClient
  bankMsg   bank.MsgClient
}

func NewFooMsgServer(moduleKey RootModuleKey, ...)

FooMsgServer {
  // ...

  return FooMsgServer {
    // ...
    modouleKey: moduleKey,
    bankQuery: bank.NewQueryClient(moduleKey),
    bankMsg: bank.NewMsgClient(moduleKey),
}
}

func (foo *FooMsgServer)

Bar(ctx context.Context, req *MsgBarRequest) (*MsgBarResponse, error) {
    balance, err := foo.bankQuery.Balance(&bank.QueryBalanceRequest{
    Address: fooMsgServer.moduleKey.Address(),
    Denom: "foo"
})

  ...

  res, err := foo.bankMsg.Send(ctx, &bank.MsgSendRequest{
    FromAddress: fooMsgServer.moduleKey.Address(), ...
})

  ...
}

这一设计也旨在具备可扩展性，以覆盖更细粒度的权限控制使用场景，例如按 denom 前缀限制只有特定模块可以铸币（如 #7459 中所讨论）。

`ModuleKey` 与 `ModuleID`

ModuleKey 可以看作模块账户的“私钥”，而 ModuleID 可以看作对应的“公钥”。根据 ADR 028，模块既可以拥有一个根模块账户，也可以拥有任意数量的子账户或派生账户，用于不同的资金池（例如质押池）或托管账户（例如群组账户）。我们也可以将模块子账户理解为类似派生密钥的机制：存在一个根密钥，然后再附加某种派生路径。ModuleID 是一个简单结构体，包含模块名以及可选的“派生”路径，并基于 ADR-028 中的 AddressHash 方法形成其地址：

type ModuleID struct {
    ModuleName string
  Path []byte
}

func (key ModuleID)

Address() []byte {
    return AddressHash(key.ModuleName, key.Path)
}

除了能够生成 ModuleID 和地址之外，ModuleKey 还包含一个名为 Invoker 的特殊函数，它是实现安全模块间访问的关键。Invoker 会创建一个 InvokeFn 闭包，该闭包在 grpc.ClientConn 接口中作为 Invoke 方法使用，并且在底层能够将消息路由到合适的 Msg 和 Query 处理器，同时对 Msg 执行适当的安全检查。这使得模块间访问甚至比 keeper 更安全，因为 keeper 的私有成员变量可能通过反射被操纵。Golang 不支持对函数闭包捕获的变量进行反射；若恶意模块想绕过 ModuleKey 的安全机制，就需要直接操纵内存。两种 ModuleKey 类型分别是 RootModuleKey 和 DerivedModuleKey：

type Invoker func(callInfo CallInfo)

func(ctx context.Context, request, response interface{
}, opts ...interface{
})

error

type CallInfo {
    Method string
  Caller ModuleID
}

type RootModuleKey struct {
    moduleName string
  invoker Invoker
}

func (rm RootModuleKey)

Derive(path []byte)

DerivedModuleKey { /* ... */
}

type DerivedModuleKey struct {
    moduleName string
  path []byte
  invoker Invoker
}

模块可以通过 RootModuleKey 上的 Derive(path []byte) 方法获取一个 DerivedModuleKey，然后使用这个 key 来为来自某个子账户的 Msg 完成认证。例如：

package foo

func (fooMsgServer *MsgServer)

Bar(ctx context.Context, req *MsgBar) (*MsgBarResponse, error) {
    derivedKey := fooMsgServer.moduleKey.Derive(req.SomePath)
    bankMsgClient := bank.NewMsgClient(derivedKey)

res, err := bankMsgClient.Balance(ctx, &bank.MsgSend{
    FromAddress: derivedKey.Address(), ...
})
  ...
}

通过这种方式，模块可以获得对根账户以及任意数量子账户的许可访问权，并从这些账户发送已认证的 Msg。底层会使用 Invoker 的 callInfo.Caller 参数来区分不同的模块账户，但无论哪种情况，Invoker 返回的函数都只允许来自根模块账户或派生模块账户的 Msg 通过。请注意，Invoker 本身会基于传入的 CallInfo 返回一个函数闭包。这使得未来的客户端实现可以为每种方法类型缓存 invoke 函数，从而避免哈希表查找的开销。这样可以将这种模块间通信方式的性能开销降到仅剩权限检查所必需的最低程度。再次强调，这个闭包只允许访问已授权的调用。无论如何进行名称伪装，都无法访问其他内容。下面给出 RootModuleKey 的 grpc.ClientConn.Invoke 实现草图：

func (key RootModuleKey)

Invoke(ctx context.Context, method string, args, reply interface{
}, opts ...grpc.CallOption)

error {
    f := key.invoker(CallInfo {
    Method: method,
    Caller: ModuleID {
    ModuleName: key.moduleName
}})

return f(ctx, args, reply)
}

`AppModule` 装配与依赖要求

在 ADR 031 中，引入了 AppModule.RegisterService(Configurator) 方法。为了支持模块间通信，我们扩展了 Configurator 接口，用于传入 ModuleKey，并允许模块使用 RequireServer() 指定其对其他模块的依赖：

type Configurator interface {
    MsgServer()

grpc.Server
   QueryServer()

grpc.Server

   ModuleKey()

ModuleKey
   RequireServer(msgServer interface{
})
}

ModuleKey 会在 RegisterService 方法本身中传递给模块，这样 RegisterServices 就成为配置模块服务的单一入口。这也旨在带来一个附带效果：大幅减少 app.go 中的样板代码。目前，ModuleKey 将基于 AppModuleBasic.Name() 创建，但未来可能会引入更灵活的系统。ModuleManager 将在底层负责模块账户的创建。由于模块之间不再直接互相访问，因此模块可能会存在未满足的依赖。为确保模块依赖在启动时得到解析，应增加 Configurator.RequireServer 方法。ModuleManager 会确保所有通过 RequireServer 声明的依赖都能在应用启动前被解析。比如，示例模块 foo 可以像下面这样声明其对 x/bank 的依赖：

package foo

func (am AppModule)

RegisterServices(cfg Configurator) {
    cfg.RequireServer((*bank.QueryServer)(nil))

cfg.RequireServer((*bank.MsgServer)(nil))
}

安全性考量

除了检查 ModuleKey 权限之外，底层路由基础设施还需要采取一些额外的安全预防措施。

递归与重入

递归或重入式的方法调用会带来潜在的安全威胁。如果模块 A 调用模块 B，而模块 B 又在同一次调用中再次调用模块 A，就可能出现问题。路由系统处理这一问题的一种基本方式，是维护一个调用栈，防止某个模块在调用栈中被引用超过一次，从而避免重入。路由器中的一个 map[string]interface{} 表可以用来执行这项安全检查。

查询

Cosmos SDK 中的查询通常不需要权限，因此只要采取基本预防措施，允许一个模块查询另一个模块 generally 不会带来重大安全威胁。路由系统需要采取的基本预防措施，是确保传递给查询方法的 sdk.Context 不允许写入 store。目前可以像 BaseApp 查询那样，通过 CacheMultiStore 来实现。

内部方法

在很多情况下，我们可能希望模块调用其他模块上那些完全不对客户端暴露的方法。为此，我们向 Configurator 增加 InternalServer 方法：

type Configurator interface {
    MsgServer()

grpc.Server
   QueryServer()

grpc.Server
   InternalServer()

grpc.Server
}

例如，x/slashing 的 Slash 必须调用 x/staking 的 Slash，但我们并不希望将 x/staking 的 Slash 暴露给终端用户和客户端。内部 protobuf 服务将在对应模块 proto 包中的 internal.proto 文件里定义。注册到 InternalServer 的服务可以被其他模块调用，但不能被外部客户端调用。另一种只供内部使用的方法方案，可能是采用这里讨论过的 hooks / plugins 机制。关于 hooks / plugin 系统的更详细评估，将在后续对此 ADR 的补充中，或以单独的 ADR 形式进行讨论。

授权

默认情况下，模块间路由要求消息由 GetSigners 返回的第一个 signer 发送。模块间路由还应接受授权中间件，例如 ADR 030 提供的那种。该中间件将允许账户授权特定模块账户代表其执行操作。授权中间件还应考虑某些模块需要对其他模块授予某种“管理员”权限的问题。这部分内容将通过单独的 ADR 或对此 ADR 的更新来处理。

未来工作

未来的其他改进可能包括：

自定义代码生成，用于：
- 简化接口（例如生成使用 sdk.Context 而不是 context.Context 的代码）
- 优化模块间调用，例如在首次调用后缓存已解析的方法
将 StoreKey 与 ModuleKey 合并为一个统一接口，使模块只拥有一个 OCAPs 句柄
通过代码生成进一步提升模块间通信性能
将 ModuleKey 的创建与 AppModuleBasic.Name() 解耦，以便应用可以覆盖根模块账户名称
模块间 hooks 与 plugins

替代方案

MsgServices 与 `x/capability`

x/capability 模块确实提供了一个规范的 object-capability 实现，可供 Cosmos SDK 中的任意模块使用，甚至可以像 #5931 中描述的那样用于模块间 OCAPs。本 ADR 所述方案的优势，主要体现在它与 Cosmos SDK 其他部分的集成方式上，具体包括：

protobuf，因此：
- 可以利用接口代码生成来获得更好的开发体验
- 模块接口可以使用 buf 进行版本化并检查破坏性变更
ADR 028 中定义的子模块账户
通用的 Msg 传递范式，以及 GetSigners 指定 signer 的方式

此外，这也是对 keeper 的完整替代，可应用于所有模块间通信；而 #5931 中的 x/capability 方案则需要按方法逐一应用。

影响

向后兼容性

本 ADR 旨在为实现模块之间更强的长期兼容性提供一条路径。在短期内，这很可能会导致某些过于宽松的 Keeper 接口发生破坏性变更，和/或完全替换 Keeper 接口。

正面影响

作为 keeper 的替代方案，更容易形成稳定的模块间接口
规范的模块间 OCAPs
改善模块开发者的开发体验，正如多位参与者在 Architecture Review Call, Dec 3 中所评论的那样
为大幅简化 app.go 奠定基础
路由器可以被配置为对模块到模块的调用强制执行原子事务

负面影响

采用该方案的模块将需要进行大规模重构

中性影响

测试用例 [可选]

参考资料

Changelog

2020-10-05: Initial Draft

Status

Proposed

Abstract

This ADR introduces a system for permissioned inter-module communication leveraging the protobuf Query and Msg service definitions defined in ADR 021 and ADR 031 which provides:

stable protobuf based module interfaces to potentially later replace the keeper paradigm
stronger inter-module object capabilities (OCAPs) guarantees
module accounts and sub-account authorization

Context

In the current Cosmos SDK documentation on the Object-Capability Model, it is stated that:

We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules.

There is currently not a thriving ecosystem of Cosmos SDK modules. We hypothesize that this is in part due to:

lack of a stable v1.0 Cosmos SDK to build modules off of. Module interfaces are changing, sometimes dramatically, from point release to point release, often for good reasons, but this does not create a stable foundation to build on.
lack of a properly implemented object capability or even object-oriented encapsulation system which makes refactors of module keeper interfaces inevitable because the current interfaces are poorly constrained.

`x/bank` Case Study

Currently the x/bank keeper gives pretty much unrestricted access to any module which references it. For instance, the SetBalance method allows the caller to set the balance of any account to anything, bypassing even proper tracking of supply. There appears to have been some later attempts to implement some semblance of OCAPs using module-level minting, staking and burning permissions. These permissions allow a module to mint, burn or delegate tokens with reference to the module’s own account. These permissions are actually stored as a []string array on the ModuleAccount type in state. However, these permissions don’t really do much. They control what modules can be referenced in the MintCoins, BurnCoins and DelegateCoins*** methods, but for one there is no unique object capability token that controls access — just a simple string. So the x/upgrade module could mint tokens for the x/staking module simple by calling MintCoins(“staking”). Furthermore, all modules which have access to these keeper methods, also have access to SetBalance negating any other attempt at OCAPs and breaking even basic object-oriented encapsulation.

Decision

Based on ADR-021 and ADR-031, we introduce the Inter-Module Communication framework for secure module authorization and OCAPs. When implemented, this could also serve as an alternative to the existing paradigm of passing keepers between modules. The approach outlined here-in is intended to form the basis of a Cosmos SDK v1.0 that provides the necessary stability and encapsulation guarantees that allow a thriving module ecosystem to emerge. Of particular note — the decision is to enable this functionality for modules to adopt at their own discretion. Proposals to migrate existing modules to this new paradigm will have to be a separate conversation, potentially addressed as amendments to this ADR.

New “Keeper” Paradigm

In ADR 021, a mechanism for using protobuf service definitions to define queriers was introduced and in ADR 31, a mechanism for using protobuf service to define Msgs was added. Protobuf service definitions generate two golang interfaces representing the client and server sides of a service plus some helper code. Here is a minimal example for the bank cosmos.bank.Msg/Send message type:

package bank

type MsgClient interface {
    Send(context.Context, *MsgSend, opts ...grpc.CallOption) (*MsgSendResponse, error)
}

type MsgServer interface {
    Send(context.Context, *MsgSend) (*MsgSendResponse, error)
}

ADR 021 and ADR 31 specifies how modules can implement the generated QueryServer and MsgServer interfaces as replacements for the legacy queriers and Msg handlers respectively. In this ADR we explain how modules can make queries and send Msgs to other modules using the generated QueryClient and MsgClient interfaces and propose this mechanism as a replacement for the existing Keeper paradigm. To be clear, this ADR does not necessitate the creation of new protobuf definitions or services. Rather, it leverages the same proto based service interfaces already used by clients for inter-module communication. Using this QueryClient/MsgClient approach has the following key benefits over exposing keepers to external modules:

Protobuf types are checked for breaking changes using buf and because of the way protobuf is designed this will give us strong backwards compatibility guarantees while allowing for forward evolution.
The separation between the client and server interfaces will allow us to insert permission checking code in between the two which checks if one module is authorized to send the specified Msg to the other module providing a proper object capability system (see below).
The router for inter-module communication gives us a convenient place to handle rollback of transactions, enabling atomicy of operations (currently a problem). Any failure within a module-to-module call would result in a failure of the entire transaction

This mechanism has the added benefits of:

reducing boilerplate through code generation, and
allowing for modules in other languages either via a VM like CosmWasm or sub-processes using gRPC

Inter-module Communication

To use the Client generated by the protobuf compiler we need a grpc.ClientConn interface implementation. For this we introduce a new type, ModuleKey, which implements the grpc.ClientConn interface. ModuleKey can be thought of as the “private key” corresponding to a module account, where authentication is provided through use of a special Invoker() function, described in more detail below. Blockchain users (external clients) use their account’s private key to sign transactions containing Msgs where they are listed as signers (each message specifies required signers with Msg.GetSigner). The authentication checks is performed by AnteHandler. Here, we extend this process, by allowing modules to be identified in Msg.GetSigners. When a module wants to trigger the execution a Msg in another module, its ModuleKey acts as the sender (through the ClientConn interface we describe below) and is set as a sole “signer”. It’s worth to note that we don’t use any cryptographic signature in this case. For example, module A could use its A.ModuleKey to create MsgSend object for /cosmos.bank.Msg/Send transaction. MsgSend validation will assure that the from account (A.ModuleKey in this case) is the signer. Here’s an example of a hypothetical module foo interacting with x/bank:

package foo

type FooMsgServer {
  // ...

  bankQuery bank.QueryClient
  bankMsg   bank.MsgClient
}

func NewFooMsgServer(moduleKey RootModuleKey, ...)

FooMsgServer {
  // ...

  return FooMsgServer {
    // ...
    modouleKey: moduleKey,
    bankQuery: bank.NewQueryClient(moduleKey),
    bankMsg: bank.NewMsgClient(moduleKey),
}
}

func (foo *FooMsgServer)

Bar(ctx context.Context, req *MsgBarRequest) (*MsgBarResponse, error) {
    balance, err := foo.bankQuery.Balance(&bank.QueryBalanceRequest{
    Address: fooMsgServer.moduleKey.Address(),
    Denom: "foo"
})

  ...

  res, err := foo.bankMsg.Send(ctx, &bank.MsgSendRequest{
    FromAddress: fooMsgServer.moduleKey.Address(), ...
})

  ...
}

This design is also intended to be extensible to cover use cases of more fine grained permissioning like minting by denom prefix being restricted to certain modules (as discussed in #7459).

`ModuleKey`s and `ModuleID`s

A ModuleKey can be thought of as a “private key” for a module account and a ModuleID can be thought of as the corresponding “public key”. From the ADR 028, modules can have both a root module account and any number of sub-accounts or derived accounts that can be used for different pools (ex. staking pools) or managed accounts (ex. group accounts). We can also think of module sub-accounts as similar to derived keys - there is a root key and then some derivation path. ModuleID is a simple struct which contains the module name and optional “derivation” path, and forms its address based on the AddressHash method from the ADR-028:

type ModuleID struct {
    ModuleName string
  Path []byte
}

func (key ModuleID)

Address() []byte {
    return AddressHash(key.ModuleName, key.Path)
}

In addition to being able to generate a ModuleID and address, a ModuleKey contains a special function called Invoker which is the key to safe inter-module access. The Invoker creates an InvokeFn closure which is used as an Invoke method in the grpc.ClientConn interface and under the hood is able to route messages to the appropriate Msg and Query handlers performing appropriate security checks on Msgs. This allows for even safer inter-module access than keeper’s whose private member variables could be manipulated through reflection. Golang does not support reflection on a function closure’s captured variables and direct manipulation of memory would be needed for a truly malicious module to bypass the ModuleKey security. The two ModuleKey types are RootModuleKey and DerivedModuleKey:

type Invoker func(callInfo CallInfo)

func(ctx context.Context, request, response interface{
}, opts ...interface{
})

error

type CallInfo {
    Method string
  Caller ModuleID
}

type RootModuleKey struct {
    moduleName string
  invoker Invoker
}

func (rm RootModuleKey)

Derive(path []byte)

DerivedModuleKey { /* ... */
}

type DerivedModuleKey struct {
    moduleName string
  path []byte
  invoker Invoker
}

A module can get access to a DerivedModuleKey, using the Derive(path []byte) method on RootModuleKey and then would use this key to authenticate Msgs from a sub-account. Ex:

package foo

func (fooMsgServer *MsgServer)

Bar(ctx context.Context, req *MsgBar) (*MsgBarResponse, error) {
    derivedKey := fooMsgServer.moduleKey.Derive(req.SomePath)
    bankMsgClient := bank.NewMsgClient(derivedKey)

res, err := bankMsgClient.Balance(ctx, &bank.MsgSend{
    FromAddress: derivedKey.Address(), ...
})
  ...
}

In this way, a module can gain permissioned access to a root account and any number of sub-accounts and send authenticated Msgs from these accounts. The Invoker callInfo.Caller parameter is used under the hood to distinguish between different module accounts, but either way the function returned by Invoker only allows Msgs from either the root or a derived module account to pass through. Note that Invoker itself returns a function closure based on the CallInfo passed in. This will allow client implementations in the future that cache the invoke function for each method type avoiding the overhead of hash table lookup. This would reduce the performance overhead of this inter-module communication method to the bare minimum required for checking permissions. To re-iterate, the closure only allows access to authorized calls. There is no access to anything else regardless of any name impersonation. Below is a rough sketch of the implementation of grpc.ClientConn.Invoke for RootModuleKey:

func (key RootModuleKey)

Invoke(ctx context.Context, method string, args, reply interface{
}, opts ...grpc.CallOption)

error {
    f := key.invoker(CallInfo {
    Method: method,
    Caller: ModuleID {
    ModuleName: key.moduleName
}})

return f(ctx, args, reply)
}

`AppModule` Wiring and Requirements

In ADR 031, the AppModule.RegisterService(Configurator) method was introduced. To support inter-module communication, we extend the Configurator interface to pass in the ModuleKey and to allow modules to specify their dependencies on other modules using RequireServer():

type Configurator interface {
    MsgServer()

grpc.Server
   QueryServer()

grpc.Server

   ModuleKey()

ModuleKey
   RequireServer(msgServer interface{
})
}

The ModuleKey is passed to modules in the RegisterService method itself so that RegisterServices serves as a single entry point for configuring module services. This is intended to also have the side-effect of greatly reducing boilerplate in app.go. For now, ModuleKeys will be created based on AppModuleBasic.Name(), but a more flexible system may be introduced in the future. The ModuleManager will handle creation of module accounts behind the scenes. Because modules do not get direct access to each other anymore, modules may have unfulfilled dependencies. To make sure that module dependencies are resolved at startup, the Configurator.RequireServer method should be added. The ModuleManager will make sure that all dependencies declared with RequireServer can be resolved before the app starts. An example module foo could declare it’s dependency on x/bank like this:

package foo

func (am AppModule)

RegisterServices(cfg Configurator) {
    cfg.RequireServer((*bank.QueryServer)(nil))

cfg.RequireServer((*bank.MsgServer)(nil))
}

Security Considerations

In addition to checking for ModuleKey permissions, a few additional security precautions will need to be taken by the underlying router infrastructure.

Recursion and Re-entry

Recursive or re-entrant method invocations pose a potential security threat. This can be a problem if Module A calls Module B and Module B calls module A again in the same call. One basic way for the router system to deal with this is to maintain a call stack which prevents a module from being referenced more than once in the call stack so that there is no re-entry. A map[string]interface{} table in the router could be used to perform this security check.

Queries

Queries in Cosmos SDK are generally un-permissioned so allowing one module to query another module should not pose any major security threats assuming basic precautions are taken. The basic precaution that the router system will need to take is making sure that the sdk.Context passed to query methods does not allow writing to the store. This can be done for now with a CacheMultiStore as is currently done for BaseApp queries.

Internal Methods

In many cases, we may wish for modules to call methods on other modules which are not exposed to clients at all. For this purpose, we add the InternalServer method to Configurator:

type Configurator interface {
    MsgServer()

grpc.Server
   QueryServer()

grpc.Server
   InternalServer()

grpc.Server
}

As an example, x/slashing’s Slash must call x/staking’s Slash, but we don’t want to expose x/staking’s Slash to end users and clients. Internal protobuf services will be defined in a corresponding internal.proto file in the given module’s proto package. Services registered against InternalServer will be callable from other modules but not by external clients. An alternative solution to internal-only methods could involve hooks / plugins as discussed here. A more detailed evaluation of a hooks / plugin system will be addressed later in follow-ups to this ADR or as a separate ADR.

Authorization

By default, the inter-module router requires that messages are sent by the first signer returned by GetSigners. The inter-module router should also accept authorization middleware such as that provided by ADR 030. This middleware will allow accounts to otherwise specific module accounts to perform actions on their behalf. Authorization middleware should take into account the need to grant certain modules effectively “admin” privileges to other modules. This will be addressed in separate ADRs or updates to this ADR.

Future Work

Other future improvements may include:

custom code generation that:
- simplifies interfaces (ex. generates code with sdk.Context instead of context.Context)
- optimizes inter-module calls - for instance caching resolved methods after first invocation
combining StoreKeys and ModuleKeys into a single interface so that modules have a single OCAPs handle
code generation which makes inter-module communication more performant
decoupling ModuleKey creation from AppModuleBasic.Name() so that app’s can override root module account names
inter-module hooks and plugins

Alternatives

MsgServices vs `x/capability`

The x/capability module does provide a proper object-capability implementation that can be used by any module in the Cosmos SDK and could even be used for inter-module OCAPs as described in #5931. The advantages of the approach described in this ADR are mostly around how it integrates with other parts of the Cosmos SDK, specifically:

protobuf so that:
- code generation of interfaces can be leveraged for a better dev UX
- module interfaces are versioned and checked for breakage using buf
sub-module accounts as per ADR 028
the general Msg passing paradigm and the way signers are specified by GetSigners

Also, this is a complete replacement for keepers and could be applied to all inter-module communication whereas the x/capability approach in #5931 would need to be applied method by method.

Consequences

Backwards Compatibility

This ADR is intended to provide a pathway to a scenario where there is greater long term compatibility between modules. In the short-term, this will likely result in breaking certain Keeper interfaces which are too permissive and/or replacing Keeper interfaces altogether.

Positive

an alternative to keepers which can more easily lead to stable inter-module interfaces
proper inter-module OCAPs
improved module developer DevX, as commented on by several particpants on Architecture Review Call, Dec 3
lays the groundwork for what can be a greatly simplified app.go
router can be setup to enforce atomic transactions for module-to-module calls

​变更记录

​状态

​摘要

​背景

​x/bank 案例研究

​决策

​新的 “Keeper” 范式

​模块间通信

​ModuleKey 与 ModuleID

​AppModule 装配与依赖要求

​安全性考量

​递归与重入

​查询

​内部方法

​授权

​未来工作

​替代方案

​MsgServices 与 x/capability

​影响

​向后兼容性

​正面影响

​负面影响

​中性影响

​测试用例 [可选]

​参考资料

​Changelog

​Status

​Abstract

​Context

​x/bank Case Study

​Decision

​New “Keeper” Paradigm

​Inter-module Communication

​ModuleKeys and ModuleIDs

​AppModule Wiring and Requirements

​Security Considerations

​Recursion and Re-entry

​Queries

​Internal Methods

​Authorization

​Future Work

​Alternatives

​MsgServices vs x/capability

​Consequences

​Backwards Compatibility

​Positive

​Negative

​Neutral

​Test Cases [optional]

​References