变更记录
- 2022-05-04: 初始草案
- 2022-08-19: 更新
状态
已提议 已实现摘要
为了让构建 Cosmos SDK 模块和应用更容易,我们提议采用一个基于依赖注入和声明式应用配置的新应用装配系统,以替换当前的app.go 代码。
背景
有很多因素导致当前状态下的 SDK 和 SDK 应用难以维护。当前复杂性的一个表征是simapp/app.go,它包含了近 100 行 import,除此之外还有 600 多行大多为样板代码的内容,而这些代码通常会被复制到每个新项目中。(更不用说还会在 simapp/simd 中复制额外的样板代码。)
启动一个应用所需的大量样板代码,使得像 ADR 053: Go Module Refactoring 中描述的那样,为 Cosmos SDK 模块发布可独立版本化的 go module 变得困难。
除了非常冗长且重复之外,app.go 还暴露了大量可能产生破坏性变更的表面积,因为大多数模块通过位置参数实例化自身,这意味着只要需要新增一个参数,即使是可选参数,也会引发破坏性变更。
我们曾做过若干尝试来改进当前状况,包括 ADR 033: Internal-Module Communication 和 一个新 SDK 的概念验证。围绕这些设计的讨论最终引出了这里描述的当前解决方案。
决策
为了改善当前状况,我们设计了一种新的“应用装配”范式来替代app.go,其内容包括:
- 对应用中的模块进行声明式配置,该配置可以序列化为 JSON 或 YAML
- 一个依赖注入(DI)框架,用于根据该配置实例化应用
依赖注入
在检查app.go 中的代码时可以发现,其中大部分代码只是用由框架提供的依赖(例如 store key)或其他模块提供的依赖(例如 keeper)来实例化模块。结合上下文,正确的依赖通常是相当明确的,因此依赖注入是一个显而易见的解决方案。模块不再需要开发者手动解析依赖,而是由模块告诉 DI 容器自己需要什么依赖,再由容器决定如何提供它。
我们考察了 golang 中若干现有的 DI 解决方案,认为 uber/dig 基于反射的方法最接近我们的需求,但仍不完全满足。基于对 SDK 需求的评估,我们设计并构建了 Cosmos SDK 的 depinject 模块,它具有以下特性:
- 通过函数式构造器进行依赖解析和提供,例如:
func(need SomeDep) (AnotherDep, error) - 支持
optional依赖的依赖注入In和Out结构体 - 通过
ManyPerContainerType标签接口支持分组依赖(每个容器可有多个) - 通过
ModuleKey支持模块作用域依赖(每个模块获得一个唯一依赖) - 通过
OnePerModuleType标签接口支持每模块一个的依赖 - 通过 GraphViz 提供复杂的调试信息和容器可视化能力
StoreKey可以是一个模块作用域依赖,并且对每个模块唯一- 模块的
AppModule实例(或等价物)可以是一个OnePerModuleType - CLI 命令可以通过
ManyPerContainerType提供
声明式应用配置
为了将模块组合成一个应用,我们将使用声明式应用配置。该配置基于 protobuf,其基本结构非常简单:cosmos.bank.module.v1.Module)。模块会被赋予一个唯一的短 name,以便在同一模块的不同版本之间共享资源,即使这些版本可能使用不同的 protobuf package 版本(例如 cosmos.bank.module.v2.Module)。所有模块配置对象都应定义 cosmos.app.v1alpha1.module descriptor option,它将为框架提供额外的有用元数据,同时也可被模块注册表索引。
一个 YAML 格式的应用配置示例如下:
baseapp 模块,其中包含 begin blocker、end blocker 和 init genesis 的排序信息。这些关注点并没有被提升到模块配置层,而是由模块自身处理,这样就可以方便地替换不同版本的 baseapp(例如为了适配不同版本的 tendermint),而不需要修改其余配置。随后,baseapp 模块会向服务端框架(它某种程度上位于 ABCI 应用之外)提供一个 abci.Application 实例。
在这个模型中,一个应用就是“层层到底全是模块”,而依赖注入/应用配置层在很大程度上与协议无关,甚至可以适配协议层上的重大破坏性变更。
模块与 Protobuf 注册
为了让依赖注入和声明式配置这两个组件能够如上所述协同工作,我们需要一种方式让模块真正完成自注册,并向容器提供依赖。 这一层还需要处理一个额外复杂点,即 protobuf 注册表初始化。回顾当前 SDKcodec 和提议中的 ADR 054: Protobuf Semver Compatible Codegen,protobuf 类型都需要显式注册。由于应用配置本身是基于 protobuf 的,并且使用 protobuf Any 类型,因此 protobuf 注册必须在应用配置自身被解码之前完成。由于我们无法预先知道会需要哪些 protobuf Any 类型,而这些类型又由模块自行定义,因此我们需要分阶段解码应用配置:
- 将应用配置 JSON/YAML 解析为原始 JSON,并收集所需模块的 type URL(不进行 proto JSON 解码)
- 基于每个所需模块提供的文件描述符和类型,构建一个 protobuf 类型注册表
- 使用该 protobuf 类型注册表,将应用配置按 proto JSON 进行解码
internal 生成代码,因此这里的代码应提供一种替代方式,将 protobuf 类型注册到类型注册表中。类似于当前 .pb.go 文件会为 foo.proto 文件生成 var File_foo_proto protoreflect.FileDescriptor,生成代码还应新增一个成员 var Types_foo_proto TypeInfo,其中 TypeInfo 是一个接口或结构体,包含注册 protobuf 生成类型和文件描述符所需的全部信息。
因此,一个模块必须提供依赖注入 provider 和 protobuf 类型,并以其模块配置对象作为输入,而该配置对象会基于其 type URL 唯一标识该模块。
基于这一点,我们定义了一个全局模块注册表,允许模块实现通过以下 API 完成注册:
*appmodule.Handler 类型是对旧版 AppModule 框架的替代,详见 ADR 063: Core Module API。
新的 app.go
基于这一设置,app.go 现在可能会像下面这样:
在现有 SDK 模块中的应用
到目前为止,我们描述的是一个在很大程度上不依赖 SDK 具体细节的系统,例如 store key、AppModule、BaseApp 等。对这些框架部分的改进,以及它们如何与这里定义的通用应用装配框架集成,见 ADR 063: Core Module API。
跨模块 Hook 的注册
跨模块 Hook 的注册
一些模块会定义一个 hooks 接口(例如StakingHooks),这样当某些事件发生时,一个模块就可以回调另一个模块。
在应用 wiring 框架下,这些 hooks 接口可以定义为 OnePerModuleType,随后消费这些 hooks 的模块可以将它们收集为一个从模块名到 hook 类型的映射(例如 map[string]FooHooks)。例如:
- 需要将所有模块的全部 keeper 暴露给提供 hooks 的模块
- 无法将 hooks 封装在另一个类型上,而该类型又不暴露全部 keeper 方法
- 更难在静态上知道哪些模块暴露了 hooks,或哪些模块在检查这些 hooks
depinject 代码生成(如下所述),hooks 的注册将在 app.go 中清晰可见。
代码生成
depinject 框架将可选地支持对应用配置和依赖注入 wiring 进行代码生成。这将带来:
- 可以像查看现有
app.go一样,将依赖注入 wiring 作为常规 Go 代码进行检查 - 依赖注入是按需启用的,仍然 100% 可以使用手动 wiring
模块语义版本控制
当我们开始创建采用语义化版本控制、并且以独立 Go module 形式存在的 SDK 模块时,某个模块的状态机破坏性变更应按如下方式处理:- 语义化主版本号应递增
- 应创建一个新的、带语义化版本的模块配置 protobuf 类型
github.com/cosmos/cosmos-sdk/x/bank,模块配置类型为 cosmos.bank.module.v1.Module,并且我们希望对该模块做一次状态机破坏性变更,那么我们应当:
- 创建一个新的 Go module
github.com/cosmos/cosmos-sdk/x/bank/v2 - 并使用模块配置 protobuf 类型
cosmos.bank.module.v2.Module
cosmos.bank.v1,但 github.com/cosmos/cosmos-sdk/x/bank/v2 将是一个独立的 Go module,并拥有独立的模块配置类型。
这种做法最终将允许我们通过配置变更,使用 appconfig 加载某个模块的新版本。
实际上,带语义化版本的 Go module 与带版本的模块配置 protobuf 类型之间应当保持 1:1 对应关系,并且每当模块发生状态机破坏性变更时,都应进行主版本号升级。
注意:在 ADR 054:模块语义版本控制 中描述的相关问题得到解决之前,作为独立 Go module 的 SDK 模块不应采用语义化版本控制。这个问题的短期解决方案仍然没有被完全明确。不过,最简单的策略很可能是使用独立的 API Go module,并遵循此评论中描述的指导原则:链接。在此之前,建议 Cosmos SDK 模块继续沿用经过验证的 基于 0 的版本控制,直到官方提供推荐方案。届时,本 ADR 的这一部分会更新;目前,这一部分应被视为未来采用语义化版本控制的设计建议。
影响
向后兼容性
适配新应用 wiring 系统的模块,无需放弃其现有的AppModule 和 NewKeeper 注册范式。这两种方式可以并存,持续到不再需要为止。
正面影响
- 新应用的 wiring 会更简单、更精炼,也更不容易出错
- 开发和测试独立 SDK 模块会更容易,而无需复制整个 simapp
- 借助这一机制,可能可以动态加载模块并升级链,而不需要通过协调停机和二进制升级来完成
- 更容易集成插件
- 依赖注入框架能够为项目中的依赖关系提供更多自动化推理能力,并支持图可视化
负面影响
- 当某个依赖缺失时,可能会让人困惑,不过错误信息、GraphViz 可视化以及全局模块注册可能对此有所帮助
中性影响
- 这将需要一定的工作和教育成本
后续讨论
本 ADR 中描述的 protobuf 类型注册系统尚未实现,并且可能需要结合代码生成重新审视。使用 DI provider 来完成这类类型注册也许会更好。参考资料
Changelog
- 2022-05-04: Initial Draft
- 2022-08-19: Updates
Status
PROPOSED ImplementedAbstract
In order to make it easier to build Cosmos SDK modules and apps, we propose a new app wiring system based on dependency injection and declarative app configurations to replace the currentapp.go code.
Context
A number of factors have made the SDK and SDK apps in their current state hard to maintain. A symptom of the current state of complexity issimapp/app.go
which contains almost 100 lines of imports and is otherwise over 600 lines of mostly boilerplate code that is
generally copied to each new project. (Not to mention the additional boilerplate which gets copied in simapp/simd.)
The large amount of boilerplate needed to bootstrap an app has made it hard to release independently versioned go
modules for Cosmos SDK modules as described in ADR 053: Go Module Refactoring.
In addition to being very verbose and repetitive, app.go also exposes a large surface area for breaking changes
as most modules instantiate themselves with positional parameters which forces breaking changes anytime a new parameter
(even an optional one) is needed.
Several attempts were made to improve the current situation including ADR 033: Internal-Module Communication
and a proof-of-concept of a new SDK. The discussions around these
designs led to the current solution described here.
Decision
In order to improve the current situation, a new “app wiring” paradigm has been designed to replaceapp.go which
involves:
- declaration configuration of the modules in an app which can be serialized to JSON or YAML
- a dependency-injection (DI) framework for instantiating apps from the that configuration
Dependency Injection
When examining the code inapp.go most of the code simply instantiates modules with dependencies provided either
by the framework (such as store keys) or by other modules (such as keepers). It is generally pretty obvious given
the context what the correct dependencies actually should be, so dependency-injection is an obvious solution. Rather
than making developers manually resolve dependencies, a module will tell the DI container what dependency it needs
and the container will figure out how to provide it.
We explored several existing DI solutions in golang and felt that the reflection-based approach in uber/dig
was closest to what we needed but not quite there. Assessing what we needed for the SDK, we designed and built
the Cosmos SDK depinject module, which has the following
features:
- dependency resolution and provision through functional constructors, ex:
func(need SomeDep) (AnotherDep, error) - dependency injection
InandOutstructs which supportoptionaldependencies - grouped-dependencies (many-per-container) through the
ManyPerContainerTypetag interface - module-scoped dependencies via
ModuleKeys (where each module gets a unique dependency) - one-per-module dependencies through the
OnePerModuleTypetag interface - sophisticated debugging information and container visualization via GraphViz
StoreKeycould be a module-scoped dependency which is unique per module- a module’s
AppModuleinstance (or the equivalent) could be aOnePerModuleType - CLI commands could be provided with
ManyPerContainerTypes
Declarative App Config
In order to compose modules into an app, a declarative app configuration will be used. This configuration is based off of protobuf and its basic structure is very simple:cosmos.bank.module.v1.Module). Modules are given a unique short name
to share resources across different versions of the same module which might have a different protobuf package
versions (ex. cosmos.bank.module.v2.Module). All module config objects should define the cosmos.app.v1alpha1.module
descriptor option which will provide additional useful metadata for the framework and which can also be indexed
in module registries.
An example app config in YAML might look like this:
baseapp module which contains the information around ordering of
begin blockers, end blockers, and init genesis. Rather than lifting these concerns up to the module config layer,
they are themselves handled by a module which could allow a convenient way of swapping out different versions of
baseapp (for instance to target different versions of tendermint), without needing to change the rest of the config.
The baseapp module would then provide to the server framework (which sort of sits outside the ABCI app) an instance
of abci.Application.
In this model, an app is modules all the way down and the dependency injection/app config layer is very much
protocol-agnostic and can adapt to even major breaking changes at the protocol layer.
Module & Protobuf Registration
In order for the two components of dependency injection and declarative configuration to work together as described, we need a way for modules to actually register themselves and provide dependencies to the container. One additional complexity that needs to be handled at this layer is protobuf registry initialization. Recall that in both the current SDKcodec and the proposed ADR 054: Protobuf Semver Compatible Codegen,
protobuf types need to be explicitly registered. Given that the app config itself is based on protobuf and
uses protobuf Any types, protobuf registration needs to happen before the app config itself can be decoded. Because
we don’t know which protobuf Any types will be needed a priori and modules themselves define those types, we need
to decode the app config in separate phases:
- parse app config JSON/YAML as raw JSON and collect required module type URLs (without doing proto JSON decoding)
- build a protobuf type registry based on file descriptors and types provided by each required module
- decode the app config as proto JSON using the protobuf type registry
internal generated code which is not registered with the global protobuf registry, this code should provide
an alternate way to register protobuf types with a type registry. In the same way that .pb.go files currently have a
var File_foo_proto protoreflect.FileDescriptor for the file foo.proto, generated code should have a new member
var Types_foo_proto TypeInfo where TypeInfo is an interface or struct with all the necessary info to register both
the protobuf generated types and file descriptor.
So a module must provide dependency injection providers and protobuf types, and takes as input its module
config object which uniquely identifies the module based on its type URL.
With this in mind, we define a global module register which allows module implementations to register themselves
with the following API:
*appmodule.Handler type referenced above is a replacement for the legacy AppModule framework, and
described in ADR 063: Core Module API.
New app.go
With this setup, app.go might now look something like this:
Application to existing SDK modules
So far we have described a system which is largely agnostic to the specifics of the SDK such as store keys,AppModule,
BaseApp, etc. Improvements to these parts of the framework that integrate with the general app wiring framework
defined here are described in ADR 063: Core Module API.
Registration of Inter-Module Hooks
Registration of Inter-Module Hooks
Some modules define a hooks interface (ex.StakingHooks) which allows one module to call back into another module
when certain events happen.
With the app wiring framework, these hooks interfaces can be defined as a OnePerModuleTypes and then the module
which consumes these hooks can collect these hooks as a map of module name to hook type (ex. map[string]FooHooks). Ex:
- needing to expose all the keepers of all modules to the module providing hooks,
- not allowing for encapsulating hooks on a different type which doesn’t expose all keeper methods,
- harder to know statically which module expose hooks or are checking for them.
app.go if depinject codegen
(described below) is used.
Code Generation
Thedepinject framework will optionally allow the app configuration and dependency injection wiring to be code
generated. This will allow:
- dependency injection wiring to be inspected as regular go code just like the existing
app.go, - dependency injection to be opt-in with manual wiring 100% still possible.
Module Semantic Versioning
When we start creating semantically versioned SDK modules that are in standalone go modules, a state machine breaking change to a module should be handled as follows:- the semantic major version should be incremented, and
- a new semantically versioned module config protobuf type should be created.
github.com/cosmos/cosmos-sdk/x/bank with the module config type
cosmos.bank.module.v1.Module, and we want to make a state machine breaking change to the module, we would:
- create a new go module
github.com/cosmos/cosmos-sdk/x/bank/v2, - with the module config protobuf type
cosmos.bank.module.v2.Module.
cosmos.bank.v1, but github.com/cosmos/cosmos-sdk/x/bank/v2 will be a separate go module with a separate module config type.
This practice will eventually allow us to use appconfig to load new versions of a module via a configuration change.
Effectively, there should be a 1:1 correspondence between a semantically versioned go module and a
versioned module config protobuf type, and major versioning bumps should occur whenever state machine breaking changes
are made to a module.
NOTE: SDK modules that are standalone go modules should not adopt semantic versioning until the concerns described in
ADR 054: Module Semantic Versioning are
addressed. The short-term solution for this issue was left somewhat unresolved. However, the easiest tactic is
likely to use a standalone API go module and follow the guidelines described in this comment: Link. For the time-being, it is recommended that
Cosmos SDK modules continue to follow tried and true 0-based versioning until an officially
recommended solution is provided. This section of the ADR will be updated when that happens and for now, this section
should be considered as a design recommendation for future adoption of semantic versioning.
Consequences
Backwards Compatibility
Modules which work with the new app wiring system do not need to drop their existingAppModule and NewKeeper
registration paradigms. These two methods can live side-by-side for as long as is needed.
Positive
- wiring up new apps will be simpler, more succinct and less error-prone
- it will be easier to develop and test standalone SDK modules without needing to replicate all of simapp
- it may be possible to dynamically load modules and upgrade chains without needing to do a coordinated stop and binary upgrade using this mechanism
- easier plugin integration
- dependency injection framework provides more automated reasoning about dependencies in the project, with a graph visualization.
Negative
- it may be confusing when a dependency is missing although error messages, the GraphViz visualization, and global module registration may help with that
Neutral
- it will require work and education