​
变更日志

  • 28.06.2021：初始草案
  • 02.12.2021：为新字段添加 Since: 注释
  • 21.07.2022：移除“同一 proto 版本中不得新增 Msg”的规则。

​
状态

草案

​
摘要

本 ADR 提供了更新 Protobuf 定义时的指南和推荐实践。这些指南面向模块开发者。

​
背景

Cosmos SDK 维护了一组 Protobuf 定义。正确设计 Protobuf 定义非常重要，以避免在同一版本内引入任何破坏性变更。原因在于不能破坏工具链（包括索引器和区块浏览器）、钱包以及其他第三方集成。 当前，在修改这些 Protobuf 定义时，Cosmos SDK 仅遵循 Buf 的建议。然而我们注意到，在某些情况下，Buf 的建议仍可能导致 SDK 中出现破坏性变更。例如：
  • 向 Msg 添加字段。添加字段本身并不违反 Protobuf 规范。然而，当向 Msg 添加新字段时，如果将新的 Msg 发送到旧节点，未知字段拒绝机制会抛出错误。
  • 将字段标记为 reserved。Protobuf 提出使用 reserved 关键字来删除字段，而无需提升包版本。然而这样做会破坏客户端的向后兼容性，因为 Protobuf 不会为 reserved 字段生成任何内容。有关此问题的更多细节，请参见 #9446。
此外，模块开发者经常还会遇到其他与 Protobuf 定义相关的问题，例如“我可以重命名字段吗？”或“我可以弃用字段吗？”本 ADR 旨在通过提供关于 Protobuf 定义允许更新方式的明确指南，回答所有这些问题。

​
决策

我们决定保留 Buf 的建议，但有以下例外：
  • UNARY_RPC：Cosmos SDK 当前不支持流式 RPC。
  • COMMENT_FIELD：Cosmos SDK 允许字段没有注释。
  • SERVICE_SUFFIX：我们使用 Query 和 Msg 服务命名约定，不使用 -Service 后缀。
  • PACKAGE_VERSION_SUFFIX：某些包（例如 cosmos.crypto.ed25519）不使用版本后缀。
  • RPC_REQUEST_STANDARD_NAME：Msg 服务的请求不带 -Request 后缀，以保持向后兼容性。
在 Buf 建议的基础上，我们增加以下针对 Cosmos SDK 的专用指南。

​
在不提升版本的情况下更新 Protobuf 定义

​
1. 模块开发者可以添加新的 Protobuf 定义

模块开发者可以添加新的 message、新的 Service、新的 rpc 端点，以及向现有消息中添加新字段。该建议遵循 Protobuf 规范，但为表述清晰起见，仍在本文档中明确写出，因为 SDK 还要求额外进行一项变更。 SDK 要求新增内容的 Protobuf 注释中包含一行，格式如下： 
// Since: cosmos-sdk <version>{, <version>...}
其中，每个 version 表示字段可用起始的某个次版本（如 0.45）或补丁版本（如 0.44.5）。这将极大帮助客户端库，它们可以选择使用反射或自定义代码生成，根据目标节点版本展示或隐藏这些字段。 例如，以下注释是有效的： 
// Since: cosmos-sdk 0.44

// Since: cosmos-sdk 0.42.11, 0.44.5
而以下注释是无效的： 
// Since cosmos-sdk v0.44

// since: cosmos-sdk 0.44

// Since: cosmos-sdk 0.42.11 0.44.5

// Since: Cosmos SDK 0.42.11, 0.44.5

​
2. 字段可以标记为 deprecated，节点也可以针对这些字段实现协议层面的破坏性处理变更

Protobuf 支持 deprecated 字段选项，该选项可用于任何字段，包括 Msg 字段。如果节点处理的 Protobuf 消息中包含非空的已弃用字段，节点在处理该消息时可以改变其行为，甚至可以是协议破坏性的方式。在可能的情况下，节点必须在不破坏共识的前提下处理向后兼容性问题（除非我们提升 proto 版本）。 例如，在 Cosmos SDK 从 v0.42 升级到 v0.43 的过程中，包含了两个会破坏 Protobuf 的变更，如下所列。SDK 团队没有将包版本从 v1beta1 提升到 v1，而是决定遵循本指南：回退这些破坏性变更、将这些变更标记为已弃用，并在节点处理包含已弃用字段的消息时修改实现。更具体地说：
  • Cosmos SDK 最近移除了对 基于时间的软件升级 的支持。因此，cosmos.upgrade.v1beta1.Plan 中的 time 字段已被标记为已弃用。此外，节点将拒绝任何包含升级 Plan 且其 time 字段非空的提案。
  • Cosmos SDK 现已支持治理拆分投票。在查询投票时，返回的 cosmos.gov.v1beta1.Vote 消息中，其 option 字段（用于单个投票选项）已被弃用，改用 options 字段（允许多个投票选项）。在可能的情况下，SDK 仍会填充已弃用的 option 字段，即当且仅当 len(options) == 1 且 options[0].Weight == 1.0 时。

​
3. 字段不得重命名

尽管官方 Protobuf 建议并未禁止重命名字段，因为这不会破坏 Protobuf 的二进制表示，但 SDK 明确禁止重命名 Protobuf 结构体中的字段。作出这一选择的主要原因是避免为客户端引入破坏性变更，因为客户端通常依赖生成类型中的硬编码字段。此外，重命名字段还会导致 Protobuf 定义的 JSON 表示发生客户端破坏性变化，而这些 JSON 表示被用于 REST 端点和 CLI 中。

​
提升 Protobuf 包版本

TODO，需要架构评审。一些主题：
  • 提升版本的频率
  • 提升版本时，Cosmos SDK 是否应同时支持两个版本？
    • 即从 v1beta1 -> v1 时，我们是否应在 Cosmos SDK 中保留两个目录，并为两个版本都提供处理器？
  • 提及 ADR-023 Protobuf 命名

​
后果

本节描述应用该决策后的结果性上下文。所有后果都应在此列出，而不仅仅是“正面”的后果。某项具体决策可能带来正面、负面和中性的后果，但它们都会在未来影响团队和项目。

​
向后兼容性

所有引入向后不兼容性的 ADR 都必须包含一个章节来描述这些不兼容性及其严重程度。ADR 必须说明作者计划如何处理这些不兼容性。若提交的 ADR 未对向后兼容性进行充分论述，可能会被直接拒绝。

​
正面影响

  • 减少工具开发者的负担
  • 提高生态系统中的兼容性
  • …

​
负面影响

{negative consequences}

​
中性影响

  • 在 Protobuf 审查中提高严谨性

​
进一步讨论

本 ADR 仍处于 DRAFT 阶段，“提升 Protobuf 包版本”部分将在我们就如何正确执行该操作作出决定后补充完善。

​
测试用例 [可选]

对于影响共识变更的 ADR，实现对应的测试用例是强制要求。其他 ADR 可根据需要选择是否包含测试用例链接。

​
参考资料

  • #9445 发布 proto 定义 v1
  • #9446 处理 v1beta1 proto 的破坏性变更

​
Changelog

  • 28.06.2021: Initial Draft
  • 02.12.2021: Add Since: comment for new fields
  • 21.07.2022: Remove the rule of no new Msg in the same proto version.

​
Status

Draft

​
Abstract

This ADR provides guidelines and recommended practices when updating Protobuf definitions. These guidelines are targeting module developers.

​
Context

The Cosmos SDK maintains a set of Protobuf definitions. It is important to correctly design Protobuf definitions to avoid any breaking changes within the same version. The reasons are to not break tooling (including indexers and explorers), wallets and other third-party integrations. When making changes to these Protobuf definitions, the Cosmos SDK currently only follows Buf’s recommendations. We noticed however that Buf’s recommendations might still result in breaking changes in the SDK in some cases. For example:
  • Adding fields to Msgs. Adding fields is a not a Protobuf spec-breaking operation. However, when adding new fields to Msgs, the unknown field rejection will throw an error when sending the new Msg to an older node.
  • Marking fields as reserved. Protobuf proposes the reserved keyword for removing fields without the need to bump the package version. However, by doing so, client backwards compatibility is broken as Protobuf doesn’t generate anything for reserved fields. See #9446 for more details on this issue.
Moreover, module developers often face other questions around Protobuf definitions such as “Can I rename a field?” or “Can I deprecate a field?” This ADR aims to answer all these questions by providing clear guidelines about allowed updates for Protobuf definitions.

​
Decision

We decide to keep Buf’s recommendations with the following exceptions:
  • UNARY_RPC: the Cosmos SDK currently does not support streaming RPCs.
  • COMMENT_FIELD: the Cosmos SDK allows fields with no comments.
  • SERVICE_SUFFIX: we use the Query and Msg service naming convention, which doesn’t use the -Service suffix.
  • PACKAGE_VERSION_SUFFIX: some packages, such as cosmos.crypto.ed25519, don’t use a version suffix.
  • RPC_REQUEST_STANDARD_NAME: Requests for the Msg service don’t have the -Request suffix to keep backwards compatibility.
On top of Buf’s recommendations we add the following guidelines that are specific to the Cosmos SDK.

​
Updating Protobuf Definition Without Bumping Version

​
1. Module developers MAY add new Protobuf definitions

Module developers MAY add new messages, new Services, new rpc endpoints, and new fields to existing messages. This recommendation follows the Protobuf specification, but is added in this document for clarity, as the SDK requires one additional change. The SDK requires the Protobuf comment of the new addition to contain one line with the following format: 
// Since: cosmos-sdk <version>{, <version>...}
Where each version denotes a minor (“0.45”) or patch (“0.44.5”) version from which the field is available. This will greatly help client libraries, who can optionally use reflection or custom code generation to show/hide these fields depending on the targetted node version. As examples, the following comments are valid: 
// Since: cosmos-sdk 0.44

// Since: cosmos-sdk 0.42.11, 0.44.5
and the following ones are NOT valid: 
// Since cosmos-sdk v0.44

// since: cosmos-sdk 0.44

// Since: cosmos-sdk 0.42.11 0.44.5

// Since: Cosmos SDK 0.42.11, 0.44.5

​
2. Fields MAY be marked as deprecated, and nodes MAY implement a protocol-breaking change for handling these fields

Protobuf supports the deprecated field option, and this option MAY be used on any field, including Msg fields. If a node handles a Protobuf message with a non-empty deprecated field, the node MAY change its behavior upon processing it, even in a protocol-breaking way. When possible, the node MUST handle backwards compatibility without breaking the consensus (unless we increment the proto version). As an example, the Cosmos SDK v0.42 to v0.43 update contained two Protobuf-breaking changes, listed below. Instead of bumping the package versions from v1beta1 to v1, the SDK team decided to follow this guideline, by reverting the breaking changes, marking those changes as deprecated, and modifying the node implementation when processing messages with deprecated fields. More specifically:
  • The Cosmos SDK recently removed support for time-based software upgrades. As such, the time field has been marked as deprecated in cosmos.upgrade.v1beta1.Plan. Moreover, the node will reject any proposal containing an upgrade Plan whose time field is non-empty.
  • The Cosmos SDK now supports governance split votes. When querying for votes, the returned cosmos.gov.v1beta1.Vote message has its option field (used for 1 vote option) deprecated in favor of its options field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated option field, that is, if and only if the len(options) == 1 and options[0].Weight == 1.0.

​
3. Fields MUST NOT be renamed

Whereas the official Protobuf recommendations do not prohibit renaming fields, as it does not break the Protobuf binary representation, the SDK explicitly forbids renaming fields in Protobuf structs. The main reason for this choice is to avoid introducing breaking changes for clients, which often rely on hard-coded fields from generated types. Moreover, renaming fields will lead to client-breaking JSON representations of Protobuf definitions, used in REST endpoints and in the CLI.

​
Incrementing Protobuf Package Version

TODO, needs architecture review. Some topics:
  • Bumping versions frequency
  • When bumping versions, should the Cosmos SDK support both versions?
    • i.e. v1beta1 -> v1, should we have two folders in the Cosmos SDK, and handlers for both versions?
  • mention ADR-023 Protobuf naming

​
Consequences

This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the “positive” ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future.

​
Backwards Compatibility

All ADRs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The ADR must explain how the author proposes to deal with these incompatibilities. ADR submissions without a sufficient backwards compatibility treatise may be rejected outright.

​
Positive

  • less pain to tool developers
  • more compatibility in the ecosystem
  • …

​
Negative

{negative consequences}

​
Neutral

  • more rigor in Protobuf review

​
Further Discussions

This ADR is still in the DRAFT stage, and the “Incrementing Protobuf Package Version” will be filled in once we make a decision on how to correctly do it.

​
Test Cases [optional]

Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable.

​
References

  • #9445 Release proto definitions v1
  • #9446 Address v1beta1 proto breaking changes