Add protobuf action serialization support by heifner · Pull Request #239 · Wire-Network/wire-sysio

heifner · 2026-03-06T23:38:23Z

Summary

Adds node-side Protocol Buffer support for smart contract action data serialization, complementing the CDT-side support in Wire-Network/wire-cdt#32.

Contracts can now use protobuf as an alternative to CDT's default datastream packing, providing ID-based field encoding, backwards-compatible schema evolution, and language-neutral message definitions.

What changed

Core (libraries/chain)

abi_def: new protobuf_types field (may_not_exist<string>) for ABI version 1.3
abi_serializer: runtime protobuf binary↔variant conversion using Google's DynamicMessageFactory, matching CDT's zpp_bits size_varint wire format (double-varint length prefix)
Custom to_variant/from_variant for abi_def to handle protobuf_types as a JSON object in ABI files but stored as a string internally
Protobuf headers forward-declared in .hpp, includes only in .cpp
protobuf added to vcpkg dependencies and linked in chain CMakeLists

Documentation

contracts/protocol-buffers.md: architecture overview, ABI v1.3 format, wire format details, clio usage, and end-to-end walkthrough for creating a protobuf contract

ABI format

Single pb<T> parameter actions use a flattened format — the action type points directly at the protobuf type with no wrapper struct:

{
  "version": "sysio::abi/1.3",
  "structs": [],
  "actions": [
    {
      "name": "greet",
      "type": "protobuf::pbexample.Greeting",
      "ricardian_contract": ""
    }
  ],
  "protobuf_types": {
    "file": [
      {
        "name": "pbexample.proto",
        "package": "pbexample",
        "messageType": [
          {
            "name": "Greeting",
            "field": [
              { "name": "from", "number": 1, "type": "TYPE_STRING", "label": "LABEL_OPTIONAL" },
              { "name": "message", "number": 2, "type": "TYPE_STRING", "label": "LABEL_OPTIONAL" },
              { "name": "importance", "number": 3, "type": "TYPE_INT32", "label": "LABEL_OPTIONAL" }
            ]
          }
        ],
        "syntax": "proto3"
      }
    ]
  }
}

Action data is flat JSON with no wrapper:
clio push action pbexample greet '{"from":"alice","message":"hello","importance":5}' -p pbexample@active

Multi-parameter actions generate a wrapper struct with one field per parameter:
{
  "structs": [
    {
      "name": "settle",
      "fields": [
        { "name": "header", "type": "protobuf::mypackage.Header" },
        { "name": "body", "type": "protobuf::mypackage.Body" }
      ]
    }
  ],
  "actions": [
    { "name": "settle", "type": "settle" }
  ]
}

clio push action mycontract settle '{"header":{"version":1},"body":{"payload":"data"}}' -p myaccount@active

Enable contracts to use Protocol Buffers as an alternative serialization format for action data. The node-side abi_serializer uses Google's libprotobuf DynamicMessageFactory to serialize/deserialize protobuf action data at runtime, matching CDT's zpp_bits size_varint wire format. Changes: - abi_def: add protobuf_types field (may_not_exist<string>) for ABI v1.3 - abi_serializer: protobuf binary<->variant conversion with double-varint length prefix format compatible with CDT's zpp_bits size_varint - abi_serializer: custom to_variant/from_variant for abi_def to handle protobuf_types as JSON object in ABI files, string internally - Forward-declare protobuf types in header, includes in .cpp only - vcpkg.json + chain CMakeLists: add protobuf dependency - 12 unit tests: ABI loading, round-trip, flattened actions, nested messages, repeated fields, mixed protobuf+regular fields, known-bytes format verification - Integration test (proto_abi_test.py): deploy and push protobuf actions on a live test chain - CLI test: clio pack/unpack round-trip with --abi-file - Documentation (contracts/protocol-buffers.md) Also fixes: move using-declarations above BOOST_AUTO_TEST_SUITE in native_overlay_tests.cpp to prevent std namespace shadowing.

Copilot

Pull request overview

Adds node-side Protocol Buffer (protobuf) support for ABI-based action data serialization/deserialization, aligned with CDT’s protobuf generation and wire format.

Changes:

Adds protobuf dependency + links libprotobuf into sysio_chain.
Extends ABI v1.3 with protobuf_types and implements runtime protobuf ↔ JSON(variant) conversion in abi_serializer.
Adds unit/integration tests and documentation for protobuf contract workflows and clio usage.

Reviewed changes

Copilot reviewed 16 out of 22 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
vcpkg.json	Adds protobuf as a dependency for the build.
libraries/chain/CMakeLists.txt	Finds/links `protobuf::libprotobuf` for node-side runtime support.
libraries/chain/include/sysio/chain/abi_serializer.hpp	Adds protobuf-related members and helpers to `abi_serializer`.
libraries/chain/include/sysio/chain/abi_def.hpp	Adds `protobuf_types` to ABI definition + declares custom JSON conversion.
libraries/chain/abi_serializer.cpp	Implements protobuf parsing, message conversion, and ABI JSON handling.
unittests/abi_tests.cpp	Adds protobuf ABI serialization/deserialization and round-trip tests.
unittests/native_overlay_tests.cpp	Refactors conditional compilation to avoid empty-suite failures.
unittests/test-contracts/CMakeLists.txt	Registers new protobuf ABI test contract.
unittests/test-contracts/proto_abi_test/CMakeLists.txt	Builds protobuf-generated headers for the test contract.
unittests/test-contracts/proto_abi_test/test.proto	Defines protobuf messages used for action input/output tests.
unittests/test-contracts/proto_abi_test/proto_abi_test.cpp	Adds a test contract using `sysio::pb<T>` actions.
unittests/test-contracts/proto_abi_test/proto_abi_test.abi	Adds ABI v1.3 example embedding `protobuf_types`.
tests/CMakeLists.txt	Installs and registers the new python integration test.
tests/proto_abi_test.py	End-to-end node test deploying and invoking protobuf actions.
tests/cli_test.py	Adds `clio --abi-file` pack/unpack tests for protobuf actions.
contracts/protocol-buffers.md	Documents protobuf ABI format, wire format, and tooling usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-09T17:45:14Z