Skip to content

UPSTREAM.md

Latest commit

 

History

History
111 lines (71 loc) · 8.82 KB

UPSTREAM.md

File metadata and controls

111 lines (71 loc) · 8.82 KB
Raw

Upstream alignment

This repository tracks ethp2p.

Mapping policy

zig-ethp2p must be a like-for-like port of ethp2p.

Every constant, parameter, wire format detail, and behavioural constraint in the Go reference has a direct Zig counterpart — same value, same semantics, same name where idiomatic Zig allows it. Nothing is omitted, approximated, or left as a comment.

Concretely this means:

  • Constants and parameters: every numeric constant or configuration value in the Go source (mesh degree, heartbeat interval, queue depths, frame size limits, etc.) is exported as a typed Zig constant in the corresponding module.
  • Wire format: byte layout, field order, protobuf field numbers, and length-prefix conventions match the Go implementation exactly. Golden-byte tests lock this down.
  • Protocol behaviour: stream open/accept patterns, selector bytes, handshake sequences, and error handling mirror the Go reference. When the Go code makes a deliberate choice (e.g. unidirectional QUIC streams, StrictNoSign), the Zig code encodes that choice explicitly — not as a side effect.
  • Test coverage: every mirrored value or behaviour is exercised by at least one test that would fail if the reference changes.

When reviewing an upstream diff, ask for each changed line in the Go source: does the Zig side have a corresponding constant, type, or behaviour that must change? If yes, change it. Do not skip a change because the Zig code happens to produce the same result for an unrelated reason.

The only items excluded from like-for-like porting are those that depend on Go/libp2p runtime internals with no Zig equivalent (e.g. go-libp2p host lifecycle, context.Context cancellation). Document any such exclusion explicitly in the relevant section below.

Zig toolchain

build.zig.zon minimum_zig_version must match the ZIG_VERSION environment variable in .github/workflows/ci.yml. CI fails the job if they differ, so bump both when raising the supported Zig release.

Pinned revision

The vendored .proto files and golden test vectors were checked against reference commit:

741d8d9cf682ff93b7d8eb56e0377ba8eea83a7e

Changes since the previous pin (db6e9417d0bbab9ded28aa3053211cdecff402ac):

  • 741d8d9: documentation and CI only — rumdl markdown lint for specs/ (SLEAC / semantic line breaks in .rumdl.toml), new markdown-lint job in .github/workflows/ci.yml, just specs installs rumdl and runs rumdl fmt specs/. No changes under broadcast/, protocol/, sim/, or vendored protobufs; Zig wire and golden vectors unchanged.

Changes since the previous pin (125bdaa → db6e941):

  • sim/strategy_gossipsub.go (db6e941): gossipsub params aligned with Prysm (beacon-chain/p2p/pubsub.go).
    • HistoryLength: 1000 → 6; HistoryGossip: 1000 → 3.
    • Added StrictNoSign + NoAuthor: messages carry no from/seqno/signature/key fields.
    • Added PeerOutboundQueueSize(600), ValidateQueueSize(600).
    • zig-ethp2p impact: all concrete parameters added as exported constants in sim/gossipsub_transport.zig (mesh_d, mesh_d_lo, mesh_d_hi, mesh_d_lazy, heartbeat_interval_ms, fanout_ttl_ms, history_length, history_gossip, max_message_size, peer_outbound_queue_size, validate_queue_size, strict_no_sign). A test asserts every value against its reference so future upstream changes are caught.

When updating:

  1. Diff proto/*.proto against:
    • broadcast/pb/broadcast.proto
    • protocol/pb/protocol.proto
    • broadcast/rs/pb/rs.proto
    • go-libp2p-pubsub/pb/rpc.proto (for proto/gossipsub_rpc.proto field numbers; sim/gossipsub_rpc_pb.zig implements RPC 1–3 and 10 (partial / PartialMessagesExtension), full ControlMessage, varint length-prefixed framing; other extension field numbers are skipped on decode. sim/gossipsub_rpc_host.zig is an in-process duplex for framed RPC, not a libp2p transport.)
  2. Run zig build test (golden bytes must still match google.golang.org/protobuf output from that tree).
  3. Bump the commit hash in this file.

QUIC / UDP transport

src/transport/eth_ec_quic.zig mirrors ALPN eth-ec-broadcast and high-level quic-go-style limits from ethp2p sim/host.go. The implementation uses pure-Zig zquic (build.zig.zon dependency); there is no vendored C QUIC/TLS stack in this repository.

Why raw QUIC and why unidirectional streams

Why raw QUIC? Direct access to QUIC's built-in multiplexing, per-stream flow control, congestion control, and RTT measurements — without adding another framing layer.

Why unidirectional streams? P2P protocols have no client/server notion; both peers are equal and can try to open a stream to each other simultaneously. With bidirectional streams that creates a simultaneous open ambiguity that must be resolved in-band. Unidirectional streams eliminate the ambiguity by design: each peer opens its own send stream independently, and there is no question about which side "owns" the stream. Opening streams in QUIC is cheap (no extra round-trip), so the cost of using two half-streams instead of one full-stream is negligible.

Bidirectional streams are viable when the protocol has a clear initiator (e.g. HTTP, where only the client opens streams). ethp2p deliberately chose UNI streams to keep the peer state machine stateless with respect to stream negotiation.

UNI stream alignment (issue #28)

The ethp2p reference uses unidirectional QUIC streams for all application protocols:

  • peer.go handshake(): both sides call conn.OpenUniStream() for the BCAST control stream (IDs 2/3, 6/7, …)
  • peer_ctrl.go handleSessionOpen, doSendChunk: conn.OpenUniStream() for SESS and CHUNK streams
  • peer_in.go runAcceptLoop: conn.AcceptUniStream() for all inbound streams

Zig alignment:

  • zquic_quic_shim.zig classifies incoming streams by QUIC stream ID (client vs server stream numbering per RFC 9000; bidi vs uni queues) and exposes tryAcceptIncomingUniStream for peer-initiated UNI streams
  • streamMakeUni opens an outgoing UNI stream via zquic
  • eth_ec_quic_peer.zig implements the PeerConn poll-driven state machine (handshake + accept-loop); broadcast/engine_quic.zig (EngineQuicHost) forwards inbound SESS/CHUNK into Engine / ChannelRs (#37)

libp2p boundary

The ethp2p sim/ QUIC transport is illustrative. Production deployments layer libp2p on top (Noise handshake, multistream-select, Yamux multiplexer, identify protocol). That layer is out of scope for zig-ethp2p; zeam handles it via rust-libp2p.

EC schemes (issue #14)

src/layer/ec_scheme.zig defines EcSchemeKind and the "reed-solomon" string aligned with broadcast/rs/types.go NewScheme. Only Reed–Solomon is implemented end-to-end; RLNC needs spec’d preamble / chunk types and a strategy implementation before wire changes.

Abstract mesh tests

src/sim/rs_mesh.zig runs the same RS settings and graph topologies as sim/scenario_test.go (TestNetwork RS / RS-ChunkLen) against layer.RsStrategy in-process (no libp2p, no Go simnet). Adjacency and per-peer stats are heap-allocated (MaxMeshNodes cap). MeshParams.partition drops selected undirected links for an initial round range (no chunks / routing across them), then restores them—this matches the intent of ethp2p CI’s TestNodeReconnection name in the simnet-rs job (that test does not exist on ethp2p main today). zig build test and zig build simtest both execute these cases. With ZIG_ETHP2P_STRESS=1 (see zig build test-stress), extra cases use higher round budgets and add 8- and 16-node ring graphs beyond the Go file’s largest fixed topology.

Specifications

Normative docs: ethp2p/specs (architecture 001-ethp2p, broadcast 002–004).

Wire compatibility

  • Length-prefixed frames match broadcast/wire.go (MaxFrameSize, big-endian uint32 length, then payload).
  • Stream opener is a single byte (protocol/protocol.go WriteSelector / ReadSelector), equal to the Protocol enum value. The Selector protobuf in protocol.proto describes that enum for codegen; the reference does not length-prefix or protobuf-encode the selector on the wire.
  • CHUNK streams (broadcast/peer_ctrl.go doSendChunk, peer_in.go processChunk): 0x03, then length-framed Chunk.Header, then exactly data_length bytes of payload (not framed).
  • BCAST / SESS high-level open helpers mirror peer.go / peer_ctrl.go / peer_in.go (see wire/bcast_stream.zig, wire/sess_stream.zig).
  • Protobuf field numbers for framed messages match the .proto definitions; tests include hex from the Go reference.