Skip to content

CLAUDE.md

Latest commit

 

History

History
413 lines (334 loc) · 17.7 KB

CLAUDE.md

File metadata and controls

413 lines (334 loc) · 17.7 KB
Raw

ethlambda Development Guide

Development reference for ethlambda - minimalist Lean Ethereum consensus client. Not to be confused with Ethereum consensus clients AKA Beacon Chain clients AKA Eth2 clients.

Quick Reference

Main branch: main Rust version: 1.97.1 (edition 2024) Test fixtures release: Download latest production fixtures from leanSpec releases

Codebase Structure (12 workspace crates)

bin/ethlambda/              # Entry point, CLI, orchestration
  └─ src/version.rs         # Build-time version info (vergen-git2)
crates/
  blockchain/               # State machine actor (GenServer pattern)
    ├─ src/lib.rs           # BlockChain actor, tick events, validator duties
    ├─ src/store.rs         # Fork choice store, block/attestation processing
    ├─ src/block_builder.rs # Block assembly (pre-built at previous slot's interval 4)
    ├─ src/aggregation.rs   # Interval-2 signature aggregation worker
    ├─ src/reaggregate.rs   # Re-aggregation of block-borne votes on import
    ├─ src/sync_status.rs   # Sync-gate tracker (suppresses duties while syncing)
    ├─ src/key_manager.rs   # Validator key management and signing
    ├─ src/metrics.rs       # Blockchain-level Prometheus metrics
    ├─ fork_choice/         # [crate] LMD GHOST implementation (3SF-mini)
    └─ state_transition/    # [crate] STF: process_slots, process_block, attestations
        ├─ src/justified_slots_ops.rs  # Relative-index helpers for justified_slots
        └─ src/metrics.rs   # State transition timing + counters
  common/
    ├─ types/               # Core types (State, Block, Attestation, Checkpoint)
    ├─ crypto/              # XMSS aggregation (leansig wrapper)
    ├─ metrics/             # Prometheus re-exports, TimingGuard, gather utilities
    └─ test-fixtures/       # Spec-fixture loading (prod dep of rpc's Hive test driver)
  net/
    ├─ api/                 # Actor protocol traits wiring BlockChain ↔ P2P
    ├─ p2p/                 # libp2p: gossipsub + req-resp (Status, BlocksByRoot, BlocksByRange)
    │   ├─ src/gossipsub/   # Topic encoding, message handling
    │   ├─ src/req_resp/    # Request/response codec and handlers
    │   └─ src/metrics.rs   # Peer connection/disconnection tracking
    └─ rpc/                 # Axum HTTP: API server + metrics server (independent ports)
  storage/                  # RocksDB backend, in-memory for tests
    └─ src/api/             # StorageBackend trait + Table enum

Key Architecture Patterns

Actor Concurrency (spawned-concurrency)

  • BlockChain: Main state machine (GenServer pattern)
  • P2P: Network event loop with libp2p swarm
  • Communication via mpsc::unbounded_channel
  • Shared storage via Arc<dyn StorageBackend> (clone Store, share backend)

Tick-Based Validator Duties (4-second slots, 5 intervals per slot)

Interval 0: Block published (at the slot boundary). The build+publish code path is merged into the previous slot's interval 4 (see below) and aligned to publish here; no attestation acceptance happens at interval 0.
Interval 1: Attestation production (all validators, including proposer)
Interval 2: Aggregation (aggregators create proofs from gossip signatures)
Interval 3: Safe target update (fork choice)
Interval 4: Accept accumulated attestations; build the NEXT slot's block and publish it aligned to that slot's interval 0 (build and publish merged into this tick)

Attestation Pipeline

Gossip → Signature verification → new_payloads (pending)
  ↓ (intervals 0/4)
promote → known_payloads (fork choice active)
  ↓
Fork choice head update

(Store buffer fields are new_payloads/known_payloads; the accessors are named extract_latest_new_attestations/extract_latest_known_attestations.)

State Transition Phases

  1. process_slots(): Advance through empty slots, update historical roots
  2. process_block(): Validate header → process attestations → update justifications/finality
  3. Justification: 3SF-mini rules (delta ≤ 5 OR n² OR n(n+1))
  4. Finalization: Source with no unjustifiable gaps to target

Development Workflow

Before Committing

make fmt                                     # Format code (cargo fmt --all)
make lint                                    # Clippy with -D warnings
make test                                    # All tests + forkchoice spec tests

Common Operations

rm -rf leanSpec && make leanSpec/fixtures                # Download latest released test fixtures
make docker-build                                        # Build Docker image (DOCKER_TAG=local)
make run-devnet                                          # Run local devnet with lean-quickstart

Testing with Local Devnet

See .claude/skills/devnet-runner/SKILL.md for running a local multi-client devnet (node roster, image tags, pause/unpause instability testing) and .claude/skills/devnet-log-review/SKILL.md for analyzing the dumped logs.

Important Patterns & Idioms

Trait Implementations

// Prefer From/Into traits over custom from_x/to_x methods
impl From<u8> for ResponseCode { fn from(code: u8) -> Self { Self(code) } }
impl From<ResponseCode> for u8 { fn from(code: ResponseCode) -> Self { code.0 } }

// Enables idiomatic .into() usage
let code: ResponseCode = byte.into();
let byte: u8 = code.into();

Ownership for Large Structures

// Prefer taking ownership to avoid cloning large data (signatures ~2.5KB)
pub fn insert_signed_block(&mut self, root: H256, signed_block: SignedBlock) { ... }

// Add .clone() at call site if needed - makes cost explicit
store.insert_signed_block(block_root, signed_block.clone());

Formatting Patterns

// Extract long arguments into variables so formatter can join lines
// Instead of:
batch.put_batch(Table::X, vec![(key, value)]).expect("msg");

// Prefer:
let entries = vec![(key, value)];
batch.put_batch(Table::X, entries).expect("msg");

Error Handling Patterns

Use inspect and inspect_err for side-effect-only error handling:

// ✅ GOOD: Use inspect_err when only logging or performing side effects on error
result
    .inspect_err(|err| warn!(%err, "Operation failed"));

// Extract complex expressions to variables for cleaner formatting
let response = Response::success(ResponsePayload::BlocksByRoot(blocks));
server.swarm.behaviour_mut().req_resp.send_response(channel, response)
    .inspect_err(|err| warn!(%peer, ?err, "Failed to send response"));

// ✅ GOOD: Use inspect + inspect_err when both branches need side effects
operation()
    .inspect(|_| metrics::inc_success())
    .inspect_err(|_| metrics::inc_failed());

// ❌ AVOID: Using if let Err when only performing side effects
if let Err(err) = result {
    warn!(%err, "Operation failed");
}

// ❌ AVOID: Using if/else for both success and error side effects
if let Err(err) = operation() {
    metrics::inc_failed();
} else {
    metrics::inc_success();
}

When NOT to use inspect_err:

// Use if let Err or match when:
// 1. Early return needed
if let Err(err) = operation() {
    error!(%err, "Fatal error");
    return false;
}

// 2. Error needs transformation (use map_err + ?)
let result = operation()
    .map_err(|err| CustomError::from(err))?;

Metrics Patterns

Registration with LazyLock:

// Module-scoped statics (preferred for state_transition metrics)
static LEAN_STATE_TRANSITION_TIME_SECONDS: LazyLock<Histogram> = LazyLock::new(|| {
    register_histogram!("lean_metric_name", "Description", vec![...]).unwrap()
});

// Function-scoped statics (used in blockchain metrics)
pub fn update_head_slot(slot: u64) {
    static LEAN_HEAD_SLOT: LazyLock<IntGauge> = LazyLock::new(|| {
        register_int_gauge!("lean_head_slot", "Latest slot").unwrap()
    });
    LEAN_HEAD_SLOT.set(slot.try_into().unwrap());
}

RAII timing guard (auto-observes duration on drop):

let _timing = metrics::time_state_transition();

All metrics use ethlambda_metrics::* re-exports — the ethlambda-metrics crate re-exports prometheus types (IntGauge, IntCounter, Histogram, etc.) and provides TimingGuard + gather_default_metrics().

Naming convention: All metrics use lean_ prefix (e.g., lean_head_slot, lean_state_transition_time_seconds).

Logging Patterns

Use tracing shorthand syntax for cleaner logs:

// ✅ GOOD: Shorthand for simple variables
let slot = block.slot;
let proposer = block.proposer_index;
info!(
    %slot,              // Shorthand for slot = %slot (Display)
    proposer,           // Shorthand for proposer = proposer
    block_root = %ShortRoot(&block_root.0),  // Named expression
    "Block imported"
);

// ❌ BAD: Verbose
info!(
    slot = %slot,
    proposer = proposer,
    ...
);

Standardized field ordering (temporal → identity → identifiers → context → metadata):

// Block logs
info!(%slot, proposer, block_root = ..., parent_root = ..., attestation_count, "...");

// Attestation logs
info!(%slot, validator, target_slot, target_root = ..., source_slot, source_root = ..., "...");

// Consensus events
info!(finalized_slot, finalized_root = ..., previous_finalized, justified_slot, "...");

// Peer events
info!(%peer_id, %direction, peer_count, our_finalized_slot, our_head_slot, "...");

Root hash truncation:

use ethlambda_types::ShortRoot;

// Always use ShortRoot for consistent 8-char display (4 bytes)
info!(block_root = %ShortRoot(&root.0), "...");

Relative Indexing (justified_slots)

// Bounded storage: index relative to finalized_slot
actual_slot = finalized_slot + 1 + relative_index
// Helper ops in justified_slots_ops.rs

Cryptography & Signatures

XMSS (eXtended Merkle Signature Scheme):

  • Post-quantum signature scheme
  • 52-byte public keys, 2536-byte signatures (SIGNATURE_SIZE in common/types/src/signature.rs)
  • Epoch-based to prevent reuse
  • Aggregation via leanVM (previously leanMultisig) for efficiency

Signature Aggregation (Two-Phase):

  1. Gossip signatures: Fresh XMSS from network → aggregate via leanVM
  2. Fallback to proofs: Reuse previous block proofs for missing validators

Networking (libp2p)

Protocols

  • Transport: QUIC over UDP (TLS 1.3)
  • Gossipsub: Blocks + Attestations (snappy raw compression)
    • Topic: /leanconsensus/{fork_digest}/{block|aggregation|attestation_N}/ssz_snappy
    • fork_digest is a 4-byte hex string (no 0x prefix); currently the dummy 12345678 agreed across clients
    • Mesh size: 8 (6-12 bounds), heartbeat: 700ms
  • Req/Resp: Status, BlocksByRoot, BlocksByRange (snappy frame compression + varint length)

Retry Strategy on Block Requests

  • Exponential backoff: doubling from INITIAL_BACKOFF_MS (5ms → 2560ms)
  • Max MAX_FETCH_RETRIES (10) attempts, random peer selection on retry

Message IDs

  • 20-byte truncated SHA256 of: domain (valid/invalid snappy) + topic + data

HTTP Servers (API + Metrics)

The RPC crate serves the API router (--api-port, default 5052) and the metrics/debug routers (--metrics-port, default 5054). When the two ports differ it binds two independent Axum servers; when they are equal it merges all three routers onto a single listener, so pointing both flags at one port is supported and not a misconfiguration. See docs/rpc.md for the full reference: CLI flags and defaults, the API endpoints (health, finalized state/block, justified checkpoint, blocks by root/slot, fork-choice tree + D3.js UI, runtime aggregator toggle), the metrics/debug endpoints (Prometheus /metrics, jemalloc heap profiling), the Hive test-driver endpoints, plus request/response shapes, status codes, and content types.

Configuration Files

Genesis: config.yaml (YAML format, cross-client compatible)

GENESIS_TIME: 1770407233
GENESIS_VALIDATORS:
  - attestation_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a..."  # 52-byte XMSS pubkeys (hex)
    proposal_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37..."
  • Validator indices are assigned sequentially (0, 1, 2, ...) based on array order
  • All genesis state fields (checkpoints, justified_slots, etc.) initialize to zero/empty defaults
  • Matches Ream/Zeam format — no extra state fields in the config file

Bootnodes: ENR records (Base64-encoded, RLP decoded for QUIC port + secp256k1 pubkey)

Testing

Test Categories

  1. Unit tests: Embedded in source files
  2. Spec tests: From leanSpec/fixtures/consensus/
    • crates/blockchain/tests/forkchoice_spectests.rs (uses on_block_without_verification via spec_test_runner)
    • crates/blockchain/tests/signature_spectests.rs
    • crates/blockchain/state_transition/tests/stf_spectests.rs (state transition)

Running Tests

cargo test --workspace --profile release-fast                       # All workspace tests
cargo test -p ethlambda-blockchain --test forkchoice_spectests
cargo test -p ethlambda-blockchain --test forkchoice_spectests -- --test-threads=1  # Sequential

Tests run under release-fast: release-grade opt-level (needed to avoid stack overflows in signature verification/aggregation) but no LTO, parallel codegen, incremental, and line-tables-only debuginfo, so rebuilds are much faster than --release. Artifacts land in target/release-fast/, separate from cargo build --release.

Common Gotchas

Aggregator Flag Required for Finalization

  • At least one node must be started with --is-aggregator to finalize blocks
  • Without this flag, attestations pass signature verification and are logged as "Attestation processed", but the signature is never stored for aggregation (the is_aggregator gate in on_gossip_attestation, store.rs), so blocks are always built with attestation_count=0
  • The attestation pipeline: gossip → verify signature → store gossip signature (only if is_aggregator) → aggregate at interval 2 → promote to known → pack into blocks
  • Symptom: justified_slot=0 and finalized_slot=0 indefinitely despite healthy block production and attestation gossip

Runtime Aggregator Toggle (Hot-Standby Model)

  • POST /lean/v0/admin/aggregator with {"enabled": bool} toggles the aggregator role at runtime without restart (ported from leanSpec PR #636)
  • GET /lean/v0/admin/aggregator returns {"is_aggregator": bool}
  • The CLI --is-aggregator flag seeds the initial value; runtime toggles are in-process only (not persisted across restarts)
  • Runtime toggles do NOT resubscribe gossip subnets — those are frozen at startup by build_swarm. Toggling ON at runtime only activates aggregation logic for subnets the node was already subscribed to
  • Operational model: standby aggregators should boot with --is-aggregator=true (so subscriptions are in place), then use the admin endpoint to rotate duties. A node booted with --is-aggregator=false and toggled ON later will have no extra subnets to aggregate

Signature Verification

  • Fork choice tests use on_block_without_verification() to skip signature checks
  • Signature spec tests use on_block() which always verifies
  • Crypto tests marked #[ignore] (slow leanVM operations)

Storage

Blocks split across BlockHeaders/BlockBodies/BlockProof; states are snapshot (States) + diff (StateDiffs) pairs; BlockRoots and LiveChain index by slot for range serving and fork choice. Attestations and gossip signatures are not persisted; they live in in-memory Store buffers consumed during the tick pipeline. See docs/data_storage.md for the full reference: what each of the eight tables holds and how it's keyed, the snapshot/diff reconstruction algorithm, the block-import write sequence, pruning rules, what never changes at runtime, and startup/restore behavior.

  • BlockProof is the only pruned block table (below the finalized boundary); get_signed_block returns None for a pruned finalized block.
  • A StateDiff omits config and validators, trusting they never mutate; breaking that invariant would silently corrupt every reconstructed state.
  • Metadata["config"] is written once at bootstrap and never rewritten; it doubles as the DB's genesis-time fingerprint on resume.

State Root Computation

  • Always computed via hash_tree_root() after full state transition
  • Must match proposer's pre-computed block.state_root

Finalization Checks

  • Use original_finalized_slot for justifiability checks during attestation processing
  • Finalization updates can occur mid-processing

justified_slots Window Shifting

  • Call shift_window() when finalization advances
  • Prunes justifications for now-finalized slots

External Dependencies

Critical:

  • leansig: XMSS signatures (leanEthereum project)
  • libssz / libssz-derive / libssz-types: SSZ serialization
  • libssz-merkle: Merkle tree hashing (hash_tree_root())
  • spawned-concurrency: Actor model
  • libp2p: P2P networking (custom LambdaClass fork)
  • vergen-git2: Build-time git commit/branch info embedded in binary

Storage:

  • rocksdb: Persistent backend
  • In-memory backend for tests

Resources

Specs: leanSpec/src/lean_spec/spec/ (Python reference implementation; fork logic under forks/<fork>/, e.g. forks/lstar/) Devnet: lean-quickstart (github.com/blockblaz/lean-quickstart) Docs: docs/ — rpc.md, metrics.md, checkpoint_sync.md, 3sf_mini.md, lmd_ghost.md (mdbook via make docs) Releases: See RELEASE.md for release process documentation

Other implementations