feat(shadow): XMSS prover-cost model for the Shadow simulator (default-off)

.claude/commands/gean-engineer.md

@@ -0,0 +1,77 @@
+---
+description: gean protocol-engineering directive — apply before any fix, feature, refactor, optimization, or test. Enforces spec-first development, gean architecture/FFI/sszgen rules, and a required post-implementation review.
+argument-hint: [task to implement]
+---
+
+You are working inside **gean**, a Go implementation of an Ethereum **lean consensus** client (`github.com/geanlabs/gean`), targeting the **devnet-5** milestone and tracking the `leanEthereum/leanSpec` Python spec pinned at `LEAN_SPEC_COMMIT_HASH` in the `Makefile`. Consensus signatures are **XMSS** post-quantum hash-based signatures, implemented in Rust and reached over CGo FFI.
+
+The task to implement:
+
+$ARGUMENTS
+
+Before writing any code for that task, follow these rules. gean runs for real operators and interoperates in multi-client lean devnets alongside **ethlambda, zeam, ream, grandine, lantern, nlean, and qlean** — code that is "green locally" but diverges from the spec breaks the network. Engineer accordingly.
+
+## Primary objective
+Optimize in this order: **spec compliance → consensus/protocol correctness → safety → clarity & maintainability → performance → operational excellence.** "Making it work" is not the goal; improving the long-term health of the client is. When goals conflict the higher one wins, and **the spec always wins.**
+
+## The spec is the source of truth
+`leanSpec` (pinned at `LEAN_SPEC_COMMIT_HASH`) defines correct behavior — not existing gean code, not another client.
+- Read the relevant spec section before touching consensus logic. It concentrates in `internal/statetransition/` (pure `ProcessSlots → ProcessBlock → verify state_root`), `internal/forkchoice/` (LMD-GHOST over ProtoArray + VoteStore), and `internal/types/`.
+- State your spec mapping internally: which spec file/function the change mirrors, why it complies, and the edge cases.
+- When gean and the spec disagree, fix gean. When the spec is genuinely ambiguous, prefer what the interop peers do and surface it. Mirror spec control flow (e.g. catch-and-skip duty patterns) rather than inventing gean-specific behavior. Use `/spec-compliant` to check for drift.
+
+## Engineering philosophy
+Prefer reduction over addition, simplicity over cleverness, readability over abstraction, composition over inheritance, explicitness over magic, determinism over assumption. Before adding code, ask:
+1. Can this be solved by deleting code?
+2. Can an existing gean abstraction be reused (`store`, `forkchoice`, `types`, `xmss`, `pending` buffers, `metrics`, `logger`, `dutygate`, `role`)?
+3. Does the existing architecture already support this?
+4. Is this actually a protocol concern, or incidental?
+5. Does this complexity earn its keep?
+
+The best change is often a smaller one. `/simplify` and `/lean-review` exist to push on this.
+
+## Repository alignment
+Make the repo read as if one team wrote every line.
+- Respect gean's boundaries: `Engine` (`internal/node/`) owns runtime wiring and is single-threaded over the `select` loop in `Run`/`onTick`; expensive XMSS proving runs **off** the tick loop on dedicated workers (aggregation, proposal, recovery). **`ForkChoice` does not live inside `ConsensusStore`** — the Engine calls fork choice with store data as parameters.
+- Preserve the slot/interval model (4s slot, 5×800ms intervals; interval responsibilities in `onTick`) and the spec's head-update-vs-proposal ordering.
+- Match surrounding naming, error handling, and testing patterns. Don't restructure files or add directories without strong justification.
+
+## Generated code & build ordering (gean specifics)
+- **Never hand-edit `internal/types/*_encoding.go`** — they are generated. Change the struct + SSZ tags and run `make sszgen`. If generation fails, stop and report; do not hand-write SSZ.
+- **Rust FFI builds before any Go that touches `xmss`.** Use the make targets: `make build`, `make ffi`, `make test` (excludes xmss/spectests/cmd), `make test-ffi`, `make test-spec`, `make test-all`, `make lint` (`go vet` + `cargo fmt --check` + `cargo clippy -D warnings`), `make fmt`. A plain `go test ./...` fails to link unless `make ffi` ran.
+- Buffer caps and consensus limits are package-level constants aligned with the pinned spec — keep them in sync, don't fork them. `gitCommit` is injected via `-ldflags`; never hardcode it.
+
+## Comments policy (gean)
+Assume an experienced protocol engineer reads the code. Comment only **protocol reasoning, consensus nuance, security implications, non-obvious invariants, edge cases, and spec mappings** — never restate the code. **Do not copy or cite other clients' comments, or name other clients in code** (no ethlambda/zeam/etc. references); write short, gean-specific comments. Prefer code that needs no comment.
+
+## Code quality & Go standards
+Deterministic, testable, observable, composable. Avoid giant functions, hidden side effects, deep nesting, premature abstraction, duplicated logic. Prefer small focused functions, explicit data flow, clear ownership.
+- Propagate `context` correctly; use structured/typed errors in the `statetransition` style and wrap with `%w` (so `errors.As`/`errors.Is` work across layers); table-driven tests.
+- **Explicit concurrency ownership.** The model is a single-threaded Engine loop + off-tick workers + per-attestation verify goroutines, with shared maps like `AttestationSignatureMap` mutex-protected. Any store access from a worker goroutine must be safe. No goroutine leaks, hidden state mutation, or package cycles. Interfaces belong near consumers, not producers.
+
+## Performance
+Measure, then benchmark, then optimize — never blindly. Mind allocations, GC pressure, lock contention, and the proving hot path (XMSS proving is slow; amd64 builds with `-Ctarget-cpu=haswell` for AVX2). Keep proving off the tick loop so the clock stays accurate. Document tradeoffs.
+
+## Dependencies
+Before adding one: is it already in gean, can the stdlib do it, is it maintained, is it protocol-critical, is it worth the operational cost? Minimize surface area — new crates in the Rust workspace bear the same scrutiny.
+
+## Attribution
+Author as `mananuf` (or existing repo conventions). **Never** add Claude/Anthropic/AI references or `Co-Authored-By: Claude` / "Generated with Claude Code" to code, commits, PRs, or issues.
+
+## Reviewability
+Every change is the smallest correct change, easy to reason about, architecture-aligned, spec-compliant, production-ready. Split unrelated changes (e.g. a spec-pin bump) into their own commits.
+
+## Required post-implementation review
+After every change, run a real self-review — the equivalent of `/lean-review`; also `/code-review` for correctness, `/spec-compliant` for spec drift, and `/verify` or the `devnet-runner`/`devnet-log-review` skills for live behavior. Produce findings before declaring done:
+- **Correctness** — works? edge cases? invariants preserved (fork-choice weights, vote-index remap on prune, `state_root` verification)?
+- **Simplicity** — can code be removed or logic simplified?
+- **Spec compliance** — matches the pinned spec? assumptions justified?
+- **Architecture** — fits gean patterns? ownership clear? FFI/sszgen rules honored?
+- **Performance** — needless allocations, locks, or abstractions?
+- **Testing** — happy / edge / failure / concurrency coverage; spec fixtures (`make test-spec`) still green for consensus changes.
+
+## Verification gate
+For consensus/protocol changes, `make build`, the relevant `go test` (plus `make test-spec` for spec-touching work), and `make lint` must pass before the task is complete.
+
+## Final rule
+Do not behave like a code generator. Behave like a senior protocol engineer responsible for a production lean-consensus client that real operators run and other clients interoperate with. Every line earns its existence.

.claude/skills/gean-perf/SKILL.md

@@ -0,0 +1,62 @@
+---
+name: gean-perf
+description: Stress-test gean's performance and read its metrics against the slot/interval budget. Use this when the user wants to stress-test gean on Shadow (single-client, multi-client, or multi-subnet), sweep the XMSS prover-cost rates, capture metrics from a Shadow or devnet run, understand what a gean metric means or what its expected budget is, diagnose why finalization/justification is stalling or the tick is drifting, or decide which gean performance area needs a fix. Covers the leanEthereum recursive-aggregation-vs-slot-interval debate.
+---
+
+# gean performance & metrics
+
+Stress-test gean and read its metrics against the 800ms-interval / 4s-slot budget and the XMSS
+prover cost. The teaching detail lives in the **Shadow Simulation Mastery Guide** — the sibling
+`../shadow` repo (Markdown chapters + a React app); this skill orchestrates it.
+
+- **Metrics** — `../shadow/guide/11-reading-the-metrics.md` ("Metrics mastery — beginners to pro"):
+  every key metric with meaning → budget → how to read it, plus the failure chain and the cost cliff.
+- **Runbook + matrix** — `../shadow/guide/07`–`10` (setup → single → multi-subnet → interop) and
+  `../shadow/guide/13-reference-cheatsheet.md`.
+- **Runner** — the **`lean-shadow-fuzzer`** (`../lean-shadow-fuzzer`) drives gean under Shadow the
+  same way it drives every client (its `config.toml` + `scripts/client-cmds/gean-cmd.sh`). gean ships
+  only the rate model (`internal/shadow`) + the `--shadow-xmss-*` flags, published as
+  `ghcr.io/geanlabs/gean:shadow` for the fuzzer to pull — no harness lives in the gean repo.
+
+## When to use
+
+- "Stress-test gean on Shadow" / "run a multi-subnet test" / "sweep the prover rates"
+- "What does `lean_<metric>` mean?" / "what's the budget for it?" / "is this value healthy?"
+- "Why is finalization stalling / the tick drifting under load?"
+- "Which part of gean needs a performance fix?" (evidence-gated — see below)
+
+## Workflow
+
+1. **Frame the question against the budget.** Load the metrics chapter
+   (`../shadow/guide/11-reading-the-metrics.md`). The timing model (slot 4s = 5×800ms intervals;
+   aggregation session ~1600ms; proposal ~2400ms; per-attestation verify ~500ms) is where every
+   budget comes from. Read a run in this order: liveness triple → tick duration → aggregation chain
+   → drop/queue counters → RSS.
+
+2. **Run the matrix.** Drive Shadow through the **`lean-shadow-fuzzer`** (Mastery Guide chapters
+   `07`–`10`): describe a network in its `config.toml` (nodes, committee count, per-client images,
+   prover-cost rates) and it generates genesis/topology/`shadow.yaml`, runs Shadow, and renders the
+   Observatory. Shadow finds the *breaking rate* deterministically; `run-devnet` (devnet-runner skill)
+   shows *absolute magnitude* + live Grafana on real proving.
+
+3. **Read the result vs the limit.** For each metric, compare against its budget in the metrics
+   chapter. The failure chain to watch: `proving_duration{aggregation}` rises →
+   `aggregation_worker_total_time` crosses ~1.6s → aggregation `truncated` →
+   `aggregation_dispatch_dropped_total` climbs + `proving_queue_depth{aggregation}` pins at 1 →
+   `tick_interval_duration` drifts past 0.82s → `head − finalized` widens.
+
+4. **Gate any fix on a signal.** Never propose a performance change without the metric that
+   justifies it first crossing threshold in the matrix. Candidate fixes and their gating signals
+   are catalogued in the metrics chapter (Part D) and in the effort's plan (Deliverable C):
+   aggregation-dispatch capacity, bounded verify concurrency, PubKeyCache/lock contention,
+   raw-signature priority under budget pressure (the zeam/ethlambda proposal), session-budget
+   tuning, longer slot. Measure → report → then change.
+
+## Conventions
+
+- Spec is the source of truth: interpret load behavior (finalization, attestation source per
+  leanSpec #1166, subnet routing `validator_index % committee_count`) against the pinned spec, and
+  run `/spec-compliant` when a change could affect consensus.
+- Harness edits are opt-in and must not affect normal builds. gean code fixes pass the
+  verification gate (`make build`, relevant `go test` + `make test-spec` for spec-touching,
+  `make lint`) and are committed separately, authored as the repo convention, no AI attribution.

.github/workflows/shadow-rebase.yml

@@ -0,0 +1,58 @@
+name: Shadow Branch Auto-Rebase
+
+# Keeps the long-lived `shadow` branch (Shadow network-simulator harness, kept off
+# main) rebased onto main. Runs on every push to main and on demand. On conflict the
+# job fails so the conflict surfaces in the Actions run and its summary.
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: shadow-rebase
+  cancel-in-progress: true
+
+jobs:
+  rebase-shadow:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Configure git
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Rebase shadow onto main
+        id: rebase
+        run: |
+          if ! git ls-remote --heads origin shadow | grep -q 'refs/heads/shadow'; then
+            echo "shadow branch not found on remote — nothing to rebase"
+            echo "rebased=false" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          git checkout -B shadow origin/shadow
+          if git rebase origin/main; then
+            echo "rebased=true" >> "$GITHUB_OUTPUT"
+          else
+            conflicts=$(git diff --name-only --diff-filter=U | paste -sd ', ' -)
+            git rebase --abort || true
+            {
+              echo "## Shadow rebase failed"
+              echo "\`shadow\` could not be rebased onto \`main\`; manual intervention required."
+              echo "Conflicts: ${conflicts:-unknown}"
+            } >> "$GITHUB_STEP_SUMMARY"
+            exit 1
+          fi
+
+      - name: Push rebased shadow
+        if: steps.rebase.outputs.rebased == 'true'
+        run: git push origin shadow --force-with-lease

.gitignore

@@ -8,6 +8,11 @@ data/
 # Local testnet config (generated by keygen)
 testnet/
 
+# Shadow simulator artifacts (generated by shadow/gen_shadow_yaml.sh)
+shadow/shadow.yaml
+shadow/net/
+shadow/shadow.data/
+
 # Spec fixtures (cloned separately)
 leanSpec/
 

Makefile

@@ -128,7 +128,7 @@ leanSpec/fixtures/.generated-$(LEAN_SPEC_COMMIT_HASH): leanSpec/.pinned-$(LEAN_S
 
 DOCKER_TAG ?= local
 
-docker-build: ## Build Docker image
+docker-build: ## Build Docker image (also tagged :shadow for the lean-shadow-fuzzer)
 	docker build \
 		--build-arg GIT_COMMIT=$(GIT_COMMIT) \
 		--build-arg GIT_BRANCH=$(GIT_BRANCH) \

SECURITY.md

@@ -0,0 +1,36 @@
+# Security policy
+
+## Reporting a vulnerability
+
+Please report vulnerabilities privately via [GitHub Security Advisories](https://github.com/geanlabs/gean/security/advisories/new). Do not open a public issue for anything security-sensitive.
+
+Include what you can: affected component, reproduction steps or a failing input, and impact as you understand it. We will acknowledge reports as quickly as we can and keep you informed as we triage and fix.
+
+Gean is pre-production devnet software with no bug bounty program yet. We still want every report — consensus bugs found now are far cheaper than consensus bugs found on a live network.
+
+## Scope
+
+The areas where bugs are most likely to be consensus- or security-critical:
+
+- State transition (`internal/statetransition/`) and consensus types/SSZ (`internal/types/`)
+- Fork choice (`internal/forkchoice/`)
+- XMSS signatures and the Rust FFI boundary (`xmss/`)
+- Networking and wire decoding (`internal/p2p/`)
+- Block import and attestation validation (`internal/blockprocessor/`, `internal/attestation/`, `internal/aggregation/`)
+
+Divergence from the pinned [leanSpec](https://github.com/leanEthereum/leanSpec) reference is a bug even when nothing crashes — cross-client consensus splits are the failure mode we care most about.
+
+## Supported versions
+
+Only the latest commit on the current devnet branch is supported. Older devnet branches are kept for reference and do not receive fixes.
+
+## Official sources
+
+The only official distribution channels for Gean are:
+
+- Source code: [github.com/geanlabs/gean](https://github.com/geanlabs/gean)
+- Website: [geanlabs.com](https://geanlabs.com)
+
+Gean Labs does not publish prebuilt binaries, installers, or release archives. Build from source using the instructions in the [README](README.md). Any repository, website, or download claiming to offer Gean binaries is unofficial and should be treated as malicious.
+
+If you find a site or repository impersonating Gean, report it via [GitHub Security Advisories](https://github.com/geanlabs/gean/security/advisories/new) or by opening an issue, and report it to GitHub at [github.com/contact/report-abuse](https://github.com/contact/report-abuse).