diff --git a/.claude/commands/gean-engineer.md b/.claude/commands/gean-engineer.md new file mode 100644 index 0000000..df56425 --- /dev/null +++ b/.claude/commands/gean-engineer.md @@ -0,0 +1,77 @@ +--- +description: gean protocol-engineering directive — apply before any fix, feature, refactor, optimization, or test. Enforces spec-first development, gean architecture/FFI/sszgen rules, and a required post-implementation review. +argument-hint: [task to implement] +--- + +You are working inside **gean**, a Go implementation of an Ethereum **lean consensus** client (`github.com/geanlabs/gean`), targeting the **devnet-5** milestone and tracking the `leanEthereum/leanSpec` Python spec pinned at `LEAN_SPEC_COMMIT_HASH` in the `Makefile`. Consensus signatures are **XMSS** post-quantum hash-based signatures, implemented in Rust and reached over CGo FFI. + +The task to implement: + +$ARGUMENTS + +Before writing any code for that task, follow these rules. gean runs for real operators and interoperates in multi-client lean devnets alongside **ethlambda, zeam, ream, grandine, lantern, nlean, and qlean** — code that is "green locally" but diverges from the spec breaks the network. Engineer accordingly. + +## Primary objective +Optimize in this order: **spec compliance → consensus/protocol correctness → safety → clarity & maintainability → performance → operational excellence.** "Making it work" is not the goal; improving the long-term health of the client is. When goals conflict the higher one wins, and **the spec always wins.** + +## The spec is the source of truth +`leanSpec` (pinned at `LEAN_SPEC_COMMIT_HASH`) defines correct behavior — not existing gean code, not another client. +- Read the relevant spec section before touching consensus logic. It concentrates in `internal/statetransition/` (pure `ProcessSlots → ProcessBlock → verify state_root`), `internal/forkchoice/` (LMD-GHOST over ProtoArray + VoteStore), and `internal/types/`. +- State your spec mapping internally: which spec file/function the change mirrors, why it complies, and the edge cases. +- When gean and the spec disagree, fix gean. When the spec is genuinely ambiguous, prefer what the interop peers do and surface it. Mirror spec control flow (e.g. catch-and-skip duty patterns) rather than inventing gean-specific behavior. Use `/spec-compliant` to check for drift. + +## Engineering philosophy +Prefer reduction over addition, simplicity over cleverness, readability over abstraction, composition over inheritance, explicitness over magic, determinism over assumption. Before adding code, ask: +1. Can this be solved by deleting code? +2. Can an existing gean abstraction be reused (`store`, `forkchoice`, `types`, `xmss`, `pending` buffers, `metrics`, `logger`, `dutygate`, `role`)? +3. Does the existing architecture already support this? +4. Is this actually a protocol concern, or incidental? +5. Does this complexity earn its keep? + +The best change is often a smaller one. `/simplify` and `/lean-review` exist to push on this. + +## Repository alignment +Make the repo read as if one team wrote every line. +- Respect gean's boundaries: `Engine` (`internal/node/`) owns runtime wiring and is single-threaded over the `select` loop in `Run`/`onTick`; expensive XMSS proving runs **off** the tick loop on dedicated workers (aggregation, proposal, recovery). **`ForkChoice` does not live inside `ConsensusStore`** — the Engine calls fork choice with store data as parameters. +- Preserve the slot/interval model (4s slot, 5×800ms intervals; interval responsibilities in `onTick`) and the spec's head-update-vs-proposal ordering. +- Match surrounding naming, error handling, and testing patterns. Don't restructure files or add directories without strong justification. + +## Generated code & build ordering (gean specifics) +- **Never hand-edit `internal/types/*_encoding.go`** — they are generated. Change the struct + SSZ tags and run `make sszgen`. If generation fails, stop and report; do not hand-write SSZ. +- **Rust FFI builds before any Go that touches `xmss`.** Use the make targets: `make build`, `make ffi`, `make test` (excludes xmss/spectests/cmd), `make test-ffi`, `make test-spec`, `make test-all`, `make lint` (`go vet` + `cargo fmt --check` + `cargo clippy -D warnings`), `make fmt`. A plain `go test ./...` fails to link unless `make ffi` ran. +- Buffer caps and consensus limits are package-level constants aligned with the pinned spec — keep them in sync, don't fork them. `gitCommit` is injected via `-ldflags`; never hardcode it. + +## Comments policy (gean) +Assume an experienced protocol engineer reads the code. Comment only **protocol reasoning, consensus nuance, security implications, non-obvious invariants, edge cases, and spec mappings** — never restate the code. **Do not copy or cite other clients' comments, or name other clients in code** (no ethlambda/zeam/etc. references); write short, gean-specific comments. Prefer code that needs no comment. + +## Code quality & Go standards +Deterministic, testable, observable, composable. Avoid giant functions, hidden side effects, deep nesting, premature abstraction, duplicated logic. Prefer small focused functions, explicit data flow, clear ownership. +- Propagate `context` correctly; use structured/typed errors in the `statetransition` style and wrap with `%w` (so `errors.As`/`errors.Is` work across layers); table-driven tests. +- **Explicit concurrency ownership.** The model is a single-threaded Engine loop + off-tick workers + per-attestation verify goroutines, with shared maps like `AttestationSignatureMap` mutex-protected. Any store access from a worker goroutine must be safe. No goroutine leaks, hidden state mutation, or package cycles. Interfaces belong near consumers, not producers. + +## Performance +Measure, then benchmark, then optimize — never blindly. Mind allocations, GC pressure, lock contention, and the proving hot path (XMSS proving is slow; amd64 builds with `-Ctarget-cpu=haswell` for AVX2). Keep proving off the tick loop so the clock stays accurate. Document tradeoffs. + +## Dependencies +Before adding one: is it already in gean, can the stdlib do it, is it maintained, is it protocol-critical, is it worth the operational cost? Minimize surface area — new crates in the Rust workspace bear the same scrutiny. + +## Attribution +Author as `mananuf` (or existing repo conventions). **Never** add Claude/Anthropic/AI references or `Co-Authored-By: Claude` / "Generated with Claude Code" to code, commits, PRs, or issues. + +## Reviewability +Every change is the smallest correct change, easy to reason about, architecture-aligned, spec-compliant, production-ready. Split unrelated changes (e.g. a spec-pin bump) into their own commits. + +## Required post-implementation review +After every change, run a real self-review — the equivalent of `/lean-review`; also `/code-review` for correctness, `/spec-compliant` for spec drift, and `/verify` or the `devnet-runner`/`devnet-log-review` skills for live behavior. Produce findings before declaring done: +- **Correctness** — works? edge cases? invariants preserved (fork-choice weights, vote-index remap on prune, `state_root` verification)? +- **Simplicity** — can code be removed or logic simplified? +- **Spec compliance** — matches the pinned spec? assumptions justified? +- **Architecture** — fits gean patterns? ownership clear? FFI/sszgen rules honored? +- **Performance** — needless allocations, locks, or abstractions? +- **Testing** — happy / edge / failure / concurrency coverage; spec fixtures (`make test-spec`) still green for consensus changes. + +## Verification gate +For consensus/protocol changes, `make build`, the relevant `go test` (plus `make test-spec` for spec-touching work), and `make lint` must pass before the task is complete. + +## Final rule +Do not behave like a code generator. Behave like a senior protocol engineer responsible for a production lean-consensus client that real operators run and other clients interoperate with. Every line earns its existence. diff --git a/.claude/skills/gean-perf/SKILL.md b/.claude/skills/gean-perf/SKILL.md new file mode 100644 index 0000000..223d4f6 --- /dev/null +++ b/.claude/skills/gean-perf/SKILL.md @@ -0,0 +1,62 @@ +--- +name: gean-perf +description: Stress-test gean's performance and read its metrics against the slot/interval budget. Use this when the user wants to stress-test gean on Shadow (single-client, multi-client, or multi-subnet), sweep the XMSS prover-cost rates, capture metrics from a Shadow or devnet run, understand what a gean metric means or what its expected budget is, diagnose why finalization/justification is stalling or the tick is drifting, or decide which gean performance area needs a fix. Covers the leanEthereum recursive-aggregation-vs-slot-interval debate. +--- + +# gean performance & metrics + +Stress-test gean and read its metrics against the 800ms-interval / 4s-slot budget and the XMSS +prover cost. The teaching detail lives in the **Shadow Simulation Mastery Guide** — the sibling +`../shadow` repo (Markdown chapters + a React app); this skill orchestrates it. + +- **Metrics** — `../shadow/guide/11-reading-the-metrics.md` ("Metrics mastery — beginners to pro"): + every key metric with meaning → budget → how to read it, plus the failure chain and the cost cliff. +- **Runbook + matrix** — `../shadow/guide/07`–`10` (setup → single → multi-subnet → interop) and + `../shadow/guide/13-reference-cheatsheet.md`. +- **Runner** — the **`lean-shadow-fuzzer`** (`../lean-shadow-fuzzer`) drives gean under Shadow the + same way it drives every client (its `config.toml` + `scripts/client-cmds/gean-cmd.sh`). gean ships + only the rate model (`internal/shadow`) + the `--shadow-xmss-*` flags, published as + `ghcr.io/geanlabs/gean:shadow` for the fuzzer to pull — no harness lives in the gean repo. + +## When to use + +- "Stress-test gean on Shadow" / "run a multi-subnet test" / "sweep the prover rates" +- "What does `lean_` mean?" / "what's the budget for it?" / "is this value healthy?" +- "Why is finalization stalling / the tick drifting under load?" +- "Which part of gean needs a performance fix?" (evidence-gated — see below) + +## Workflow + +1. **Frame the question against the budget.** Load the metrics chapter + (`../shadow/guide/11-reading-the-metrics.md`). The timing model (slot 4s = 5×800ms intervals; + aggregation session ~1600ms; proposal ~2400ms; per-attestation verify ~500ms) is where every + budget comes from. Read a run in this order: liveness triple → tick duration → aggregation chain + → drop/queue counters → RSS. + +2. **Run the matrix.** Drive Shadow through the **`lean-shadow-fuzzer`** (Mastery Guide chapters + `07`–`10`): describe a network in its `config.toml` (nodes, committee count, per-client images, + prover-cost rates) and it generates genesis/topology/`shadow.yaml`, runs Shadow, and renders the + Observatory. Shadow finds the *breaking rate* deterministically; `run-devnet` (devnet-runner skill) + shows *absolute magnitude* + live Grafana on real proving. + +3. **Read the result vs the limit.** For each metric, compare against its budget in the metrics + chapter. The failure chain to watch: `proving_duration{aggregation}` rises → + `aggregation_worker_total_time` crosses ~1.6s → aggregation `truncated` → + `aggregation_dispatch_dropped_total` climbs + `proving_queue_depth{aggregation}` pins at 1 → + `tick_interval_duration` drifts past 0.82s → `head − finalized` widens. + +4. **Gate any fix on a signal.** Never propose a performance change without the metric that + justifies it first crossing threshold in the matrix. Candidate fixes and their gating signals + are catalogued in the metrics chapter (Part D) and in the effort's plan (Deliverable C): + aggregation-dispatch capacity, bounded verify concurrency, PubKeyCache/lock contention, + raw-signature priority under budget pressure (the zeam/ethlambda proposal), session-budget + tuning, longer slot. Measure → report → then change. + +## Conventions + +- Spec is the source of truth: interpret load behavior (finalization, attestation source per + leanSpec #1166, subnet routing `validator_index % committee_count`) against the pinned spec, and + run `/spec-compliant` when a change could affect consensus. +- Harness edits are opt-in and must not affect normal builds. gean code fixes pass the + verification gate (`make build`, relevant `go test` + `make test-spec` for spec-touching, + `make lint`) and are committed separately, authored as the repo convention, no AI attribution. diff --git a/.github/workflows/shadow-rebase.yml b/.github/workflows/shadow-rebase.yml new file mode 100644 index 0000000..bdc64ba --- /dev/null +++ b/.github/workflows/shadow-rebase.yml @@ -0,0 +1,58 @@ +name: Shadow Branch Auto-Rebase + +# Keeps the long-lived `shadow` branch (Shadow network-simulator harness, kept off +# main) rebased onto main. Runs on every push to main and on demand. On conflict the +# job fails so the conflict surfaces in the Actions run and its summary. + +on: + push: + branches: [main] + workflow_dispatch: + +permissions: + contents: write + +concurrency: + group: shadow-rebase + cancel-in-progress: true + +jobs: + rebase-shadow: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Configure git + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Rebase shadow onto main + id: rebase + run: | + if ! git ls-remote --heads origin shadow | grep -q 'refs/heads/shadow'; then + echo "shadow branch not found on remote — nothing to rebase" + echo "rebased=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + git checkout -B shadow origin/shadow + if git rebase origin/main; then + echo "rebased=true" >> "$GITHUB_OUTPUT" + else + conflicts=$(git diff --name-only --diff-filter=U | paste -sd ', ' -) + git rebase --abort || true + { + echo "## Shadow rebase failed" + echo "\`shadow\` could not be rebased onto \`main\`; manual intervention required." + echo "Conflicts: ${conflicts:-unknown}" + } >> "$GITHUB_STEP_SUMMARY" + exit 1 + fi + + - name: Push rebased shadow + if: steps.rebase.outputs.rebased == 'true' + run: git push origin shadow --force-with-lease diff --git a/.gitignore b/.gitignore index 758257a..261d6bb 100644 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,11 @@ data/ # Local testnet config (generated by keygen) testnet/ +# Shadow simulator artifacts (generated by shadow/gen_shadow_yaml.sh) +shadow/shadow.yaml +shadow/net/ +shadow/shadow.data/ + # Spec fixtures (cloned separately) leanSpec/ diff --git a/Makefile b/Makefile index 8f46dd7..dbc158d 100644 --- a/Makefile +++ b/Makefile @@ -128,7 +128,7 @@ leanSpec/fixtures/.generated-$(LEAN_SPEC_COMMIT_HASH): leanSpec/.pinned-$(LEAN_S DOCKER_TAG ?= local -docker-build: ## Build Docker image +docker-build: ## Build Docker image (also tagged :shadow for the lean-shadow-fuzzer) docker build \ --build-arg GIT_COMMIT=$(GIT_COMMIT) \ --build-arg GIT_BRANCH=$(GIT_BRANCH) \ diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..86abb8c --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,36 @@ +# Security policy + +## Reporting a vulnerability + +Please report vulnerabilities privately via [GitHub Security Advisories](https://github.com/geanlabs/gean/security/advisories/new). Do not open a public issue for anything security-sensitive. + +Include what you can: affected component, reproduction steps or a failing input, and impact as you understand it. We will acknowledge reports as quickly as we can and keep you informed as we triage and fix. + +Gean is pre-production devnet software with no bug bounty program yet. We still want every report — consensus bugs found now are far cheaper than consensus bugs found on a live network. + +## Scope + +The areas where bugs are most likely to be consensus- or security-critical: + +- State transition (`internal/statetransition/`) and consensus types/SSZ (`internal/types/`) +- Fork choice (`internal/forkchoice/`) +- XMSS signatures and the Rust FFI boundary (`xmss/`) +- Networking and wire decoding (`internal/p2p/`) +- Block import and attestation validation (`internal/blockprocessor/`, `internal/attestation/`, `internal/aggregation/`) + +Divergence from the pinned [leanSpec](https://github.com/leanEthereum/leanSpec) reference is a bug even when nothing crashes — cross-client consensus splits are the failure mode we care most about. + +## Supported versions + +Only the latest commit on the current devnet branch is supported. Older devnet branches are kept for reference and do not receive fixes. + +## Official sources + +The only official distribution channels for Gean are: + +- Source code: [github.com/geanlabs/gean](https://github.com/geanlabs/gean) +- Website: [geanlabs.com](https://geanlabs.com) + +Gean Labs does not publish prebuilt binaries, installers, or release archives. Build from source using the instructions in the [README](README.md). Any repository, website, or download claiming to offer Gean binaries is unofficial and should be treated as malicious. + +If you find a site or repository impersonating Gean, report it via [GitHub Security Advisories](https://github.com/geanlabs/gean/security/advisories/new) or by opening an issue, and report it to GitHub at [github.com/contact/report-abuse](https://github.com/contact/report-abuse). diff --git a/cmd/gean/flags.go b/cmd/gean/flags.go index 8c868d2..e8dc9c9 100644 --- a/cmd/gean/flags.go +++ b/cmd/gean/flags.go @@ -6,6 +6,7 @@ import ( "fmt" "io" "net" + "os" "path/filepath" "strconv" "strings" @@ -29,6 +30,14 @@ type config struct { committeeCountSet bool AggregateSubnetIDs []uint64 DataDir string + + // Shadow*Rate model XMSS prover cost for the Shadow network simulator, which + // does not charge CPU time. Each is in signature-units per second; a + // non-positive rate (the default) disables the delay, so real deployments are + // unaffected. + ShadowAggregateSignaturesRate float64 + ShadowVerifySignatureRate float64 + ShadowVerifyAggregatedSignaturesRate float64 } type configPaths struct { @@ -56,6 +65,9 @@ func parseConfig(args []string, stderr io.Writer) (config, error) { fs.Uint64Var(&cfg.CommitteeCount, "attestation-committee-count", uint64(types.AttestationCommitteeCount), "Number of attestation subnets (overrides config.yaml ATTESTATION_COMMITTEE_COUNT)") fs.StringVar(&aggregateSubnetIDs, "aggregate-subnet-ids", "", "Comma-separated subnet IDs (requires --is-aggregator)") fs.StringVar(&cfg.DataDir, "data-dir", "./data", "Pebble database directory") + fs.Float64Var(&cfg.ShadowAggregateSignaturesRate, "shadow-xmss-aggregate-signatures-rate", 0, "Shadow simulator: signatures/sec rate for aggregation cost; n-signature op sleeps n/rate sec (0 disables; env GEAN_SHADOW_XMSS_AGGREGATE_SIGNATURES_RATE)") + fs.Float64Var(&cfg.ShadowVerifySignatureRate, "shadow-xmss-verify-signature-rate", 0, "Shadow simulator: signatures/sec rate for gossip-attestation verify cost (0 disables; env GEAN_SHADOW_XMSS_VERIFY_SIGNATURE_RATE)") + fs.Float64Var(&cfg.ShadowVerifyAggregatedSignaturesRate, "shadow-xmss-verify-aggregated-signatures-rate", 0, "Shadow simulator: signatures/sec rate for aggregated-signature verify cost (0 disables; env GEAN_SHADOW_XMSS_VERIFY_AGGREGATED_SIGNATURES_RATE)") if err := fs.Parse(args); err != nil { return cfg, err @@ -88,6 +100,9 @@ func parseConfig(args []string, stderr io.Writer) (config, error) { fmt.Fprintln(stderr, "--aggregate-subnet-ids requires --is-aggregator") return cfg, errInvalidConfig } + if err := resolveShadowRates(fs, &cfg, stderr); err != nil { + return cfg, err + } subnetIDs, err := parseAggregateSubnetIDs(aggregateSubnetIDs, stderr) if err != nil { @@ -100,6 +115,48 @@ func parseConfig(args []string, stderr io.Writer) (config, error) { return cfg, nil } +// shadowRateSpec ties each Shadow prover-rate flag to its env-var fallback and +// the config field it fills. +type shadowRateSpec struct { + flag string + env string + target *float64 +} + +func shadowRateSpecs(cfg *config) []shadowRateSpec { + return []shadowRateSpec{ + {"shadow-xmss-aggregate-signatures-rate", "GEAN_SHADOW_XMSS_AGGREGATE_SIGNATURES_RATE", &cfg.ShadowAggregateSignaturesRate}, + {"shadow-xmss-verify-signature-rate", "GEAN_SHADOW_XMSS_VERIFY_SIGNATURE_RATE", &cfg.ShadowVerifySignatureRate}, + {"shadow-xmss-verify-aggregated-signatures-rate", "GEAN_SHADOW_XMSS_VERIFY_AGGREGATED_SIGNATURES_RATE", &cfg.ShadowVerifyAggregatedSignaturesRate}, + } +} + +// resolveShadowRates applies the GEAN_SHADOW_* env-var fallback for any prover-rate +// flag the user did not pass explicitly (precedence: flag > env > 0, matching the +// convention the other lean clients expose), then rejects negative rates. A Shadow +// harness can inject per-node rates via the environment without rewriting argv. +func resolveShadowRates(fs *flag.FlagSet, cfg *config, stderr io.Writer) error { + set := map[string]bool{} + fs.Visit(func(f *flag.Flag) { set[f.Name] = true }) + for _, sp := range shadowRateSpecs(cfg) { + if !set[sp.flag] { + if raw, ok := os.LookupEnv(sp.env); ok { + v, err := strconv.ParseFloat(strings.TrimSpace(raw), 64) + if err != nil { + fmt.Fprintf(stderr, "invalid %s=%q: %v\n", sp.env, raw, err) + return errInvalidConfig + } + *sp.target = v + } + } + if *sp.target < 0 { + fmt.Fprintf(stderr, "--%s must not be negative (env %s)\n", sp.flag, sp.env) + return errInvalidConfig + } + } + return nil +} + // resolveCommitteeCount picks the effective attestation committee count. // Precedence is the lean-network convention: an explicit // --attestation-committee-count flag wins, otherwise the shared config.yaml diff --git a/cmd/gean/flags_test.go b/cmd/gean/flags_test.go index 5b33757..ba5d258 100644 --- a/cmd/gean/flags_test.go +++ b/cmd/gean/flags_test.go @@ -31,6 +31,91 @@ func TestParseConfig_ValidDefaults(t *testing.T) { if cfg.IsAggregator || cfg.CommitteeCount != 1 || cfg.DataDir != "./data" || len(cfg.AggregateSubnetIDs) != 0 { t.Fatalf("unexpected role/storage defaults: %+v", cfg) } + if cfg.ShadowAggregateSignaturesRate != 0 || cfg.ShadowVerifySignatureRate != 0 || cfg.ShadowVerifyAggregatedSignaturesRate != 0 { + t.Fatalf("shadow rates must default to disabled: %+v", cfg) + } +} + +func TestParseConfig_ShadowRates(t *testing.T) { + args := append(validFlagArgs(), + "--shadow-xmss-aggregate-signatures-rate", "5", + "--shadow-xmss-verify-signature-rate", "2", + "--shadow-xmss-verify-aggregated-signatures-rate", "100", + ) + var stderr bytes.Buffer + cfg, err := parseConfig(args, &stderr) + if err != nil { + t.Fatalf("parseConfig returned error: %v\nstderr:\n%s", err, stderr.String()) + } + if cfg.ShadowAggregateSignaturesRate != 5 || + cfg.ShadowVerifySignatureRate != 2 || + cfg.ShadowVerifyAggregatedSignaturesRate != 100 { + t.Fatalf("unexpected shadow rates: %+v", cfg) + } +} + +func TestParseConfig_ShadowRatesFromEnv(t *testing.T) { + t.Setenv("GEAN_SHADOW_XMSS_AGGREGATE_SIGNATURES_RATE", "7.5") + t.Setenv("GEAN_SHADOW_XMSS_VERIFY_SIGNATURE_RATE", "3") + t.Setenv("GEAN_SHADOW_XMSS_VERIFY_AGGREGATED_SIGNATURES_RATE", "9") + + var stderr bytes.Buffer + cfg, err := parseConfig(validFlagArgs(), &stderr) + if err != nil { + t.Fatalf("parseConfig returned error: %v\nstderr:\n%s", err, stderr.String()) + } + if cfg.ShadowAggregateSignaturesRate != 7.5 || + cfg.ShadowVerifySignatureRate != 3 || + cfg.ShadowVerifyAggregatedSignaturesRate != 9 { + t.Fatalf("env rates not applied: %+v", cfg) + } +} + +func TestParseConfig_ShadowRateFlagBeatsEnv(t *testing.T) { + t.Setenv("GEAN_SHADOW_XMSS_AGGREGATE_SIGNATURES_RATE", "7.5") + + args := append(validFlagArgs(), "--shadow-xmss-aggregate-signatures-rate", "2") + var stderr bytes.Buffer + cfg, err := parseConfig(args, &stderr) + if err != nil { + t.Fatalf("parseConfig returned error: %v\nstderr:\n%s", err, stderr.String()) + } + if cfg.ShadowAggregateSignaturesRate != 2 { + t.Fatalf("flag should win over env, got %v", cfg.ShadowAggregateSignaturesRate) + } +} + +func TestParseConfig_InvalidShadowEnv(t *testing.T) { + t.Setenv("GEAN_SHADOW_XMSS_VERIFY_SIGNATURE_RATE", "not-a-number") + + var stderr bytes.Buffer + _, err := parseConfig(validFlagArgs(), &stderr) + if err == nil { + t.Fatal("expected unparseable env rate to fail") + } + if !strings.Contains(stderr.String(), "GEAN_SHADOW_XMSS_VERIFY_SIGNATURE_RATE") { + t.Fatalf("env error message not found:\n%s", stderr.String()) + } +} + +func TestParseConfig_NegativeShadowRate(t *testing.T) { + for _, flag := range []string{ + "--shadow-xmss-aggregate-signatures-rate", + "--shadow-xmss-verify-signature-rate", + "--shadow-xmss-verify-aggregated-signatures-rate", + } { + t.Run(flag, func(t *testing.T) { + args := append(validFlagArgs(), flag, "-1") + var stderr bytes.Buffer + _, err := parseConfig(args, &stderr) + if err == nil { + t.Fatal("expected negative shadow rate to fail") + } + if !strings.Contains(stderr.String(), flag+" must not be negative") { + t.Fatalf("negative rate message not found:\n%s", stderr.String()) + } + }) + } } func TestParseConfig_MissingRequiredFlags(t *testing.T) { diff --git a/cmd/gean/main.go b/cmd/gean/main.go index 64facc3..3162e02 100644 --- a/cmd/gean/main.go +++ b/cmd/gean/main.go @@ -10,6 +10,7 @@ import ( "github.com/geanlabs/gean/internal/metrics" "github.com/geanlabs/gean/internal/node" "github.com/geanlabs/gean/internal/role" + "github.com/geanlabs/gean/internal/shadow" ) func main() { @@ -83,7 +84,12 @@ func run(cfg config) error { } aggCtl := role.NewWithHook(cfg.IsAggregator, metrics.SetIsAggregator) - n := node.New(s, fc, p2pHost, inputs.keyManager, aggCtl, cfg.CommitteeCount) + shadowRates := shadow.Rates{ + AggregateSignatures: cfg.ShadowAggregateSignaturesRate, + VerifySignature: cfg.ShadowVerifySignatureRate, + VerifyAggregatedSignatures: cfg.ShadowVerifyAggregatedSignaturesRate, + } + n := node.New(s, fc, p2pHost, inputs.keyManager, aggCtl, cfg.CommitteeCount, shadowRates) startNodeNetworking(ctx, n, s, p2pHost, inputs.bootnodes) apiAddr, metricsAddr := startHTTPServers(cfg, s, fc, aggCtl) diff --git a/cmd/keygen/main.go b/cmd/keygen/main.go index 9904bf0..bb4b660 100644 --- a/cmd/keygen/main.go +++ b/cmd/keygen/main.go @@ -43,7 +43,10 @@ func run(args []string, stderr io.Writer) error { len(m.Validators), len(m.Nodes)) } - genesisTime := uint64(time.Now().Unix()) + 30 + genesisTime := opts.GenesisTime + if genesisTime == 0 { + genesisTime = uint64(time.Now().Unix()) + uint64(opts.GenesisDelay) + } if err := writeConfigYAML(opts.OutputDir, genesisTime, m.Validators); err != nil { return err } @@ -66,6 +69,8 @@ func parseOptions(args []string, stderr io.Writer) (options, error) { fs.IntVar(&opts.Nodes, "nodes", 3, "Number of nodes") fs.StringVar(&opts.OutputDir, "output", "testnet", "Output directory") fs.IntVar(&opts.BasePort, "base-port", 9000, "Base P2P port (incremented per node)") + fs.Uint64Var(&opts.GenesisTime, "genesis-time", 0, "Absolute genesis Unix time; 0 uses now + --genesis-delay") + fs.IntVar(&opts.GenesisDelay, "genesis-delay", 30, "Seconds from now until genesis when --genesis-time is unset") if err := fs.Parse(args); err != nil { return opts, err @@ -82,13 +87,16 @@ func parseOptions(args []string, stderr io.Writer) (options, error) { if opts.Nodes > 65535-opts.BasePort+1 { return opts, fmt.Errorf("%w: base port range exceeds 1..65535", errInvalidOptions) } + if opts.GenesisDelay < 0 { + return opts, fmt.Errorf("%w: genesis delay must be >= 0", errInvalidOptions) + } return opts, nil } func logSummary(opts options, genesisTime uint64, m manifest) { log.Println("---") log.Printf("output: %s", opts.OutputDir) - log.Printf("genesis time: %d (in 30 seconds: %s)", genesisTime, + log.Printf("genesis time: %d (%s)", genesisTime, time.Unix(int64(genesisTime), 0).Format(time.RFC3339)) log.Printf("validators: %d, nodes: %d", len(m.Validators), len(m.Nodes)) log.Println("") diff --git a/cmd/keygen/model.go b/cmd/keygen/model.go index 7801702..a8ab2c7 100644 --- a/cmd/keygen/model.go +++ b/cmd/keygen/model.go @@ -1,10 +1,12 @@ package main type options struct { - Validators int - Nodes int - OutputDir string - BasePort int + Validators int + Nodes int + OutputDir string + BasePort int + GenesisTime uint64 + GenesisDelay int } type manifest struct { diff --git a/internal/aggregation/aggregate.go b/internal/aggregation/aggregate.go index 0cc8ff5..71a59d0 100644 --- a/internal/aggregation/aggregate.go +++ b/internal/aggregation/aggregate.go @@ -8,6 +8,7 @@ import ( "github.com/geanlabs/gean/internal/logger" "github.com/geanlabs/gean/internal/metrics" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/store" "github.com/geanlabs/gean/internal/types" "github.com/geanlabs/gean/xmss" @@ -82,7 +83,9 @@ func (e *unitCostEstimator) maxUnitsWithin(budget time.Duration) int { } // observe folds a completed aggregation's realized per-unit cost into the estimate -// with an exponential moving average, damping single-pass noise. +// with an exponential moving average, damping single-pass noise. Under Shadow the +// duration includes the modeled prover sleep, so the estimate calibrates to the +// simulated cost just as it would to real hardware. func (e *unitCostEstimator) observe(duration time.Duration, units int) { if e == nil || units <= 0 || duration <= 0 { return @@ -92,7 +95,7 @@ func (e *unitCostEstimator) observe(duration time.Duration, units int) { e.perUnitSeconds = alpha*sample + (1-alpha)*e.perUnitSeconds } -func aggregateFromSnapshot(snap *Snapshot, cache *xmss.PubKeyCache, deadline time.Time, estimator *unitCostEstimator) ([]*types.SignedAggregatedAttestation, []store.PayloadKV, []store.AttestationDeleteKey, bool) { +func aggregateFromSnapshot(snap *Snapshot, cache *xmss.PubKeyCache, deadline time.Time, shadowRates shadow.Rates, estimator *unitCostEstimator) ([]*types.SignedAggregatedAttestation, []store.PayloadKV, []store.AttestationDeleteKey, bool) { if snap == nil || cache == nil { return nil, nil, nil, false } @@ -198,6 +201,10 @@ func aggregateFromSnapshot(snap *Snapshot, cache *xmss.PubKeyCache, deadline tim aggStart := time.Now() proofBytes, err := xmss.AggregateWithChildren(*rawPubkeysBuf, *rawSigsBuf, *childProofsBuf, dataRootHash, slot) + // Charge virtual time for the proving cost Shadow would otherwise not + // account; the capacity-1 dispatch channel then drops the next slot's + // work if proving can't keep up, exactly as on real hardware. + shadowRates.SleepAggregate(len(*rawIDsBuf) + len(*childProofsBuf)) aggDuration := time.Since(aggStart) if err != nil { logger.Error(logger.Signature, "aggregate: failed slot=%d raw=%d children=%d duration=%v: %v", diff --git a/internal/aggregation/aggregate_test.go b/internal/aggregation/aggregate_test.go index 2e4e9ce..4e9a06e 100644 --- a/internal/aggregation/aggregate_test.go +++ b/internal/aggregation/aggregate_test.go @@ -4,6 +4,7 @@ import ( "testing" "time" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/store" "github.com/geanlabs/gean/internal/types" "github.com/geanlabs/gean/xmss" @@ -92,7 +93,7 @@ func TestAggregateFromSnapshotExpiredDeadlineReportsTruncation(t *testing.T) { snap := aggregateTestSnapshot(5) cache := xmss.NewPubKeyCache() - aggs, payloads, deletes, truncated := aggregateFromSnapshot(snap, cache, time.Now().Add(-time.Second), newUnitCostEstimator()) + aggs, payloads, deletes, truncated := aggregateFromSnapshot(snap, cache, time.Now().Add(-time.Second), shadow.Rates{}, newUnitCostEstimator()) if !truncated { t.Fatal("expected truncation with expired deadline") @@ -105,7 +106,7 @@ func TestAggregateFromSnapshotExpiredDeadlineReportsTruncation(t *testing.T) { func TestAggregateFromSnapshotZeroDeadlineProcessesAll(t *testing.T) { snap := aggregateTestSnapshot(5) - _, _, _, truncated := aggregateFromSnapshot(snap, xmss.NewPubKeyCache(), time.Time{}, newUnitCostEstimator()) + _, _, _, truncated := aggregateFromSnapshot(snap, xmss.NewPubKeyCache(), time.Time{}, shadow.Rates{}, newUnitCostEstimator()) if truncated { t.Fatal("zero deadline must never truncate") diff --git a/internal/aggregation/worker.go b/internal/aggregation/worker.go index cfe4260..f8f7612 100644 --- a/internal/aggregation/worker.go +++ b/internal/aggregation/worker.go @@ -7,6 +7,7 @@ import ( "github.com/geanlabs/gean/internal/logger" "github.com/geanlabs/gean/internal/metrics" "github.com/geanlabs/gean/internal/proving" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/store" "github.com/geanlabs/gean/internal/types" "github.com/geanlabs/gean/xmss" @@ -34,6 +35,7 @@ func RunWorker( cache *xmss.PubKeyCache, publisher Publisher, gate *proving.Gate, + shadowRates shadow.Rates, ) { // One estimator lives across dispatches so it keeps calibrating to this // node's real per-unit prover cost. The worker is single-threaded, so no @@ -66,7 +68,7 @@ func RunWorker( // (and their signature deletes) regrows the next snapshot until // no session can ever finish inside a slot. workerStart := time.Now() - aggs, payloads, deletes, truncated := aggregateFromSnapshot(dispatch.Snapshot, cache, workerStart.Add(sessionBudget), estimator) + aggs, payloads, deletes, truncated := aggregateFromSnapshot(dispatch.Snapshot, cache, workerStart.Add(sessionBudget), shadowRates, estimator) if gate != nil { gate.Release(false) } diff --git a/internal/aggregation/worker_test.go b/internal/aggregation/worker_test.go index 92dede7..a4b92ff 100644 --- a/internal/aggregation/worker_test.go +++ b/internal/aggregation/worker_test.go @@ -6,6 +6,7 @@ import ( "testing" "time" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/types" ) @@ -25,7 +26,7 @@ func TestRunWorkerReturnsWhenDispatchChannelCloses(t *testing.T) { done := make(chan struct{}) go func() { - RunWorker(context.Background(), dispatches, nil, nil, nil, nil) + RunWorker(context.Background(), dispatches, nil, nil, nil, nil, shadow.Rates{}) close(done) }() @@ -39,7 +40,7 @@ func TestRunWorkerSkipsNilSnapshot(t *testing.T) { done := make(chan struct{}) go func() { - RunWorker(context.Background(), dispatches, nil, nil, nil, nil) + RunWorker(context.Background(), dispatches, nil, nil, nil, nil, shadow.Rates{}) close(done) }() diff --git a/internal/node/engine.go b/internal/node/engine.go index 3b4ab97..7a605a7 100644 --- a/internal/node/engine.go +++ b/internal/node/engine.go @@ -12,6 +12,7 @@ import ( "github.com/geanlabs/gean/internal/pending" "github.com/geanlabs/gean/internal/proving" "github.com/geanlabs/gean/internal/role" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/store" "github.com/geanlabs/gean/internal/types" "github.com/geanlabs/gean/xmss" @@ -37,6 +38,7 @@ type Engine struct { AggCtl *role.Controller DutyGate *dutygate.Gate CommitteeCount uint64 + Shadow shadow.Rates Pending *pending.BlockBuffer PendingAttestations *pending.AttestationBuffer @@ -64,6 +66,7 @@ func New( keys *xmss.KeyManager, aggCtl *role.Controller, committeeCount uint64, + shadowRates shadow.Rates, ) *Engine { p2p.SetClientGitCommit(gitCommit) e := &Engine{ @@ -74,6 +77,7 @@ func New( AggCtl: aggCtl, DutyGate: dutygate.New(logDutyGateEvent), CommitteeCount: committeeCount, + Shadow: shadowRates, Pending: pending.NewBlockBuffer(), PendingAttestations: pending.NewAttestationBuffer(PendingAttestationsPerRootCap, PendingAttestationsTotalCap), BlockCh: make(chan *types.SignedBlock, 64), diff --git a/internal/node/gossip.go b/internal/node/gossip.go index 743f54a..5475d3e 100644 --- a/internal/node/gossip.go +++ b/internal/node/gossip.go @@ -49,6 +49,7 @@ func (e *Engine) onGossipAttestation(att *types.SignedAttestation) { metrics.IncPqSigAttestationSigsTotal() verifyStart := time.Now() err = attestation.VerifyGossipAttestation(e.Store, att.ValidatorID, att.Data, dataRoot, att.Signature[:]) + e.Shadow.SleepVerify() metrics.ObservePqSigVerificationTime(time.Since(verifyStart).Seconds()) if err != nil { metrics.IncPqSigAttestationSigsInvalid() @@ -79,6 +80,7 @@ func (e *Engine) onGossipAggregatedAttestation(agg *types.SignedAggregatedAttest } verifyStart := time.Now() err := attestation.VerifyAggregatedGossipAttestation(e.Store, agg.Data, agg.Proof.Participants, agg.Proof.Proof) + e.Shadow.SleepVerifyAggregated(int(types.BitlistCount(agg.Proof.Participants))) metrics.ObservePqSigAggVerificationTime(time.Since(verifyStart).Seconds()) if err != nil { metrics.IncPqSigAggregatedInvalid() diff --git a/internal/node/node_test.go b/internal/node/node_test.go index 20094dc..8f1fdc8 100644 --- a/internal/node/node_test.go +++ b/internal/node/node_test.go @@ -9,6 +9,7 @@ import ( "github.com/geanlabs/gean/internal/forkchoice" "github.com/geanlabs/gean/internal/logger" "github.com/geanlabs/gean/internal/role" + "github.com/geanlabs/gean/internal/shadow" "github.com/geanlabs/gean/internal/storage" "github.com/geanlabs/gean/internal/store" "github.com/geanlabs/gean/internal/syncer" @@ -46,7 +47,7 @@ func makeTestEngine() *Engine { fc := forkchoice.New(0, genesisRoot, [32]byte{}) - return New(s, fc, nil, nil, role.New(false), 1) + return New(s, fc, nil, nil, role.New(false), 1, shadow.Rates{}) } func TestEngineCreation(t *testing.T) { diff --git a/internal/node/workers.go b/internal/node/workers.go index 070e4c7..9f1a45c 100644 --- a/internal/node/workers.go +++ b/internal/node/workers.go @@ -8,7 +8,7 @@ import ( func (e *Engine) startWorkers(ctx context.Context) { go e.runFetchBatcher(ctx) - go aggregation.RunWorker(ctx, e.AggregationDispatchCh, e.Store, e.Store.PubKeyCache, e.P2P, e.ProvingGate) + go aggregation.RunWorker(ctx, e.AggregationDispatchCh, e.Store, e.Store.PubKeyCache, e.P2P, e.ProvingGate, e.Shadow) go e.runProposalWorker(ctx) go e.runRecoveryWorker(ctx) go e.runAttestationWorker(ctx) diff --git a/internal/shadow/shadow.go b/internal/shadow/shadow.go new file mode 100644 index 0000000..6d55c45 --- /dev/null +++ b/internal/shadow/shadow.go @@ -0,0 +1,47 @@ +// Package shadow models XMSS prover cost as artificial virtual-time sleeps for +// the Shadow network simulator, which does not charge CPU time — without them +// the prover would run "free" in virtual time, making multi-client sims +// unrepresentative. +// +// Each rate is expressed in signature-units per second: an n-unit operation +// sleeps n/rate seconds, mirroring the convention the other lean clients expose +// so a single sweep config feeds every client the same sig/s rates. A +// non-positive rate disables that delay, so real deployments (rates unset) are +// unaffected. +// +// Every Sleep* method must run off the Engine tick loop (aggregation worker, +// per-attestation verify goroutines) so the node clock stays accurate. +package shadow + +import "time" + +// Rates holds the per-operation prover rates. The zero value is fully disabled. +type Rates struct { + AggregateSignatures float64 + VerifySignature float64 + VerifyAggregatedSignatures float64 +} + +// SleepAggregate charges virtual time for aggregating units input signatures. +func (r Rates) SleepAggregate(units int) { + sleep(r.AggregateSignatures, units) +} + +// SleepVerify charges virtual time for verifying a single signature. +func (r Rates) SleepVerify() { + sleep(r.VerifySignature, 1) +} + +// SleepVerifyAggregated charges virtual time for verifying an aggregated +// signature over units participants. +func (r Rates) SleepVerifyAggregated(units int) { + sleep(r.VerifyAggregatedSignatures, units) +} + +func sleep(rate float64, units int) { + if rate <= 0 || units <= 0 { + return + } + // rate is signature-units per second, so an n-unit op costs n/rate seconds. + time.Sleep(time.Duration(float64(units) / rate * float64(time.Second))) +} diff --git a/internal/shadow/shadow_test.go b/internal/shadow/shadow_test.go new file mode 100644 index 0000000..4bb91b7 --- /dev/null +++ b/internal/shadow/shadow_test.go @@ -0,0 +1,40 @@ +package shadow + +import ( + "testing" + "time" +) + +// A non-positive rate or zero unit count must be a true no-op so production +// nodes (rates unset) never pay any delay. +func TestSleepDisabled(t *testing.T) { + cases := []struct { + name string + rate float64 + units int + }{ + {"zero rate", 0, 1_000_000}, + {"negative rate", -1, 1_000_000}, + {"zero units", 1, 0}, + {"negative units", 1, -5}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + start := time.Now() + sleep(tc.rate, tc.units) + if elapsed := time.Since(start); elapsed > time.Millisecond { + t.Fatalf("expected no-op, slept %v", elapsed) + } + }) + } +} + +// A positive rate sleeps the n/rate floor: more units cost more virtual time. +func TestSleepScalesWithUnits(t *testing.T) { + const rate = 1000.0 // 1000 signature-units per second → 1ms each + start := time.Now() + sleep(rate, 5) // expect ~5ms + if elapsed := time.Since(start); elapsed < 4*time.Millisecond { + t.Fatalf("expected at least ~5ms, slept %v", elapsed) + } +}