Skip to content

Commit 7d9bee6

Browse files
committed
evmigration fixes
1 parent 0550937 commit 7d9bee6

File tree

7 files changed

+212
-64
lines changed

7 files changed

+212
-64
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,4 +31,5 @@ supernode/__debug*
3131
/data
3232
/release
3333
AGENTS.md
34-
analysis
34+
analysis
35+
.claude/settings.json

docs/evm-migration.md

Lines changed: 85 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ delegations, supernode registration, and optionally validator state — to a new
1414
address derived from the same mnemonic under the EVM HD path.
1515

1616
The migration is:
17-
- **One-time**: runs automatically at supernode startup when a legacy key is detected
17+
18+
- **One-time**: runs automatically at superno0de startup when a legacy key is detected
1819
- **Rerunnable**: safe to retry if interrupted at any point
1920
- **Self-authenticating**: uses dual signatures (legacy + new key) embedded in the
2021
message, so no Cosmos-level tx signing is needed
@@ -24,57 +25,59 @@ The migration is:
2425
### Prerequisites
2526

2627
1. Your supernode binary must be the EVM-compatible version.
27-
2. The connected Lumera chain must have the `evm` module active.
28-
3. You need the **mnemonic** used to create your original supernode key.
28+
2. The connected Lumera chain must have the`evm` module active.
29+
3. You need the**mnemonic** used to create your original supernode key.
2930

3031
### Migration Steps
3132

3233
1. **Derive your new EVM key** from the same mnemonic:
34+
3335
```bash
3436
supernode keys recover --name evm-key --mnemonic "your twelve or twenty four words ..."
3537
```
38+
3639
This creates an `eth_secp256k1` key under the name `evm-key` using HD path
3740
`m/44'/60'/0'/0/0`. The resulting address will be different from your legacy
3841
address — this is expected.
39-
4042
2. **Add `evm_key_name` to your config.yaml** under the `supernode` section:
43+
4144
```yaml
4245
supernode:
4346
key_name: mykey # your existing legacy key name
4447
evm_key_name: evm-key # the name you used in step 1
4548
identity: lumera1... # your current legacy address
4649
```
47-
4850
3. **Restart the supernode**. On startup it will:
49-
- Detect the legacy `secp256k1` key under `key_name`
50-
- Validate that `evm_key_name` points to a valid `eth_secp256k1` key
51+
52+
- Detect the legacy`secp256k1` key under`key_name`
53+
- Validate that`evm_key_name` points to a valid`eth_secp256k1` key
5154
- Query the chain for an existing migration record (handles reruns)
52-
- Run a pre-flight `MigrationEstimate` to check if the migration would succeed
55+
- Run a pre-flight`MigrationEstimate` to check if the migration would succeed
5356
- Sign the migration payload with both keys
54-
- Broadcast `MsgClaimLegacyAccount` (or `MsgMigrateValidator` for validators)
57+
- Broadcast`MsgClaimLegacyAccount` (or`MsgMigrateValidator` for validators)
5558
- Wait for block confirmation (DeliverTx)
5659
- Delete the legacy key from the keyring
57-
- Update `config.yaml`: `key_name` -> `evm-key`, `identity` -> new address,
58-
`evm_key_name` cleared
59-
60+
- Update`config.yaml`:`key_name` ->`evm-key`,`identity` -> new address,`evm_key_name` cleared
6061
4. **After migration completes**, your config will look like:
62+
6163
```yaml
6264
supernode:
6365
key_name: evm-key
6466
identity: lumera1<new-evm-address>
6567
```
68+
6669
The `evm_key_name` field is automatically removed. No further action is needed.
6770

6871
### Troubleshooting
6972

70-
| Error | Cause | Fix |
71-
| ----- | ----- | --- |
72-
| `no evm_key_name configured` | Legacy key detected but config missing `evm_key_name` | Add `evm_key_name` to config and restart |
73-
| `not an eth_secp256k1 key` | `evm_key_name` points to a secp256k1 key (wrong derivation) | Re-derive using `supernode keys recover` (uses coin type 60) |
74-
| `new address mismatch` | On-chain migration record has a different destination address than your local EVM key | Your `evm_key_name` doesn't match the key originally used for migration; fix the config |
75-
| `migration estimate indicates migration would fail` | Chain rejected the pre-flight check | Check `rejection_reason` in logs; migration may be disabled or account may not exist |
76-
| `migration tx was not confirmed` | Tx was not included in a block within 60s | Restart to retry; the check is idempotent |
77-
| `failed to save updated config` | Config file write error after successful migration | Manually update config as instructed in the error message |
73+
| Error | Cause | Fix |
74+
| ----------------------------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
75+
| `no evm_key_name configured` | Legacy key detected but config missing `evm_key_name` | Add `evm_key_name` to config and restart |
76+
| `not an eth_secp256k1 key` | `evm_key_name` points to a secp256k1 key (wrong derivation) | Re-derive using `supernode keys recover` (uses coin type 60) |
77+
| `new address mismatch` | On-chain migration record has a different destination address than your local EVM key | Your `evm_key_name` doesn't match the key originally used for migration; fix the config |
78+
| `migration estimate indicates migration would fail` | Chain rejected the pre-flight check | Check `rejection_reason` in logs; migration may be disabled or account may not exist |
79+
| `migration tx was not confirmed` | Tx was not included in a block within 60s | Restart to retry; the check is idempotent |
80+
| `failed to save updated config` | Config file write error after successful migration | Manually update config as instructed in the error message |
7881

7982
## Chain-Side Reference
8083

@@ -106,12 +109,12 @@ validator operators with `ErrUseValidatorMigration`.
106109

107110
### Migration parameters (chain-side)
108111

109-
| Param | Default | Description |
110-
| ----- | ------- | ----------- |
111-
| `enable_migration` | `true` | Master switch |
112-
| `migration_end_time` | `0` | Unix timestamp deadline (0 = no deadline) |
113-
| `max_migrations_per_block` | `50` | Rate limit |
114-
| `max_validator_delegations` | `2000` | Max delegators for validator migration |
112+
| Param | Default | Description |
113+
| ----------------------------- | -------- | ----------------------------------------- |
114+
| `enable_migration` | `true` | Master switch |
115+
| `migration_end_time` | `0` | Unix timestamp deadline (0 = no deadline) |
116+
| `max_migrations_per_block` | `50` | Rate limit |
117+
| `max_validator_delegations` | `2000` | Max delegators for validator migration |
115118

116119
### Fee waiving
117120

@@ -175,29 +178,60 @@ Update config (key_name, identity, evm_key_name) and save
175178
The chain expects different signing protocols for each key type:
176179
177180
**Legacy (secp256k1):**
181+
178182
```
179183
supernode: hash = SHA256(payload)
180184
sig = kr.Sign(hash) -- internally: Sign(SHA256(hash))
181185
chain: VerifySignature(hash, sig) -- internally: verify(SHA256(hash), sig)
182186
```
183187
184188
**EVM (eth_secp256k1):**
189+
185190
```
186191
supernode: sig = kr.Sign(payload) -- internally: Sign(Keccak256(payload))
187192
chain: VerifySignature(payload, sig) -- internally: verify(Keccak256(payload), sig)
188193
```
189194
190-
### Key Architectural Decisions
195+
### P2P Bootstrap Refresh After Migration
191196
192-
- **No `kr.Rename`**: The Cosmos SDK `Rename` method uses amino armor export/import
193-
internally, which doesn't support `eth_secp256k1` keys. Instead, after migration
194-
the EVM key keeps its original name and `config.yaml key_name` is updated to point
195-
to it.
197+
When an EVM migration occurs, all supernodes change their on-chain addresses
198+
simultaneously. Without intervention, the P2P layer would keep stale addresses
199+
in its routing table for up to 10 minutes (the normal bootstrap refresh
200+
interval), causing handshake failures.
196201
197-
- **SYNC broadcast + polling**: `BROADCAST_MODE_SYNC` only waits for `CheckTx`.
198-
Local state (keyring, config) is only mutated after `waitForTxConfirmation` polls
199-
`GetTx` and confirms block inclusion via `DeliverTx`.
202+
Three mechanisms work together to resolve this:
203+
204+
1. **Immediate bootstrap refresh**: After migration completes and the Lumera
205+
client is reloaded, `start.go` calls `p2pService.NotifyEVMMigration()`.
206+
This triggers an immediate `SyncBootstrapOnce()` which re-queries
207+
`ListSuperNodes()` from the chain and updates the routing table with
208+
current addresses.
209+
210+
2. **Accelerated refresh window**: After the migration signal, the bootstrap
211+
refresher switches from the normal 10-minute interval to a **1-minute
212+
interval for 5 cycles**. This catches peers that migrate slightly later
213+
(staggered startup, network delays). After 5 accelerated cycles it
214+
automatically reverts to the normal 10-minute cadence.
215+
216+
3. **Staggered startup (devnet)**: In the devnet startup script, validator N
217+
waits `(N-1) * 5` seconds before starting the supernode when EVM migration
218+
is pending. This spreads migrations across ~20 seconds so that by the time
219+
later validators query the chain, earlier validators have already committed
220+
their migration records.
200221
222+
**Code locations:**
223+
224+
| Component | File | Key symbol |
225+
| --- | --- | --- |
226+
| Migration notify channel | `p2p/kademlia/dht.go` | `migrationNotify` field |
227+
| Accelerated refresher | `p2p/kademlia/bootstrap.go` | `StartBootstrapRefresher()`, `NotifyEVMMigration()` |
228+
| P2P interface method | `p2p/p2p.go` | `NotifyEVMMigration()` |
229+
| Startup integration | `supernode/cmd/start.go` | `evmMigrationOccurred` flag |
230+
231+
### Key Architectural Decisions
232+
233+
- **SYNC broadcast + polling**: `BROADCAST_MODE_SYNC` only waits for `CheckTx`.
234+
- Local state (keyring, config) is only mutated after `waitForTxConfirmation` polls `GetTx` and confirms block inclusion via `DeliverTx`.
201235
- **`migrationChainClient` interface**: Chain interactions (queries + broadcast) are
202236
abstracted behind an interface, enabling comprehensive unit testing without a live
203237
gRPC connection.
@@ -206,26 +240,26 @@ chain: VerifySignature(payload, sig) -- internally: verify(Keccak256(payloa
206240
207241
### New Files
208242
209-
| File | Description |
210-
| ---- | ----------- |
211-
| `supernode/cmd/evmigration.go` | Core migration logic: chain detection, key validation, dual signing, broadcast, config update |
212-
| `supernode/cmd/evmigration_test.go` | Unit tests with mock chain client |
213-
| `tests/integration/evmigration/evmigration_test.go` | Integration tests for keyring lifecycle, signing protocol, config persistence |
243+
| File | Description |
244+
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------- |
245+
| `supernode/cmd/evmigration.go` | Core migration logic: chain detection, key validation, dual signing, broadcast, config update |
246+
| `supernode/cmd/evmigration_test.go` | Unit tests with mock chain client |
247+
| `tests/integration/evmigration/evmigration_test.go` | Integration tests for keyring lifecycle, signing protocol, config persistence |
214248
215249
### Modified Files
216250
217-
| File | Changes |
218-
| ---- | ------- |
219-
| `pkg/keyring/keyring.go` | Switched to EVM defaults: `DefaultHDPath = "m/44'/60'/0'/0/0"`, `EthSecp256k1Option()`, `evmcryptocodec.RegisterInterfaces()` |
220-
| `pkg/keyring/keyring_test.go` | Added 10 tests for EVM key creation, derivation, signing, legacy-vs-EVM address differences |
221-
| `pkg/lumera/codec/encoding.go` | Registers `evmcryptocodec` and `evmigrationtypes` interfaces |
222-
| `pkg/lumera/interface.go` | Added `Conn() *grpc.ClientConn` to Client interface |
223-
| `pkg/lumera/client.go` | Implemented `Conn()` method |
224-
| `pkg/lumera/lumera_mock.go` | Added `Conn()` to mock client |
225-
| `pkg/testutil/lumera.go` | Added `Conn()` to `MockLumeraClient` |
226-
| `supernode/config/config.go` | Added `EVMKeyName string` field to `SupernodeConfig` |
227-
| `supernode/cmd/start.go` | Integrated `requireEVMChain()` and `ensureLegacyAccountMigrated()` at startup |
228-
| `go.mod` / `go.sum` | Added `github.com/cosmos/evm` dependency |
251+
| File | Changes |
252+
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
253+
| `pkg/keyring/keyring.go` | Switched to EVM defaults:`DefaultHDPath = "m/44'/60'/0'/0/0"`, `EthSecp256k1Option()`, `evmcryptocodec.RegisterInterfaces()` |
254+
| `pkg/keyring/keyring_test.go` | Added 10 tests for EVM key creation, derivation, signing, legacy-vs-EVM address differences |
255+
| `pkg/lumera/codec/encoding.go` | Registers `evmcryptocodec` and `evmigrationtypes` interfaces |
256+
| `pkg/lumera/interface.go` | Added `Conn() *grpc.ClientConn` to Client interface |
257+
| `pkg/lumera/client.go` | Implemented `Conn()` method |
258+
| `pkg/lumera/lumera_mock.go` | Added `Conn()` to mock client |
259+
| `pkg/testutil/lumera.go` | Added `Conn()` to `MockLumeraClient` |
260+
| `supernode/config/config.go` | Added `EVMKeyName string` field to `SupernodeConfig` |
261+
| `supernode/cmd/start.go` | Integrated `requireEVMChain()` and `ensureLegacyAccountMigrated()` at startup |
262+
| `go.mod` / `go.sum` | Added `github.com/cosmos/evm` dependency |
229263
230264
### Key Types and Functions
231265

p2p/kademlia/bootstrap.go

Lines changed: 62 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,10 @@ import (
1616
)
1717

1818
const (
19-
bootstrapRefreshInterval = 10 * time.Minute
20-
defaultSuperNodeP2PPort int = 4445
19+
bootstrapRefreshInterval = 10 * time.Minute
20+
bootstrapAcceleratedInterval = 1 * time.Minute
21+
bootstrapAcceleratedCycles = 5
22+
defaultSuperNodeP2PPort int = 4445
2123
)
2224

2325
// seed a couple of obviously bad addrs (unless in integration tests)
@@ -301,6 +303,10 @@ func (s *DHT) SyncBootstrapOnce(ctx context.Context, bootstrapNodes string) erro
301303

302304
// StartBootstrapRefresher runs SyncBootstrapOnce every 10 minutes (idempotent upserts).
303305
// This keeps replication_info and routing table current as the validator set changes.
306+
//
307+
// When NotifyEVMMigration is called, the refresher immediately runs a sync and
308+
// temporarily switches to an accelerated 1-minute interval for 5 cycles so that
309+
// peer address changes from EVM migration are picked up quickly.
304310
func (s *DHT) StartBootstrapRefresher(ctx context.Context, bootstrapNodes string) {
305311
go func() {
306312
// Initial sync
@@ -310,25 +316,78 @@ func (s *DHT) StartBootstrapRefresher(ctx context.Context, bootstrapNodes string
310316
logtrace.FieldError: err.Error(),
311317
})
312318
}
313-
t := time.NewTicker(bootstrapRefreshInterval)
319+
320+
acceleratedRemaining := 0
321+
currentInterval := bootstrapRefreshInterval
322+
t := time.NewTicker(currentInterval)
314323
defer t.Stop()
315324

316325
for {
317326
select {
318327
case <-ctx.Done():
319328
return
329+
case <-s.migrationNotify:
330+
// EVM migration detected — immediate sync + accelerated refresh
331+
logtrace.Info(ctx, "EVM migration notified — running immediate bootstrap sync and switching to accelerated refresh", logtrace.Fields{
332+
logtrace.FieldModule: "p2p",
333+
"accelerated_cycles": bootstrapAcceleratedCycles,
334+
"accelerated_interval": bootstrapAcceleratedInterval.String(),
335+
})
336+
if err := s.SyncBootstrapOnce(ctx, bootstrapNodes); err != nil {
337+
logtrace.Warn(ctx, "post-migration bootstrap sync failed", logtrace.Fields{
338+
logtrace.FieldModule: "p2p",
339+
logtrace.FieldError: err.Error(),
340+
})
341+
}
342+
acceleratedRemaining = bootstrapAcceleratedCycles
343+
currentInterval = bootstrapAcceleratedInterval
344+
t.Reset(currentInterval)
320345
case <-t.C:
321346
if err := s.SyncBootstrapOnce(ctx, bootstrapNodes); err != nil {
322347
logtrace.Warn(ctx, "periodic bootstrap sync failed", logtrace.Fields{
323348
logtrace.FieldModule: "p2p",
324349
logtrace.FieldError: err.Error(),
325350
})
326351
}
352+
if acceleratedRemaining > 0 {
353+
acceleratedRemaining--
354+
if acceleratedRemaining == 0 {
355+
logtrace.Info(ctx, "Accelerated bootstrap refresh complete — reverting to normal interval", logtrace.Fields{
356+
logtrace.FieldModule: "p2p",
357+
})
358+
currentInterval = bootstrapRefreshInterval
359+
t.Reset(currentInterval)
360+
}
361+
}
327362
}
328363
}
329364
}()
330365
}
331366

367+
// NotifyEVMMigration flushes stale P2P state (credential cache and pooled
368+
// connections) and signals the bootstrap refresher to immediately re-sync the
369+
// peer list from the chain with temporarily accelerated refresh interval.
370+
// This must be called after an EVM account migration completes so that the
371+
// new local identity is used for all subsequent handshakes.
372+
func (s *DHT) NotifyEVMMigration() {
373+
// 1. Clear the global KeyExchanger cache so new handshakes use the
374+
// post-migration local identity in HKDF key derivation.
375+
ltc.ClearKeyExchangerCache()
376+
377+
// 2. Close all pooled connections — they were established with the old
378+
// identity and will fail authentication if reused.
379+
if s.network != nil {
380+
s.network.connPool.Release()
381+
}
382+
383+
// 3. Signal the bootstrap refresher to re-sync peers immediately.
384+
select {
385+
case s.migrationNotify <- struct{}{}:
386+
default:
387+
// already pending — no need to double-signal
388+
}
389+
}
390+
332391
// ConfigureBootstrapNodes wires to the new sync/refresher (no pings here).
333392
func (s *DHT) ConfigureBootstrapNodes(ctx context.Context, bootstrapNodes string) error {
334393
// One-time sync; start refresher in the background

p2p/kademlia/dht.go

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,11 @@ type DHT struct {
7777
routingAllow map[[32]byte]struct{} // blake3(peerID) -> exists
7878
routingAllowReady atomic.Bool
7979
routingAllowCount atomic.Int64
80+
81+
// migrationNotify is signalled by NotifyEVMMigration to trigger an
82+
// immediate bootstrap refresh and temporarily accelerate the refresh
83+
// interval (1 min for the first 5 cycles after migration).
84+
migrationNotify chan struct{}
8085
}
8186

8287
// bootstrapIgnoreList seeds the in-memory ignore list with nodes that are
@@ -280,15 +285,16 @@ func NewDHT(ctx context.Context, store Store, metaStore MetaStore, options *Opti
280285
}
281286

282287
s := &DHT{
283-
metaStore: metaStore,
284-
store: store,
285-
options: options,
286-
done: make(chan struct{}),
287-
cache: memory.NewKeyValue(),
288-
bsConnected: &sync.Map{},
289-
ignorelist: NewBanList(ctx),
290-
replicationMtx: sync.RWMutex{},
291-
rqstore: rqstore,
288+
metaStore: metaStore,
289+
store: store,
290+
options: options,
291+
done: make(chan struct{}),
292+
cache: memory.NewKeyValue(),
293+
bsConnected: &sync.Map{},
294+
ignorelist: NewBanList(ctx),
295+
replicationMtx: sync.RWMutex{},
296+
rqstore: rqstore,
297+
migrationNotify: make(chan struct{}, 1),
292298
}
293299

294300
// Check that keyring is provided

0 commit comments

Comments
 (0)