Problem
The in-app shielded-balance sync (deckard-core::shielded::ShieldedHandle, landed in #11) does a full cold sync on every launch because the Railgun provider uses kohaku's in-memory MemoryDatabase. Measured on mainnet:
| RPC |
Cold sync wall-clock |
Notes |
publicnode (free, the default) |
stalled >14 min, never completed |
rate-limited at Railgun getLogs/Subsquid volume |
| Alchemy archive |
647 s (≈10.8 min) |
completed; CPU 1.8–6.6% the whole time |
The app degrades honestly throughout (shows "Public · private balance still syncing", never a fake balance), so this is a UX/perf issue, not a correctness one.
Root cause (measured, not guessed)
A probe of the two syncer stages:
chain head: 25,280,659
subsquid head: 25,280,573 → RPC gap is only 86 blocks (1 getLogs batch)
full history span: 10,587,646 blocks
So the RpcSyncer gap-fill is trivial (~1 s). The entire ~11 min is the SubsquidSyncer paging the full mainnet Railgun history (10.5M blocks → millions of commitments + nullifiers). CPU stays low → it's latency-bound (hundreds of sequential GraphQL page round-trips), not note-decryption-bound.
Therefore "Subsquid-only" and RpcSyncer config tweaks (batch_size/batch_delay) save ~1 s and do NOT help. The Subsquid page size (20000) isn't exposed for tuning. Downloading the full UTXO tree is fundamental to Railgun's model — you can't know your balance without it.
The fix: persist the Railgun DB
kohaku supports it cleanly: RailgunBuilder::with_database(Arc<dyn Database>). The Database trait is a 3-method KV store (get/set/delete); kohaku even ships a ~40-line FilesystemDatabase (currently commented out of its mod.rs). So:
- Implement a small disk-backed
Database in deckard-core (copy the fs.rs pattern) and pass it via with_database. Turns ~11 min/launch into ~11 min once, then incremental (seconds).
- Security requirement: the DB stores synced UTXO data = the user's private notes/balances. kohaku keeps key material out of it, but for a privacy wallet, writing the private balance to a plaintext file is a privacy-at-rest regression. Encrypt values at rest (reuse the keystore's XChaCha20-Poly1305 envelope pattern, keyed off the session). This makes it a small feature, not a 40-line drop-in.
Workaround for demos now
- Use a fast archive RPC (Alchemy/Infura) via Settings → Custom RPC; the free
publicnode default stalls.
- Pre-sync once (~11 min) and keep the app open (in-memory DB is lost on close). When "private balance still syncing" resolves to the Private/Public split, the demo is ready.
Related deferred follow-ups (from the #11 codex review)
- Sync timeout/backoff in
shielded.rs — needs a tokio time feature; would make a bad-RPC stall fail honestly instead of "syncing forever" (directly relevant: publicnode stalls indefinitely today).
- Zeroize the viewing key end-to-end (it's a
String on the wire + in ViewOnlySigner).
- Commit an SDK-generated fixture for the derivation KAT's provenance.
Evidence
Temporary [MEASURE] eprintln instrumentation in crates/deckard-core/src/shielded.rs (uncommitted) produced the timings above.
Problem
The in-app shielded-balance sync (
deckard-core::shielded::ShieldedHandle, landed in #11) does a full cold sync on every launch because the Railgun provider uses kohaku's in-memoryMemoryDatabase. Measured on mainnet:publicnode(free, the default)getLogs/Subsquid volumeThe app degrades honestly throughout (shows "Public · private balance still syncing", never a fake balance), so this is a UX/perf issue, not a correctness one.
Root cause (measured, not guessed)
A probe of the two syncer stages:
So the
RpcSyncergap-fill is trivial (~1 s). The entire ~11 min is theSubsquidSyncerpaging the full mainnet Railgun history (10.5M blocks → millions of commitments + nullifiers). CPU stays low → it's latency-bound (hundreds of sequential GraphQL page round-trips), not note-decryption-bound.Therefore "Subsquid-only" and RpcSyncer config tweaks (
batch_size/batch_delay) save ~1 s and do NOT help. The Subsquid page size (20000) isn't exposed for tuning. Downloading the full UTXO tree is fundamental to Railgun's model — you can't know your balance without it.The fix: persist the Railgun DB
kohaku supports it cleanly:
RailgunBuilder::with_database(Arc<dyn Database>). TheDatabasetrait is a 3-method KV store (get/set/delete); kohaku even ships a ~40-lineFilesystemDatabase(currently commented out of itsmod.rs). So:Databaseindeckard-core(copy thefs.rspattern) and pass it viawith_database. Turns ~11 min/launch into ~11 min once, then incremental (seconds).Workaround for demos now
publicnodedefault stalls.Related deferred follow-ups (from the #11 codex review)
shielded.rs— needs a tokiotimefeature; would make a bad-RPC stall fail honestly instead of "syncing forever" (directly relevant:publicnodestalls indefinitely today).Stringon the wire + inViewOnlySigner).Evidence
Temporary
[MEASURE]eprintln instrumentation incrates/deckard-core/src/shielded.rs(uncommitted) produced the timings above.