improve fork performance by avoiding copying unnescessary grate poll stack - #1385
Open
qianxichen233 wants to merge 1 commit into
Open
qianxichen233 wants to merge 1 commit into
qianxichen233 wants to merge 1 commit into
Conversation
lind eagerly built a full 32-worker grate stack pool (32 x 8MB stack slots, ~256MB) for every static-build cage, both at initial exec time and again on every single fork -- regardless of whether the program ever registers a grate handler. Since fork_vmmap must fork-copy every accessible byte, this made fork enormously expensive for ordinary (non-grate) programs: profiling bash showed 90%+ of fork's wall time inside process_vm_writev copying that unused arena, dominated by the kernel zero-filling pages that were about to be immediately copied over (mostly with more zeros). Three coordinated changes: - instance.rs: only mprotect(RW) a static cage's own required memory at boot, not the full stack_arena_base + grate arena. The arena stays PROT_NONE by default. - lind-3i's create_worker: activate each grate worker's own 8MB stack slot lazily, when that worker is actually created, instead of relying on a blanket upfront reservation. - execute.rs / lind-multi-process's fork_call: stop unconditionally building a worker pool for every static cage at exec and fork time. Gate it on the module actually exporting the grate entry trampoline (pass_fptr_to_wt, i.e. --compile-grate builds), with a new per-cage registration flag in lind_platform_const propagated across fork so a genuine grate-using cage's children still get a pool rebuilt. Measured on bash (lind-wasm-apps): fork ~108x faster (65.6ms -> 0.61ms/fork), fork+exec ~41-61x faster, pipe (2 forks/op) ~129x faster. A dedicated fork microbenchmark (simple fork, fork+exec, nested fork chains, fork after dirtying varying heap sizes) confirms the same pattern and shows post-fix cost finally scales with actual dirtied memory instead of being swamped by the fixed 256MB copy. Verified: all 15/15 grate-tests pass, and a direct sweep of process_tests/memory_tests/signal_tests (bypassing an unrelated, pre-existing wasmtestreport harness bug) shows no exit-code regressions vs. unmodified main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
End-to-End Test ReportTest Previewgrate harnessGrate Test Report
Cases
static harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
wasm-math harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
wasm-filesystem harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
wasm-memory harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
Test Results by Category
wasm-process harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
wasm-signals harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
Test Results by Category
wasm-networking harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
wasm-dynamic-linking harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
Test Results by Category
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Yaxuan-w
requested changes
Sep 18, 2026
|
|
||
| // The slot starts PROT_NONE (see new_started_impl_with_lind); activate | ||
| // just this worker's own slot now. Leave the guard page before it | ||
| // PROT_NONE so stack overflow still traps. |
Member
There was a problem hiding this comment.
so stack overflow still traps
The 4K guard in front of each slot never enters the vmmap but our vmmap walks gaps in ascending order starting from address 0. This comment seems doesn't hold bc a worker stack overflow now silently clobbers whatever the user just mmapped there.
|
|
||
| /// Tracks, per cage, whether a grate handler has ever been registered. | ||
| /// Lets `fork` skip rebuilding a worker pool for cages that never use grates. | ||
| pub static GRATE_HANDLER_REGISTERED: OnceLock<RwLock<Vec<bool>>> = OnceLock::new(); |
| // initialize the grate pool for later use in grate calls and | ||
| // other syscalls that require re-entry into wasmtime runtime. | ||
| init_grate_pool(); | ||
| unregister_grate_handler(cageid); |
Member
There was a problem hiding this comment.
We probably need to keep unregister_grate_handler(cageid) unconditional and clear the flag in the branch that does not register.
If a cage with a registered handler execs into a module that does not export pass_fptr_to_wt, the stale handler stays in GRATE_POOL. Its 32 worker Stores keep the old module and old linear memory alive, and a grate call targeting that cage id runs into the replaced image. GRATE_HANDLER_REGISTERED[cageid] is also never cleared, so later forks rebuild a pool for nothing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each stack in grate pool used to always assigned with RW permission at start even if they aren't used. This is a huge allocation. During fork, all the RW memories will be copied entirely including those unused grate stacks (8MB * grate_worker_number). This is a huge amount of unnescessary work.
This PR make grate worker's stack always assigned with PROT_NONE at start, so that fork will skip these slots as they are not used. Only when the grate worker is activated, the worker's stack is mmap-ed into RW and can be potentially copied at fork.
speed up fork 10-100 times faster depending on different scenarios