Add wasm32-unknown-emscripten target support#1969
Draft
guybedford wants to merge 1 commit into
Draft
Conversation
Adds `wasm32-unknown-emscripten` as a target for mio, plus a CI job that runs the suite under Node. Resolves tokio-rs#642. Emscripten exposes a real epoll backed by its runtime event loop, so the existing Linux epoll selector is reused rather than adding a new backend. The wasm `compile_error!` guard is relaxed to let emscripten through, and the `epoll`/`eventfd`/pipe-waker cfg lists gain emscripten. Because emscripten cannot block in `epoll_wait` without JSPI/ASYNCIFY, two non-blocking readiness paths are added alongside the normal `Poll::poll`: * `Poll::new_with_callback` arms a persistent callback on the epoll fd via the runtime's `emscripten_epoll_set_callback`, delivering ready events on a fresh host tick whenever the set makes progress. The callback state is boxed, owned by the arming selector, and disarmed on drop; it may freely re-enter mio to (de)register sources. * `Registry::poll_ready` does a zero-timeout drain of the same epoll set as a readiness probe where a blocking wait is impossible. AF_UNIX support is stream-only: emscripten's node-backed sockets have no datagram primitive, so `UnixDatagram` and the `socketpair`-based helpers are not compiled there. Sockets set `O_NONBLOCK` via `fcntl` since emscripten's `socket(2)` silently strips `SOCK_NONBLOCK`/`SOCK_CLOEXEC`. The suite spawns OS threads (socket-peer test harnesses), so std is rebuilt with atomics via -Zbuild-std and linked -pthread with -sPROXY_TO_PTHREAD so the main thread can block; JSPI (-sJSPI) provides the return-to-host suspension for blocking reads/writes, and NODERAWFS/NODERAWSOCKETS back the filesystem and sockets with node's. This needs nightly + rust-src and a JSPI-capable Node (26+, or 22 with --experimental-wasm-jspi). No custom target spec is required: nightly now emits the __main_argc_argv entry point (rust-lang/rust#158937). Doctests are skipped on this target: rustdoc does not apply the emcc link args, so the examples cannot be linked with the socket/thread runtime. Temporary, until the dependencies land upstream: * Cargo.toml patches libc to guybedford/libc#emscripten for the emscripten epoll/pthread externs (rust-lang/libc#5270). * the CI job builds against the guybedford/emscripten `cf` fork, which carries the epoll callback (emscripten-core/emscripten#27207), AF_UNIX pathname stream sockets, and multicast getsockopt patches this target depends on. Suite result: 148 passed, 0 failed, 3 ignored under Node - green on CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR for #642.
Adds
wasm32-unknown-emscriptenas a target for mio, with a CI job running the suite under Node. Opening as a draft for now since it depends on unreleased patchsets for Emscripten and Rust's libc.This lays the groundwork for Tokio support for Emscripten. Initially I did not plan to PR Mio, but in discussion with @Darksonn and @Noah-Kennedy it was suggested to use Mio if possible, and this has worked out well in my opinion, only requiring one additional Emscripten-specific API explained below.
Once emscripten-core/emscripten#27207 lands, Emscripten exposes a real epoll backed by its runtime event loop, so the existing Linux epoll selector can reused rather than adding a new backend.
The readiness model works under different modes - direct blocking
epoll_waitis fully supported under Emscripten's JSPI or pthread proxy modes, but when threading support is not available in browsers/node, an alternative new API may be used instead -emscripten_epoll_set_callbackto support a single thread cooperative event loop mode. Instead of callingepoll_wait, a thread can register a persistent callback on the epoll, which is delivered by the JS event loop back to that thread possibly as soon as the next host tick after the stack unwinds. This allows the epoll model to work exactly the same, but without blocking in a hosted single-threaded JS event loop. Events keep re-firing while the set stays ready, and level-triggering andmaxeventsoverflow drain exactly as a blockingepoll_waitloop would. It is just another consumer of the same ready list concept, supporting all ofEPOLLET/EPOLLONESHOT/EPOLLEXCLUSIVE/maxevents. ANULLcallback can be used to unregister.For Mio, to support non-JSPI / thread use cases with this new
emscripten_epoll_set_callbackresults in a new optional Emscripten-only API -Poll::new_with_callback(capacity, closure), which maps directly to this callback for the lifetime of thePoll. It boxes the user closure plus a persistentEventsbuffer; a smallextern "C-unwind"trampoline refills that buffer from the delivered ready set on each tick and invokes the closure. The callback holds no borrows across the call, so the closure may freely re-enter mio to register/deregister sources, and it is disarmed (then freed) on drop.Patch Sets
This PR will remain a draft while it relies on the following upstream patches:
In addition Rust nightly is needed for two additional patches only landed recently.
Test status
Tests run directly on the Emscripten Node.js build with
NODERAWSOCKETSandNODERAWFSto provide a transparent runner without further harness configuration being necessary.Tests use JSPI to be able to support the harness integration, but can work without JSPI when integrating with the callback model - a dedicated
emscriptentest is included for these.tcp_stream::raw_fd(pending net: support sync connect for BoundSocket nodejs/node#64375), plus twoPOLLRDHUP/close-event cases shared with other targets.Early feedback on the approach very much welcome.