feat: ship napi-rs consumable archives for wasm32-wasip1#225
Conversation
The emnapi-basic-mt target requires wasm atomics and cannot be compiled for the non-threaded wasm32-wasip1 target. (cherry picked from commit 24f5852)
|
|
||
| release-package: | ||
| timeout-minutes: 15 | ||
| name: CI release package (wasm32-wasip1 archives) | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: ./.github/actions/setup-target-ci | ||
| with: | ||
| target: 'release-package' | ||
| # keep in sync with MAIN_EM_VERSION in main.yml so the release | ||
| # rehearsal uses the same emscripten as the production release lane | ||
| em-version: '4.0.23' | ||
| needs-emscripten: 'true' | ||
| needs-wasi-sdk: 'true' | ||
| needs-v8-headers: 'false' | ||
| - name: NPM Build | ||
| run: npm run build --workspaces --if-present | ||
| - name: Release | ||
| run: node ./script/release.js | ||
| - name: Validate release packages | ||
| run: node ./script/test-release-package.js --prepared |
There was a problem hiding this comment.
These aren't needed. The release is triggered manually via the main target in workflows/main.yml.
There was a problem hiding this comment.
Removed the job — agreed that CI doesn't need to rehearse the release. I kept script/test-release-package.js itself, since it's what catches the archive layout the published 2.0.0-alpha.2 shipped with (env-bound cleanup hooks, missing plain wasip1 archives). If you'd like, it could run in the manual release lane instead — a single node ./script/test-release-package.js --prepared between release.js and npm publish in main.yml — so a broken archive set can't be published. Happy to drop the script entirely if you'd rather not carry it.
| # napi-rs links the emnapi archive against plain `extern "C"` declarations, so | ||
| # every `napi_*` reference must resolve through the `env` wasm import module, | ||
| # except `napi_add_env_cleanup_hook` / `napi_remove_env_cleanup_hook` which | ||
| # napi-rs imports from the `napi` module (EMNAPI_NAPI_RS_IMPORTS, see | ||
| # src/async_cleanup_hook.c). async_work.c and threadsafe_function.c must stay | ||
| # out of the archive so that async work and thread-safe functions remain wasm | ||
| # imports resolved by the JavaScript implementations from @emnapi/core plugins. | ||
| if(EMNAPI_BUILD_FOR_NAPI_RS) | ||
| add_library(${EMNAPI_NAPI_RS_MT_TARGET_NAME} STATIC ${EMNAPI_SRC} ${UV_SRC}) | ||
| target_include_directories(${EMNAPI_NAPI_RS_MT_TARGET_NAME} PUBLIC ${EMNAPI_INCLUDE}) | ||
| target_compile_definitions(${EMNAPI_NAPI_RS_MT_TARGET_NAME} | ||
| PUBLIC ${EMNAPI_DEFINES} "NAPI_EXTERN=__attribute__((__import_module__(\"env\")))" | ||
| ) | ||
|
|
||
| if(EMNAPI_BUILD_BASIC_LIBS) | ||
| add_library(${EMNAPI_BASIC_NAPI_RS_MT_TARGET_NAME} STATIC | ||
| ${ENAPI_BASIC_SRC} | ||
| ${UV_SRC} | ||
| set(EMNAPI_NAPI_RS_SRC ${ENAPI_BASIC_SRC} ${UV_SRC}) | ||
| if(IS_WASM32_WASIP1_THREADS) | ||
| set(EMNAPI_NAPI_RS_TARGET ${EMNAPI_NAPI_RS_MT_TARGET_NAME}) | ||
| list(APPEND EMNAPI_NAPI_RS_SRC | ||
| "${CMAKE_CURRENT_SOURCE_DIR}/src/thread/async_worker_create.c" | ||
| "${CMAKE_CURRENT_SOURCE_DIR}/src/thread/async_worker_init.S" | ||
| ) | ||
| target_include_directories(${EMNAPI_BASIC_NAPI_RS_MT_TARGET_NAME} PUBLIC ${EMNAPI_INCLUDE}) | ||
| target_compile_definitions(${EMNAPI_BASIC_NAPI_RS_MT_TARGET_NAME} | ||
| PUBLIC ${EMNAPI_DEFINES} "NAPI_EXTERN=__attribute__((__import_module__(\"env\")))" | ||
| ) | ||
| else() | ||
| set(EMNAPI_NAPI_RS_TARGET ${EMNAPI_NAPI_RS_TARGET_NAME}) | ||
| endif() | ||
|
|
||
| add_library(${EMNAPI_NAPI_RS_TARGET} STATIC ${EMNAPI_NAPI_RS_SRC}) | ||
| target_include_directories(${EMNAPI_NAPI_RS_TARGET} PUBLIC ${EMNAPI_INCLUDE}) | ||
| target_compile_definitions(${EMNAPI_NAPI_RS_TARGET} | ||
| PUBLIC ${EMNAPI_DEFINES} "NAPI_EXTERN=__attribute__((__import_module__(\"env\")))" | ||
| PRIVATE "EMNAPI_NAPI_RS_IMPORTS" | ||
| ) |
There was a problem hiding this comment.
Regarding the static library names:
Those with "basic" in the name do not include the C-based async work and tsfn translation units; those without "basic" include the full C implementation.
Those with "mt" in the name are linked with -matomics and -mbulk-memory flags; those without "mt" are not.
- For the
wasm32-wasip1-threads(multi-threaded) target, napi-rs links against:
- The C implementation for async work and tsfn (ensuring consistency with Emscripten).
libemnapi-napi-rs-mt.a.
- For the
wasm32-wasip1(single-threaded) target, napi-rs links against:
- The JS implementation for async work and tsfn.
libemnapi-basic-napi-rs.a.
Alternatively, if you find the naming too confusing, we could simply standardize on libemnapi-napi-rs.a and place it in both the wasm32-wasip1 and wasm32-wasip1-threads directories.
There was a problem hiding this comment.
Done — reworked to exactly this wiring (and renamed per your naming scheme):
wasm32-wasip1-threads:libemnapi-napi-rs-mt.ais back to the full C composition (same sources aslibemnapi-mt), so async work / TSFN share the one C implementation with Emscripten.wasm32-wasip1: the plain target is nowlibemnapi-basic-napi-rs.a(basic composition, JS implementations).
Before reworking we validated the full-C mt archive against napi-rs's threaded WASI lane: link clean, spec suite green (incl. TSFN throw-from-callback and Promise tests), and the artifact's import list confirms the C implementation is active (no napi_create_threadsafe_function/napi_*_async_work imports; the napi module contains exactly the two cleanup hooks).
One required amendment found while validating: the EMNAPI_NAPI_RS_IMPORTS redeclaration has to live in src/emnapi_internal.h rather than async_cleanup_hook.c. Under the full composition threadsafe_function.c also references napi_add/remove_env_cleanup_hook, so with the guard in only one TU the two object files bind the hooks to different import modules and wasm-ld rejects the link (import module mismatch). The published 2.0.0-alpha.2 fails a probe link with 4 such errors (both TUs × both hooks) — with the header placement both TUs agree and the link is clean.
Build a napi-rs flavored static archive for both wasip1 targets:
- wasm32-wasip1-threads keeps emnapi-napi-rs-mt on the FULL source
list (${EMNAPI_SRC} ${UV_SRC}): the same C async work and
thread-safe function implementation that the emscripten emnapi-mt
archive uses, so both toolchains share one implementation.
- plain wasm32-wasip1 gains emnapi-basic-napi-rs (there was no napi-rs
archive for the non-threaded target at all): basic composition, with
async work and thread-safe functions left as wasm imports resolved
by the JavaScript implementations from @emnapi/core plugins.
- emnapi-basic-napi-rs-mt is removed: it was never shipped in the npm
package and nothing links it.
The archives keep resolving plain napi_* references through the env
wasm import module (NAPI_EXTERN), but napi_add_env_cleanup_hook /
napi_remove_env_cleanup_hook are re-declared under the napi import
module (EMNAPI_NAPI_RS_IMPORTS): napi-rs imports these two hooks from
the napi module, and an archive that imports them from env fails the
napi-rs link with a wasm-ld import module mismatch — which is exactly
how the published 2.0.0-alpha.2 archive breaks. The re-declaration
lives in src/emnapi_internal.h rather than in async_cleanup_hook.c
because under the full composition threadsafe_function.c also calls
the two hooks: with the guard in only one translation unit the two
object files disagree on the hooks' import module and wasm-ld rejects
the link.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T34fMrYCf2V13Mg8BKmnda
Add script/test-release-package.js, a release gate that validates the
packed emnapi npm package (run node ./script/release.js first):
1. tarball-content-allowlist: the tarball ships exactly
lib/wasm32-wasip1/{libemnapi.a,libemnapi-basic-napi-rs.a} and
lib/wasm32-wasip1-threads/{libemnapi.a,libemnapi-mt.a,
libemnapi-napi-rs-mt.a} and no other lib/wasm32-wasip1* entries
2. symbol-profile (llvm-nm): both napi-rs archives define
emnapi_create_env/emnapi_delete_env and the async cleanup hook
implementations (napi_add/remove_async_cleanup_hook); the -mt
archive must DEFINE napi_create_async_work /
napi_create_threadsafe_function / napi_call_threadsafe_function
(full composition — the C implementation shared with emscripten's
emnapi-mt) while the plain archive must NOT define them (they come
from @emnapi/core plugins) and stays free of pthread/TLS symbols
3. import-module: link a napi-rs-like probe against each archive and
assert the env cleanup hooks are imported from wasm module "napi"
on both archives; on the -mt archive the async work / TSFN symbols
are NOT imported at all (they resolve in-archive, whose C code in
turn imports the JS-implemented parts of Node-API from "env"); on
the plain archive they are imported from "env" and no OTHER
Node-API import exists (--import-undefined turns any missing
archive member into a silent env import, so an unexpected
napi_*/node_api_* import means the archive failed to provide a
definition). This is the check the published 2.0.0-alpha.2 fails:
its archive binds the cleanup hooks to env and wasm-ld rejects the
napi-rs link with an import module mismatch, and it ships no plain
wasip1 archives at all
4. version-equality: emnapi / @emnapi/runtime / @emnapi/core share
one version (published together by the Release workflow and bumped
as one unit by the root bump script); independently released
packages are not asserted against
5. core-plugins-subpath: the packed @emnapi/core resolves
./plugins from a scratch consumer and exposes the asyncWork/tsfn
plugin factories
--prepared validates the existing release output; --tarball <file>
validates a pre-built tarball, which is how the published
2.0.0-alpha.2 regression was captured.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T34fMrYCf2V13Mg8BKmnda
7af0a5a to
977599e
Compare
2 participants
This is the packaging half of the WASI work (companion to #220, which is the runtime memory-view correctness fix). It ships the
wasm32-wasip1archives that napi-rs and rolldown need, and fixes the napi-rs archive composition so the published artifacts are actually consumable. The two were tangled together in the original 60k-line branch; splitting them lets each be reviewed on its own terms.What's here
wasm32-wasip1archives. v2 published nolib/wasm32-wasip1archives at all. This rebuilds and ships them (libemnapi.a,libemnapi-basic-napi-rs.a), building viashare/cmake/wasi-sdk-p1.cmakedirectly, with a pre-copy sysroot assert + post-copy destination assert so a dirty rerun can't mask a missing archive.emnapi-napi-rs-mtkeeps the full C composition (same sources aslibemnapi-mt— async work and TSFN share one C implementation with Emscripten);napiwasm module, so they are rebound via anEMNAPI_NAPI_RS_IMPORTSguard (NAPI_VERSION >= 8) placed insrc/emnapi_internal.h— under the full composition bothasync_cleanup_hook.candthreadsafe_function.creference the hooks, so the guard must apply to both TUs or wasm-ld rejects the link with an import-module mismatch (this is exactly how the published2.0.0-alpha.2fails: a probe link errors 4× — both TUs × both hooks);emnapi-basic-napi-rstarget (basic composition — JS async work/TSFN) for plain wasip1;emnapi-basic-napi-rs-mtis removed — v1.11.2 published that filename, but it was never consumable in v2, and napi-rs doesn't link it under this wiring.napi_create_threadsafe_function/napi_create_async_work(C impl in-archive) and import only the two cleanup hooks fromnapi; the plain archive must not define them (JS impls); negative controls include the published2.0.0-alpha.2layout and member-deletion of both targets.Shipped sets:
wasip1 = {libemnapi, libemnapi-basic-napi-rs},threads = {libemnapi, libemnapi-mt, libemnapi-napi-rs-mt}.Consumer validation
napi-rs and rolldown were both built against packed tarballs of this branch merged with #220, with all their emnapi patches and vendored archives deleted — full WASI test lanes green on both. After the recomposition to the full-C mt archive, the napi-rs threaded lane was re-validated against the new composition (link clean, spec suite green, with the C TSFN verified active via the artifact's import list).