feat: integrate numpy as statically-linked built-in module#7
Merged
Conversation
Add numpy (https://github.com/nanvix/numpy) as a git submodule and wire it into the setup/build/test pipeline following the same pattern used for cymem. Build changes: - Add deps/numpy submodule (branch nanvix/v1.26.4) - Build numpy static archives via Meson cross-compilation in Docker - Register _np_multiarray_umath as a CPython built-in in Setup.local - Install numpy Python package with generated version.py and __config__.py - Add __kernel_tanl libm compatibility stub for Nanvix newlib Runtime patches (patches/numpy_static_builtin.patch): - Skip CPUID/XGETBV instructions that hang on Nanvix microkernel - Rename built-in to _np_multiarray_umath to break circular import deadlock with CPython per-module import lock - Defer npy_cache_import calls during init (#ifndef __nanvix__) - Pre-register partially-initialised module in sys.modules Runtime bootstrap (patches/nanvix_numpy_bootstrap.py): - find_spec-based meta-path finder redirects numpy.core._multiarray_umath to the renamed _np_multiarray_umath built-in - Permissive stub modules for unlinked C extensions (linalg, fft, random) Test: - Add test_102_numpy.py functional test (array creation + sum) - All 102 tests pass (including flatbuffers which depends on numpy)
The NumPy Meson build requires Cython to transpile .pyx files, but it was not available in the CI container. Create a virtual environment during setup and activate it during build so that cython is on PATH.
- Use 'python3' from PATH instead of hardcoded /usr/bin/python3 so Meson runs with the venv Python that has Cython installed. - Add -Wl,--allow-multiple-definition to link args to resolve 'multiple definition of environ' between libposix.a and libc.a. - Apply the patch during build alongside the existing numpy patches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integrate nanvix/numpy into nanvix-python, following the same pattern used for cymem.
NumPy's core C extension (
_multiarray_umath) is statically linked into the CPython binary as a built-in module, enablingimport numpyand basic array operations on Nanvix.Changes
Build pipeline (
z)deps/numpysubmodule (branchnanvix/v1.26.4, shallow)_np_multiarray_umathas a CPython built-in inModules/Setup.localversion.pyand__config__.pystubs__kernel_tanllibm compatibility stub for Nanvix newlibC source patches (
patches/numpy_static_builtin.patch)CPUID/XGETBVinstructions that hang on the Nanvix microkernel_np_multiarray_umathto break circular import deadlock with CPython's per-module import locknpy_cache_importcalls during module init (#ifndef __nanvix__)sys.modulesto prevent re-entrantPyInitRuntime bootstrap (
patches/nanvix_numpy_bootstrap.py)find_spec-based meta-path finder redirectsnumpy.core._multiarray_umathto the renamed built-in_StubModule) for C extensions not statically linked (linalg, fft, random, tests)Test
test_102_numpy.py— validates array creation + sumflatbufferswhich depends on numpyPatch documentation
See
patches/numpy_static_builtin.mdfor a detailed explanation of each patch section.