[build] E: Build libcrypto.so + libssl.so alongside static archives#1
Closed
esaurez wants to merge 1 commit into
Closed
[build] E: Build libcrypto.so + libssl.so alongside static archives#1esaurez wants to merge 1 commit into
esaurez wants to merge 1 commit into
Conversation
Adds shared-library outputs to the Nanvix OpenSSL build: 1. Switches Configure from 'no-shared' to 'shared no-pinshared'. OpenSSL upstream documents that linking a .so from no-shared-built .a archives breaks the provider system at runtime (core_dispatch is a non-PIC data pointer; OPENSSL_USE_NODELETE is only defined for enable-shared builds). The proper fix is to let OpenSSL build PIC objects via its own configure flag. no-pinshared skips the DSO self-pin path which is not needed on Nanvix (single-process, no unload). 2. Adds libcrypto.so and libssl.so build targets, linked from the (now PIC) static archives via --whole-archive. libssl.so depends on libcrypto.so via DT_NEEDED. libposix/libc/libm symbols are left UND and bind at dlopen time against the host executable's .dynsym -- same model as libxml2.so / libffi.so. 3. Embeds -lgcc into each .so because OpenSSL's bignum/hash arithmetic uses 64-bit operations on 32-bit i686, generating calls to libgcc helpers (__udivdi3, __umoddi3, ...) that libgcc marks hidden -- so they can't be resolved against python.elf's .dynsym at dlopen time. Only stateless arithmetic helpers are pulled in; the structural test below asserts that no stateful libgcc symbols (frame registry, ifunc CPU dispatch state, gcov, split-stack, ...) leak in. See nanvix-todo/libgcc-stateful-symbols-blocklist.md for the audit. The check enforces a hard fail across all three tiers (correctness-critical / instrumentation / split-stack). 4. Adds .nanvix/z.py output_files entries and Makefile.nanvix package/verify-package rules for the new .so files. 5. Strips libcrypto-lib-bss_log.o from libcrypto.a as a defensive post-process. Not strictly needed under the new shared config (the providers/ directory replaces the legacy BIO_s_log path) but kept idempotent in case a future Configure change re-enables it. Runtime dependencies on esaurez/nanvix (must be merged + released before this .so is functionally complete): - esaurez/nanvix#27 (DT_RUNPATH + .init_array) -- already in flight - esaurez/nanvix#28 (diamond DT_NEEDED) -- already in flight (libssl.so depends on libcrypto.so which is itself depended on by _ssl.so + _hashlib.so -- a diamond) - New nanvix PR: pthread_once implementation + pthread_key_create destructor acceptance. Without these, every EVP/SSL operation fails silently. Validation: - Standalone C probe PROBE_PASS (EVP_MD_fetch + OSSL_PROVIDER_load both succeed) - libcrypto.so: 5.5 MB, DT_SONAME=libcrypto.so, full provider+EVP API exported, structural libgcc check passes - libssl.so: 1.4 MB, DT_NEEDED libcrypto.so, full SSL API exported Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Owner
Author
|
Superseded by upstream PR nanvix#252, which carries the same .so build addition -- rebased onto current upstream version branch, scope-trimmed (CI workflow downgrade, nanvix.toml version downgrade, .gitignore tweaks, .zutils-version downgrade, z.ps1/z.sh additions all dropped as unrelated env drift), and esaurez/* / python.elf / nanvix-todo references cleaned from commit message and PR body. Closing this fork PR; tracking continues upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Produce a position-independent
libcrypto.soandlibssl.soin addition to the existing static archives. Mirrors the patterns already shipped by esaurez/libxml2#1 and esaurez/libffi#1.libssl.sodeclaresDT_NEEDED libcrypto.so. Both leavelibposix/libc/libmsymbols unresolved so they bind to the host executable's.dynsymat dlopen time — the same model used by every Nanvix port-library.sotoday.Why
This unblocks the Group B unbundling of
_ssl+_hashlibin CPython (seenanvix-todo/phase2-3-unbundle-bundled-libs.md). Today_ssl.cpython-312.so(6.4 MB) and_hashlib.cpython-312.so(4.8 MB) each statically bundle their own copy of libcrypto — that's 6.3 MB of duplicated code shipped in the cpython ramfs.With
libcrypto.soavailable:_ssl.so→ ~80 KB (stripped) +DT_NEEDED libssl.so, libcrypto.so_hashlib.so→ ~30 KB (stripped) +DT_NEEDED libcrypto.solibssl.so(1.4 MB) andlibcrypto.so(5.5 MB) ship once in/lib/Net ramfs savings: ~11 MB across the cpython release.
Implementation
Configureflagsno-shared no-dso. Now:shared no-pinshared. OpenSSL upstream documents that linking a.sofromno-shared-built.aarchives breaks the provider system at runtime (core_dispatchis a non-PIC data pointer;OPENSSL_USE_NODELETEis only defined forenable-sharedbuilds). The proper fix is to let OpenSSL build PIC objects via its own configure flag.no-pinsharedskips the DSO self-pin path which is not needed on Nanvix (single-process, no unload).$(LIBCRYPTO_SO)targetgcc -shared -fPIC -nostdlib --whole-archive libcrypto.a -lgcc -Wl,-soname,libcrypto.so$(LIBSSL_SO)targetgcc -shared -fPIC -nostdlib --whole-archive libssl.a -lcrypto -lgcc -Wl,-soname,libssl.so. Order matters: libcrypto.so must exist before libssl.so links so that-lcryptoresolves to the.so(and emitsDT_NEEDED) rather than the.a(which would bundle libcrypto into libssl.so and defeat dedup).-lgccembedded__udivdi3,__umoddi3, ...) that libgcc marks hidden, so they can't be resolved againstpython.elf's.dynsymat dlopen time. Embedding-lgccships those leaf helpers inside the.so.nanvix-todo/libgcc-stateful-symbols-blocklist.mdfor the audit. The check enforces a hard fail across all three tiers (correctness-critical / instrumentation / split-stack).bss_log.ostripproviders/directory replaces the legacyBIO_s_logpath) but kept idempotent in case a future Configure change re-enables it..nanvix/z.py_BUILD_OUTPUTS+release()+_verify_release.aand.sovariants.Validation
End-to-end runtime validation requires the loader +
pthread_oncefixes from esaurez/nanvix (see Dependencies). With those in place:Standalone C probe (statically linked against
libcrypto.a+libssl.a):Downstream Python test (cpython.elf using the new
libcrypto.so+libssl.soviaDT_NEEDED):Runtime dependencies
.init_arrayinvocation and DT_NEEDED chain walking. Required at runtime.DT_NEEDEDhandling. The chain_ssl.so + _hashlib.so → libcrypto.sois a diamond atlibcrypto.so, so [zutils] E: Align zutil wrapper scripts with canonical versions nanvix/openssl#28 is required.pthread_onceimplementation +pthread_key_createdestructor acceptance. Without this PR, every OpenSSL EVP/SSL operation fails silently (OPENSSL_init_cryptoreturns 0, no errors queued).Downstream consumer
feat/phase3b-unbundle-openssl— Phase 3b CPython unbundling that switches_ssl/_hashlibto dlopen the .so chain. Will be filed alongside this PR.