Skip to content

[build] E: Build libcrypto.so + libssl.so alongside static archives#1

Closed
esaurez wants to merge 1 commit into
nanvix/v3.5.0from
feat/build-shared-library
Closed

[build] E: Build libcrypto.so + libssl.so alongside static archives#1
esaurez wants to merge 1 commit into
nanvix/v3.5.0from
feat/build-shared-library

Conversation

@esaurez

@esaurez esaurez commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

Produce a position-independent libcrypto.so and libssl.so in addition to the existing static archives. Mirrors the patterns already shipped by esaurez/libxml2#1 and esaurez/libffi#1.

libssl.so declares DT_NEEDED libcrypto.so. Both leave libposix/libc/libm symbols unresolved so they bind to the host executable's .dynsym at dlopen time — the same model used by every Nanvix port-library .so today.

Why

This unblocks the Group B unbundling of _ssl + _hashlib in CPython (see nanvix-todo/phase2-3-unbundle-bundled-libs.md). Today _ssl.cpython-312.so (6.4 MB) and _hashlib.cpython-312.so (4.8 MB) each statically bundle their own copy of libcrypto — that's 6.3 MB of duplicated code shipped in the cpython ramfs.

With libcrypto.so available:

  • _ssl.so → ~80 KB (stripped) + DT_NEEDED libssl.so, libcrypto.so
  • _hashlib.so → ~30 KB (stripped) + DT_NEEDED libcrypto.so
  • libssl.so (1.4 MB) and libcrypto.so (5.5 MB) ship once in /lib/

Net ramfs savings: ~11 MB across the cpython release.

Implementation

Change Detail
Configure flags Was: no-shared no-dso. Now: shared no-pinshared. OpenSSL upstream documents that linking a .so from no-shared-built .a archives breaks the provider system at runtime (core_dispatch is a non-PIC data pointer; OPENSSL_USE_NODELETE is only defined for enable-shared builds). The proper fix is to let OpenSSL build PIC objects via its own configure flag. no-pinshared skips the DSO self-pin path which is not needed on Nanvix (single-process, no unload).
New $(LIBCRYPTO_SO) target gcc -shared -fPIC -nostdlib --whole-archive libcrypto.a -lgcc -Wl,-soname,libcrypto.so
New $(LIBSSL_SO) target gcc -shared -fPIC -nostdlib --whole-archive libssl.a -lcrypto -lgcc -Wl,-soname,libssl.so. Order matters: libcrypto.so must exist before libssl.so links so that -lcrypto resolves to the .so (and emits DT_NEEDED) rather than the .a (which would bundle libcrypto into libssl.so and defeat dedup).
-lgcc embedded OpenSSL bignum/hash arithmetic uses 64-bit operations on 32-bit i686, generating calls to libgcc helpers (__udivdi3, __umoddi3, ...) that libgcc marks hidden, so they can't be resolved against python.elf's .dynsym at dlopen time. Embedding -lgcc ships those leaf helpers inside the .so.
Structural test Hard-fails if stateful libgcc symbols (frame registry, ifunc CPU dispatch state, gcov, split-stack, ...) accidentally get pulled in. See nanvix-todo/libgcc-stateful-symbols-blocklist.md for the audit. The check enforces a hard fail across all three tiers (correctness-critical / instrumentation / split-stack).
bss_log.o strip Defensive post-process — not strictly needed under the new shared config (the providers/ directory replaces the legacy BIO_s_log path) but kept idempotent in case a future Configure change re-enables it.
.nanvix/z.py _BUILD_OUTPUTS + release() + _verify_release Ship both .a and .so variants.

Validation

$ readelf -d libcrypto.so | grep -E 'SONAME|NEEDED'
 0x0000000e (SONAME)  Library soname: [libcrypto.so]

$ readelf -d libssl.so | grep -E 'SONAME|NEEDED'
 0x00000001 (NEEDED)  Shared library: [libcrypto.so]
 0x0000000e (SONAME)  Library soname: [libssl.so]

$ nm -D libcrypto.so | grep -E ' T (EVP_DigestInit|SHA256|OPENSSL_init_crypto)$'
... T EVP_DigestInit
... T OPENSSL_init_crypto
... T SHA256

End-to-end runtime validation requires the loader + pthread_once fixes from esaurez/nanvix (see Dependencies). With those in place:

Standalone C probe (statically linked against libcrypto.a + libssl.a):

PROBE: started
OpenSSL 3.5.0 8 Apr 2025
PROBE: OPENSSL_init_crypto=1
PROBE: OSSL_PROVIDER_load=0x6001fa70
PROBE: EVP_MD_fetch=0x60031d70
PROBE: EVP_get_digestbyname=0x403e8500
PROBE_PASS

Downstream Python test (cpython.elf using the new libcrypto.so + libssl.so via DT_NEEDED):

OPENSSL_CHAIN_PASS  -- SSLContext + sha256 direct call both work

Runtime dependencies

Downstream consumer

Adds shared-library outputs to the Nanvix OpenSSL build:

1. Switches Configure from 'no-shared' to 'shared no-pinshared'. OpenSSL upstream documents that linking a .so from no-shared-built .a archives breaks the provider system at runtime (core_dispatch is a non-PIC data pointer; OPENSSL_USE_NODELETE is only defined for enable-shared builds). The proper fix is to let OpenSSL build PIC objects via its own configure flag. no-pinshared skips the DSO self-pin path which is not needed on Nanvix (single-process, no unload).

2. Adds libcrypto.so and libssl.so build targets, linked from the (now PIC) static archives via --whole-archive. libssl.so depends on libcrypto.so via DT_NEEDED. libposix/libc/libm symbols are left UND and bind at dlopen time against the host executable's .dynsym -- same model as libxml2.so / libffi.so.

3. Embeds -lgcc into each .so because OpenSSL's bignum/hash arithmetic uses 64-bit operations on 32-bit i686, generating calls to libgcc helpers (__udivdi3, __umoddi3, ...) that libgcc marks hidden -- so they can't be resolved against python.elf's .dynsym at dlopen time. Only stateless arithmetic helpers are pulled in; the structural test below asserts that no stateful libgcc symbols (frame registry, ifunc CPU dispatch state, gcov, split-stack, ...) leak in. See nanvix-todo/libgcc-stateful-symbols-blocklist.md for the audit. The check enforces a hard fail across all three tiers (correctness-critical / instrumentation / split-stack).

4. Adds .nanvix/z.py output_files entries and Makefile.nanvix package/verify-package rules for the new .so files.

5. Strips libcrypto-lib-bss_log.o from libcrypto.a as a defensive post-process. Not strictly needed under the new shared config (the providers/ directory replaces the legacy BIO_s_log path) but kept idempotent in case a future Configure change re-enables it.

Runtime dependencies on esaurez/nanvix (must be merged + released before this .so is functionally complete):

- esaurez/nanvix#27 (DT_RUNPATH + .init_array) -- already in flight

- esaurez/nanvix#28 (diamond DT_NEEDED) -- already in flight (libssl.so depends on libcrypto.so which is itself depended on by _ssl.so + _hashlib.so -- a diamond)

- New nanvix PR: pthread_once implementation + pthread_key_create destructor acceptance. Without these, every EVP/SSL operation fails silently.

Validation:

- Standalone C probe PROBE_PASS (EVP_MD_fetch + OSSL_PROVIDER_load both succeed)

- libcrypto.so: 5.5 MB, DT_SONAME=libcrypto.so, full provider+EVP API exported, structural libgcc check passes

- libssl.so: 1.4 MB, DT_NEEDED libcrypto.so, full SSL API exported

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@esaurez

esaurez commented Jun 9, 2026

Copy link
Copy Markdown
Owner Author

Superseded by upstream PR nanvix#252, which carries the same .so build addition -- rebased onto current upstream version branch, scope-trimmed (CI workflow downgrade, nanvix.toml version downgrade, .gitignore tweaks, .zutils-version downgrade, z.ps1/z.sh additions all dropped as unrelated env drift), and esaurez/* / python.elf / nanvix-todo references cleaned from commit message and PR body. Closing this fork PR; tracking continues upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant