Skip to content

[GLUTEN-8232][VL] Allow to enable dynamic openssl link in VCPKG packaging#11444

Open
zhouyuan wants to merge 11 commits into
apache:mainfrom
zhouyuan:wip_openssl_dynamic
Open

[GLUTEN-8232][VL] Allow to enable dynamic openssl link in VCPKG packaging#11444
zhouyuan wants to merge 11 commits into
apache:mainfrom
zhouyuan:wip_openssl_dynamic

Conversation

@zhouyuan
Copy link
Copy Markdown
Member

@zhouyuan zhouyuan commented Jan 19, 2026 •
edited
Loading

What changes are proposed in this pull request?

This patch enabled dynamic openssl to include FIPS feature in vcpkg build.
The feature is off by default, it can be enabled by set env VCPKG_DYNAMIC_OPENSSL to ON

In the runtime user will need to ensure the libssl.so & libcrypt.so are available otherwise Gluten will fail to run

How was this patch tested?

pass GHA

Related issue: #8232

@zhouyuan zhouyuan force-pushed the wip_openssl_dynamic branch from ce8f244 to b55532a Compare January 19, 2026 14:35
@zhouyuan zhouyuan changed the title [GLUTEN-8232][VL] enable dynamic openssl in vcpkg [GLUTEN-8232][VL] Enable dynamic openssl lib in vcpkg packaging Jan 19, 2026
@zhouyuan zhouyuan requested a review from philo-he January 19, 2026 14:43
@zhouyuan zhouyuan force-pushed the wip_openssl_dynamic branch from 9febdcd to 5079457 Compare January 19, 2026 17:44
philo-he
philo-he reviewed Jan 20, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@philo-he philo-he left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two comments. Please check if they make sense. Thanks.

Comment thread .github/workflows/velox_backend_x86.yml Outdated
run: |
docker pull apache/gluten:vcpkg-centos-7
docker run -v $GITHUB_WORKSPACE:/work -w /work apache/gluten:vcpkg-centos-7 bash -c "
docker pull apache/gluten:vcpkg-centos-9
Copy link
Copy Markdown
Member

@philo-he philo-he Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest setting container field with centos 9 image, which is a standard way. I assume it's compatible with GHA checkout.

Copy link
Copy Markdown
Member Author

@zhouyuan zhouyuan May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Comment thread dev/vcpkg/triplets/x64-linux-avx.cmake
endif()

if(PORT STREQUAL "openssl")
set(VCPKG_LIBRARY_LINKAGE dynamic)
Copy link
Copy Markdown
Member

@philo-he philo-he Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use dynamic link only when FIPS is enabled?

My understanding is that FIPS is not a requirement for many users. Then, the current static link for openssl may be good for them.

Copy link
Copy Markdown
Member Author

@zhouyuan zhouyuan May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only enable this when VCPKG_DYNAMIC_OPENSSL is on

@zhouyuan zhouyuan force-pushed the wip_openssl_dynamic branch from aabd20e to 2f47e7d Compare January 20, 2026 18:06
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 7, 2026

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions Bot added the stale stale label Mar 7, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 17, 2026

This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.

@github-actions github-actions Bot closed this Mar 17, 2026
@FelixYBW
Copy link
Copy Markdown
Contributor

FelixYBW commented May 15, 2026

@zhouyuan @philo-he has a fix of #8293

@FelixYBW FelixYBW reopened this May 16, 2026
@github-actions github-actions Bot removed the stale stale label May 17, 2026
@philo-he
Copy link
Copy Markdown
Member

philo-he commented May 21, 2026

FYI. From what I recall based on my previous investigation, the preferred approach is to dynamically link OpenSSL installed in user's environment to use the FIPS feature, rather than going through vcpkg.

@FelixYBW
Copy link
Copy Markdown
Contributor

FelixYBW commented May 22, 2026

FYI. From what I recall based on my previous investigation, the preferred approach is to dynamically link OpenSSL installed in user's environment to use the FIPS feature, rather than going through vcpkg.

I see. Since we didn't specify an absolute path for link, the difference here is only the openssl version liked to Gluten. FIPS is transparent at link time.

@zhouyuan
[VL] enable dynamic openssl in vcpkg
44b6235 
Signed-off-by: Yuan <yuanzhou@apache.org>

fix

Signed-off-by: Yuan <yuanzhou@apache.org>

install libcrypto/libssl

Signed-off-by: Yuan <yuanzhou@apache.org>

fix

Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_openssl_dynamic branch from 73d297a to 44b6235 Compare May 25, 2026 14:01
@zhouyuan
refactor
f7e9cdd 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan marked this pull request as ready for review May 25, 2026 14:08
zhouyuan added 3 commits May 25, 2026 15:10
@zhouyuan
adding arm support
8b8d3f5 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
refine
3116f64 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
remove deprecated file
30980cd 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_openssl_dynamic branch from eb17800 to 30980cd Compare May 25, 2026 14:33
@github-actions github-actions Bot removed the INFRA label May 25, 2026
zhouyuan added 3 commits May 25, 2026 15:41
@zhouyuan
Revert "remove deprecated file"
d02e9b4 
This reverts commit 30980cd.
@zhouyuan
fix
3289a34
@zhouyuan
merge with original cmake
357869b 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions github-actions Bot added the INFRA label May 25, 2026
zhouyuan added 3 commits May 26, 2026 13:39
@zhouyuan
skip copying libs
422fe94 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
add test
baf5225
@zhouyuan
Revert "add test"
dcf9816 
This reverts commit baf5225.
@zhouyuan
Copy link
Copy Markdown
Member Author

zhouyuan commented May 26, 2026

with this feature the libvelox.so will link dynamically

ldd cpp/build/releases/libvelox.so
        linux-vdso.so.1 (0x00007ffd4d590000)
        libgluten.so => not found
        libssl.so.3 => /lib64/libssl.so.3 (0x00007ff0e4c7e000)
        libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007ff0dbe00000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff0dbd23000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff0dba00000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff0e4d81000)
        libz.so.1 => /lib64/libz.so.1 (0x00007ff0e4c62000)

@zhouyuan zhouyuan requested a review from philo-he May 26, 2026 13:11
@zhouyuan zhouyuan changed the title [GLUTEN-8232][VL] Enable dynamic openssl lib in vcpkg packaging [GLUTEN-8232][VL] Allow to enable dynamic openssl lib in VCPKG packaging May 26, 2026
@zhouyuan zhouyuan changed the title [GLUTEN-8232][VL] Allow to enable dynamic openssl lib in VCPKG packaging [GLUTEN-8232][VL] Allow to enable dynamic openssl link in VCPKG packaging May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@philo-he philo-he Awaiting requested review from philo-he

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

BUILD INFRA

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zhouyuan @FelixYBW @philo-he