Fix Clang module builds using outdated build flags #5425
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`_append_requires_flags` won't actually regenerate the flags unless `opt.build` is true, but the callers never set it, so changing module dependencies won't affect the flags until a clean build.
|
Note that this might also affect MSVC, but I can't test that and am unfamiliar with their modules implementation. |
refi64
commented
Aug 1, 2024
| end | ||
|
|
||
| -- for cpp file we need to check after appendings the flags | ||
| if build == nil then | ||
| if not provide and not compiler_support.has_module_extension(opt.cppfile) then |
There was a problem hiding this comment.
I'm a little...uncertain about this. It essentially means that, for non-modules, should_build gets called twice, one before the requires flags are added and one after, In theory, the second one wouldn't be needed, because _append_requires_flags seems to always compute the flags from opt.module.requires, which was already passed to the first call. However, the above comment explicitly says the check needs to happen after the flags are added, which makes me think that there's a distinct reason for that? But I don't know what that reason is, so I just left the second call in place. If I can safely remove this, I'll just update the PR.
|
Hello, i'm on vacancy, i'll review it when i come back |
|
can u try https://github.com/Arthapz/xmake/tree/fix-module-scanning and confirm that the issue is fixed or not ? |
@refi64 Does it work now? and can you provide a reproducible problem project example? |
|
@Arthapz no, that doesn't seem to fix it. @waruqi The simplest repro I have is:
|
i can't reproduce
|
|
I was able to reproduce it on my macOS. and I tried this patch https://github.com/Arthapz/xmake/tree/fix-module-scanning , it still does not work. $ cd tests/projects/c++/modules/hello
ruki:hello ruki$ rm -rf build/; rm -rf .xmake/
ruki:hello ruki$ echo 'export module hello2;
export void nothing() {}' > src/hello2.mpp
ruki:hello ruki$ xmake f --toolchain=llvm -c
checking for platform ... macosx
checking for architecture ... x86_64
checking for Xcode directory ... /Applications/Xcode.app
ruki:hello ruki$ xmake project -k compile_commands
[ 0%]: <hello> generating.module.deps src/main.cpp
[ 0%]: <hello> generating.module.deps src/hello2.mpp
[ 0%]: <hello> generating.module.deps src/hello.mpp
[ 0%]: <hello> generating.module.deps /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.c
ppm
[ 0%]: <hello> generating.module.deps /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.c
ompat.cppm
create ok!
warning: some modules have got culled, use verbose (-v) mode to more informations
ruki:hello ruki$ xmake
[ 42%]: <hello> compiling.module.release hello
[ 71%]: compiling.release src/main.cpp
[ 85%]: linking.release hello
[100%]: build ok, spent 1.384s
warning: some modules have got culled, use verbose (-v) mode to more informations
ruki:hello ruki$ vim ./src/main.cpp # edit and add `import hello2;`
ruki:hello ruki$ cat ./src/main.cpp
import hello2;
import hello;
int main() {
hello::say("hello module!");
return 0;
}
ruki:hello ruki$ xmake
[ 0%]: <hello> generating.module.deps src/main.cpp
[ 50%]: <hello> compiling.module.release hello2
[ 75%]: compiling.release src/main.cpp
error: src/main.cpp:1:8: fatal error: module 'hello2' not found
1 | import hello2;
| ~~~~~~~^~~~~~
1 error generated.
> in src/main.cpp |
|
full verbose logs ruki:hello ruki$ rm -rf .xmake/
ruki:hello ruki$ rm -rf build/
ruki:hello ruki$ xmake f --toolchain=llvm -c -vD
checking for platform ... macosx
checking for architecture ... x86_64
checking for Xcode directory ... /Applications/Xcode.app
checking for SDK version of Xcode for macosx (x86_64) ... 14.0
checking for clang ... /usr/local/Cellar/llvm/18.1.5/bin/clang
checking for the c++ compiler (cxx) ... clang
checking for /usr/local/Cellar/llvm/18.1.5/bin/clang ... ok
checking for flags (-fPIC) ... ok
> clang "-fPIC" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/M
acOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (clang_modules) ... ok
> clang "-fmodules" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platfor
ms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (clang_modules_ts) ... no
> clang "-fmodules-ts" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Plat
forms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checkinfo: @programdir/core/sandbox/modules/os.lua:273: clang: error: unknown argument: '-fmodules-ts'
stack traceback:
[C]: in function 'error'
[@programdir/core/base/os.lua:973]:
[@programdir/core/sandbox/modules/os.lua:273]: in function 'runv'
[@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
checking for /usr/local/Cellar/llvm/18.1.5/bin/clang ... ok
configure
{
theme = default
ndk_stdcxx = true
plat = macosx
network = public
toolchain = llvm
buildir = build
ccache = true
kind = static
proxy_pac = pac.lua
clean = true
arch = x86_64
host = macosx
mode = release
xcode = /Applications/Xcode.app
}
ruki:hello ruki$ xmake project -k compile_commands -vD
configure
{
plat = macosx
network = public
clean = true
theme = default
arch = x86_64
proxy_pac = pac.lua
ndk_stdcxx = true
mode = release
xcode = /Applications/Xcode.app
buildir = build
kind = static
ccache = true
toolchain = llvm
host = macosx
}
checking for flags (clang_print_library_module_manifest_path) ... no
> clang "-print-library-module-manifest-path" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/
Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checkinfo: @programdir/core/sandbox/modules/os.lua:273: clang: error: unknown argument: '-print-library-module-
manifest-path'
stack traceback:
[C]: in function 'error'
[@programdir/core/base/os.lua:973]:
[@programdir/core/sandbox/modules/os.lua:273]: in function 'runv'
[@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
checking for flags (-fvisibility-inlines-hidden) ... ok
> clang "-fvisibility-inlines-hidden" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents
/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (-O3) ... ok
> clang "-O3" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/Mac
OSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (-std=c++20) ... ok
> clang "-std=c++20" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platfo
rms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (-DNDEBUG) ... ok
> clang "-DNDEBUG" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platform
s/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
[ 0%]: <hello> generating.module.deps src/main.cpp
checking for /usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps ... ok
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c src/main.cpp -o build/.objs/hello/macosx/x86_64/release/src/main.cpp.o -Qunused-arguments -m64 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibilit
y=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
[ 0%]: <hello> generating.module.deps src/hello2.mpp
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c src/hello2.mpp -o build/.objs/hello/macosx/x86_64/release/src/hello2.mpp.o -Qunused-arguments -m64 -isys
root /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisib
ility=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
[ 0%]: <hello> generating.module.deps src/hello.mpp
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c src/hello.mpp -o build/.objs/hello/macosx/x86_64/release/src/hello.mpp.o -Qunused-arguments -m64 -isysro
ot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibil
ity=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
[ 0%]: <hello> generating.module.deps /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.c
ppm
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.cppm -o build/.objs/hello/macosx/
x86_64/release/usr/local/Cellar/llvm/18.1.5/bin/__/lib/c++/__/__/share/libc++/v1/std.cppm.o -Qunused-arguments
-m64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.s
dk -fvisibility=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
[ 0%]: <hello> generating.module.deps /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.c
ompat.cppm
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c /usr/local/Cellar/llvm/18.1.5/bin/../lib/c++/../../share/libc++/v1/std.compat.cppm -o build/.objs/hello/
macosx/x86_64/release/usr/local/Cellar/llvm/18.1.5/bin/__/lib/c++/__/__/share/libc++/v1/std.compat.cppm.o -Qunu
sed-arguments -m64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDK
s/MacOSX14.0.sdk -fvisibility=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
checking for flags (clang_module_file) ... ok
> clang "-fmodule-file=/var/folders/32/w9cz0y_14hs19lkbs6v6_fm80000gn/T/.xmake501/240819/_4A6038DE78C84C108ED19
5C53DC08D20.pcm" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/
MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for flags (clang_module_output) ... ok
> clang "-fmodule-output=" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/
Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
create ok!
warning: some modules have got culled, because it is not consumed by its target nor flagged as a public module
with add_files("xxx.mpp", {public = true})
hello:
hello2 -> src/hello2.mpp
ruki:hello ruki$ xmake project -k compile_commands -vD^C
ruki:hello ruki$ xmake -vD
[ 42%]: <hello> compiling.module.release hello
/usr/local/Cellar/llvm/18.1.5/bin/clang -c -Qunused-arguments -m64 -isysroot /Applications/Xcode.app/Contents/D
eveloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibility=hidden -fvisibility-inlines-hidde
n -O3 -std=c++20 -DNDEBUG -x c++-module -fmodule-output=build/.gens/hello/macosx/x86_64/release/rules/bmi/cache
/modules/c83caa2b/hello.pcm -o build/.objs/hello/macosx/x86_64/release/src/hello.mpp.o src/hello.mpp
checking for flags (-fdiagnostics-color=always) ... ok
> clang "-fdiagnostics-color=always" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/
Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
[ 71%]: compiling.release src/main.cpp
/usr/local/Cellar/llvm/18.1.5/bin/clang -c -Qunused-arguments -m64 -isysroot /Applications/Xcode.app/Contents/D
eveloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibility=hidden -fvisibility-inlines-hidde
n -O3 -std=c++20 -DNDEBUG -fmodule-file=hello=build/.gens/hello/macosx/x86_64/release/rules/bmi/cache/modules/c
83caa2b/hello.pcm -o build/.objs/hello/macosx/x86_64/release/src/main.cpp.o src/main.cpp
checking for flags (-MMD -MF) ... ok
> clang "-MMD" "-MF" "/dev/null" "-Qunused-arguments" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Deve
loper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk"
checking for clang++ ... /usr/local/Cellar/llvm/18.1.5/bin/clang++
checking for the linker (ld) ... clang++
checking for /usr/local/Cellar/llvm/18.1.5/bin/clang++ ... ok
checking for flags (-fPIC) ... ok
> clang++ "-fPIC" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Deve
loper/SDKs/MacOSX14.0.sdk" "-m64" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.plat
form/Developer/SDKs/MacOSX14.0.sdk"
[ 85%]: linking.release hello
/usr/local/Cellar/llvm/18.1.5/bin/clang++ -o build/macosx/x86_64/release/hello build/.objs/hello/macosx/x86_64/
release/src/main.cpp.o build/.objs/hello/macosx/x86_64/release/src/hello.mpp.o -m64 -isysroot /Applications/Xco
de.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -Wl,-x -Wl,-dead_strip
build cache stats:
cache directory: build/.build_cache
cache hit rate: 0%
cache hit: 0
cache hit total time: 0.000s
cache miss: 0
cache miss total time: 0.000s
new cached files: 0
remote cache hit: 0
remote new cached files: 0
preprocess failed: 0
compile fallback count: 0
compile total time: 0.000s
[100%]: build ok, spent 1.284s
warning: some modules have got culled, because it is not consumed by its target nor flagged as a public module
with add_files("xxx.mpp", {public = true})
hello:
hello2 -> src/hello2.mpp
ruki:hello ruki$ xmake -vD^C
ruki:hello ruki$ vim ./src/main.cpp
ruki:hello ruki$ xmake -vD
[ 0%]: <hello> generating.module.deps src/main.cpp
/usr/local/Cellar/llvm/18.1.5/bin/clang-scan-deps --format=p1689 -- /usr/local/Cellar/llvm/18.1.5/bin/clang -x
c++ -c src/main.cpp -o build/.objs/hello/macosx/x86_64/release/src/main.cpp.o -Qunused-arguments -m64 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibilit
y=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -DNDEBUG
[ 50%]: <hello> compiling.module.release hello2
/usr/local/Cellar/llvm/18.1.5/bin/clang -c -Qunused-arguments -m64 -isysroot /Applications/Xcode.app/Contents/D
eveloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibility=hidden -fvisibility-inlines-hidde
n -O3 -std=c++20 -DNDEBUG -x c++-module -fmodule-output=build/.gens/hello/macosx/x86_64/release/rules/bmi/cache
/modules/c83caa2b/hello2.pcm -o build/.objs/hello/macosx/x86_64/release/src/hello2.mpp.o src/hello2.mpp
[ 75%]: compiling.release src/main.cpp
/usr/local/Cellar/llvm/18.1.5/bin/clang -c -Qunused-arguments -m64 -isysroot /Applications/Xcode.app/Contents/D
eveloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fvisibility=hidden -fvisibility-inlines-hidde
n -O3 -std=c++20 -DNDEBUG -fmodule-file=hello=build/.gens/hello/macosx/x86_64/release/rules/bmi/cache/modules/c
83caa2b/hello.pcm -o build/.objs/hello/macosx/x86_64/release/src/main.cpp.o src/main.cpp
error: @programdir/core/main.lua:329: @programdir/actions/build/main.lua:148: @programdir/modules/async/runjobs
.lua:325: @programdir/modules/private/action/build/object.lua:91: @programdir/modules/core/tools/gcc.lua:916: s
rc/main.cpp:1:8: fatal error: module 'hello2' not found
1 | import hello2;
| ~~~~~~~^~~~~~
1 error generated.
stack traceback:
[C]: in function 'error'
[@programdir/core/base/os.lua:973]:
[@programdir/modules/core/tools/gcc.lua:916]: in function 'catch'
[@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
[@programdir/modules/core/tools/gcc.lua:857]:
[C]: in function 'xpcall'
[@programdir/core/base/utils.lua:275]:
[@programdir/core/tool/compiler.lua:278]: in function 'compile'
[@programdir/modules/private/action/build/object.lua:91]: in function 'script'
[@programdir/modules/private/action/build/object.lua:122]: in function 'build_object'
[@programdir/modules/private/action/build/object.lua:147]: in function 'jobfunc'
[@programdir/modules/async/runjobs.lua:241]:
[C]: in function 'xpcall'
[@programdir/core/base/utils.lua:275]: in function 'trycall'
[@programdir/core/sandbox/modules/try.lua:117]: in function 'try'
[@programdir/modules/async/runjobs.lua:223]: in function 'cotask'
[@programdir/core/base/scheduler.lua:406]:
stack traceback:
[C]: in function 'error'
@programdir/core/base/os.lua:973: in function 'base/os.raiselevel'
(...tail calls...)
@programdir/core/main.lua:329: in upvalue 'cotask'
@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
ruki:hello ruki$ |
|
Of it was working under gcc, but not with clang |
Merged
|
thanks, I will try it in these days. |
it still does not work. same errors. ruki-2:hello ruki$ xmake
[ 0%]: <hello> generating.module.deps src/main.cpp
[ 37%]: <hello> compiling.module.release hello2
[ 75%]: compiling.release src/main.cpp
error: src/main.cpp:2:8: fatal error: module 'hello2' not found
2 | import hello2;
| ~~~~~~~^~~~~~
1 error generated.
> in src/main.cppyou can try #5425 (comment) |
|
well i don't get it to reproduce |
|
I can stably reproduce it on both macOS and Linux. llvm 14-18. Follow my above reproduction process. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
_append_requires_flagswon't actually regenerate the flags unlessopt.buildis true, but the callers never set it, so changing module dependencies won't affect the flags until a clean build.