Compatibility Issues About Python Wrapper for UUtils

[HansBug]

Does this library support Win32 compile?

I'm trying to create a cross-platform Python wrapper library for UUtils with pybind11, but when I was trying to compile it on Win32, the unittest in ctest command (just the command mentioned in README) failed: https://github.com/HansBug/pyuutils/actions/runs/12074240838/job/33671876202#step:4:595

Seems something wrong with win32. If win32 are not planned to get supported, we will give up supporting this arch.

BTW, our working progress:

1. Linux, both x86_64/aarch64 passed compiling, PyPI wheel building and unittests
2. Windows, AMD64 passed all those, but for Win32, unittest for UUtils failed
3. MacOS, x86_64 passed all those, but for arm64 there is still something wrong with pypi wheel building.

And currently only a few functions (hash computations) supported, just for the very early try. We did these for academical purpose, and maybe we will create more wrappers for other uppaal cpp libraries in the future. Python libraries can significantly boost up the researches, experiments and open-source developments. So we may need some help from your official team.

[HansBug]

Some new attempts:

1. I just tried many archs on linux (e.g. i686), most of them are well supported
2. but something wrong on macos arm64 arch, but failed: https://github.com/HansBug/pyuutils/actions/runs/12075076585/job/33674231945#step:4:429 ctest failed. and the xxhash library's compile arch seems to be x86_64, i have tried all what i can do to fix that, but failed.

So I decide to give up the support for macos arm64 temporarily, if you are willing to support macos arm64 plz let me know, I think maybe the cmake scripts of UUtils may need some changes.

Also I found that in compile.sh of UUtils, only x86 archs are officialy supported. LoL

[mikucionisaau]

UUtils contains some platform specific code (like measuring CPU and memory consumption), so it is not portable out of the box.

We use MINGW to cross-compile for Windows (also available on most Linux distributions).
The 32bit builds used to work, but we abandoned them some time ago as there is no use for it.

Your macos arm64 logs show that the build succeeded, but tests have failed.

We haven't ported to arm64 yet (I've got arm64 mac only a month ago, and given the Apple attitude it looks like a massive waste of time and resources).

[mikucionisaau]

May I ask what are you trying to achieve?
UUtils are low level utilities used by other C++ libraries, I don't think there is anything interesting for python.

[HansBug]

UUtils contains some platform specific code (like measuring CPU and memory consumption), so it is not portable out of the box.
We use MINGW to cross-compile for Windows (also available on most Linux distributions).
The 32bit builds used to work, but we abandoned them some time ago as there is no use for it.

Got it. We will stop trying win32 build.

We haven't ported to arm64 yet (I've got arm64 mac only a month ago, and given the Apple attitude it looks like a massive waste of time and resources).

The same, when you plan to support that plz let me know.

May I ask what are you trying to achieve?
UUtils are low level utilities used by other C++ libraries, I don't think there is anything interesting for python.

Well, actually we are planning to create python wrapper for UDBM and other C++ libs of uppaal, we are building devops on github action. We are currently not familiar enough to all the source code & project structure of UDBM, so we selected a simpler one (UUtils) as experimental attempt.

We know that the UUtils is just a utils library for other libs. So after we completed the github action workflow, we will try to create UDBM python wrapper libraries, that is useful for academical usages because python has really much scientific libs supported.

[HansBug]

one more thing, there is something wrong with compiling on windows-amd64.

in code https://github.com/UPPAALModelChecker/UUtils/blob/main/src/base/platform.c#L59 this means only mingw compiler supported (otherwise the function base_getMemInfo will gone, i tried with compiling on visual studio and met that problem)

but, when i tried to compile it on windows gcc/g++, the ctest will fail, the failed cases are all runtime errors with exit codes

• compiling on gcc/g++ 8.1: https://github.com/HansBug/pyuutils/actions/runs/12088908630/job/33713286211#step:15:288
• compiling on gcc/g++ 12: https://github.com/HansBug/pyuutils/actions/runs/12089143907/job/33713952872#step:15:300

this issue is in src/base/platform.c, and due to the relationship between UDBM and UUtils, i think this issue should be solved, otherwise the wrapping for UDBM will fail as well.

BTW, about the compiling on visual studio, here is the error: https://github.com/HansBug/pyuutils/actions/runs/12104621198/job/33748252338#step:18:85 . The ctest can pass because unittest donest covered the platform.c, but actually those functions are gone because __MINGW32__ cannot be triggered.

[HansBug]

@mikucionisaau i just noticed in pr #32 (https://github.com/UPPAALModelChecker/UUtils/pull/32/files#diff-57a29e4ff645988a79ebbf4c1d9a6734558bc8f7e7149ed5102911fa7cf16c7f ), the arm64 support for macos seems to be added, but as i tested just now, it still cannot pass the ctest on my side: https://github.com/HansBug/pyuutils/actions/runs/12138911583/job/33845356591 .

and i noticed the code in compile.sh, arm64-darwin seems to be targetted to x86_64-darwin.

        arm64-darwin*)
            BUILD_TARGET=x86_64-darwin
            ;;
        *)

hmmm, so is this a expected result of the current version? or that is a mistake?

some more information

i found that when cross compiling arm64 version on windows + cibuildwheel (it actually runs on a macos x86_64 env, how cibuildwheel works ), the xxhash seems to use x86_64 instead of arm64 (which is expected), like this: https://github.com/HansBug/pyuutils/actions/runs/12138657433/job/33844581993#step:4:292

  -- xxHash build type: Release
  -- Architecture: x86_64
  -- Got xxHash: /Users/runner/work/pyuutils/pyuutils/build/_deps/xxhash-src

i used the extra variables in the cmake command, but didnt help

cmake -S UUtils -B build -DCMAKE_OSX_ARCHITECTURES="arm64" -DCMAKE_SYSTEM_PROCESSOR=arm64 -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_TRY_COMPILE_OSX_ARCHITECTURES=arm64

i read the xxhash source code, this is their implement in CMakeLists.txt

if("${CMAKE_VERSION}" VERSION_LESS "3.10")
  # Can not enable DISPATCH mode since it fails to recognize architecture.
else()
  CMAKE_HOST_SYSTEM_INFORMATION(RESULT PLATFORM QUERY OS_PLATFORM)
  message(STATUS "Architecture: ${PLATFORM}")
endif()

it will directly detect your env and seems the -Dxxxx didnt affect it.

i wonder this is why i got the ctest error when compiling UUtils on macos arm64

  The following tests FAILED:
  	  8 - base_crash_allocator_0 (BAD_COMMAND)
  	 12 - base_crash_allocator_4 (BAD_COMMAND)
  	 13 - base_int_utils_10 (BAD_COMMAND)
  	 14 - base_int_utils_10000 (BAD_COMMAND)
  	 15 - base_item_allocator_1 (BAD_COMMAND)
  	 16 - base_item_allocator_10 (BAD_COMMAND)
  	 17 - base_item_allocator_100000 (BAD_COMMAND)
  	 18 - base_meta (BAD_COMMAND)
  	 19 - base_random_seed (BAD_COMMAND)
  	 20 - test_sequencefilter (BAD_COMMAND)
  	 23 - debug_new_0 (BAD_COMMAND)
  	 24 - debug_new_1 (BAD_COMMAND)
  	 26 - debug_new_3 (BAD_COMMAND)
  	 32 - hash_compute (BAD_COMMAND)
  	 33 - hash_tables_0 (BAD_COMMAND)
  	 34 - hash_tables_1 (BAD_COMMAND)
  	 35 - hash_tables_10000 (BAD_COMMAND)
  	 36 - hash_tables_1000000 (BAD_COMMAND)

[mikucionisaau]

Yes, I made a copy-paste mistake.. Anyway, I am adding proper Darwin arm64 here: #33

[mikucionisaau]

I am finished with my fixes, please checkout main branch.

[HansBug]

I am finished with my fixes, please checkout main branch.

well, thank you for this fix.

I just tested the latest main commit, the full-platform result is: https://github.com/HansBug/pyuutils/actions/runs/12142878131/job/33861064998#step:3:1

macos-arm64 is now properly supported, the only pity is python3.8 doesnt seems to support apple m1, so python3.8's macos arm64 is unavailable.

btw, i noticed another thing:

1. when macos-12 is used as github action runner, its architecture is x86_64 (uname -m shows x86_64)
2. while when macos-14 is used, its architecture is arm64

this also appeared in your pr's unittest:

1. arm64 build: https://github.com/HansBug/pyuutils/actions/runs/12142878131/job/33861064998#step:3:1 xxhash's architecture is arm64
2. x86_64 build: https://github.com/UPPAALModelChecker/UUtils/actions/runs/12142137898/job/33855879519#step:4:41 xxhash's architecture is arm64, and the actual ctest running environment is arm64 as well, not x86_64
   so... im not sure if this unittest is reliable. but well its not a problem, our test shows this code is compitable with macos-arm64

new information: macos-12 is deprecated by github just now, i tried macos-13, it is x86_64 arch as well.