ImportError: DLL load failed

[yves-surrel]

Hi
I managed (painfully) to have

pip install .

compile the whole thing without error, on Windows 10 with the latest Microsoft Build Tools. But importing gpyfft in python 3.7.6 raises the following error:

import gpyfft
Traceback (most recent call last):

  File "<ipython-input-2-3687f3cada1c>", line 1, in <module>
    import gpyfft

  File "C:\Users\Yves\Anaconda3\lib\site-packages\gpyfft\__init__.py", line 6, in <module>
    from .gpyfftlib import GpyFFT, GpyFFT_Error, Plan

ImportError: DLL load failed: La procédure spécifiée est introuvable.

(the specified procedure could not be found). Any idea of what could be happening?

[geggo]

I have gpyfft compiled and running on Windows with Anaconda Python 3.x, typically via
python setup.py build
python setup.py install

It could be you have an older version of the clfft library or is found somewhere else on the path. You could try to use DependencyWalker to check for which libraries are actually loaded.

Gregor

[yves-surrel]

Hi Gregor Thanks for the answer. Actually, I also recompiled clFFT, so it is not a version problem. It may be related to FFTW, as I cannot manage to have cmake find the libraries: I set the env variable FFTW_ROOT, but I have the warning message CMake Warning (dev) at CMakeLists.txt:153 (find_package): Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables. Run "cmake --help-policy CMP0074" for policy details. Use the cmake_policy command to set the policy and suppress this warning. Environment variable FFTW_ROOT is set to: D:\Developpement\fftw-3.3.5-dll64 For compatibility, CMake is ignoring the variable. This warning is for project developers. Use -Wno-dev to suppress it.

…

-- Could NOT find FFTW (missing: FFTW_LIBRARIES) -- FindFFTW looked for single precision libraries named: fftw3f or libfftw3f-3 -- FindFFTW looked for double precision libraries named: fftw3 or libfftw3-3 I tried to edit FindFFTW.cmake and put set( FFTW_LIBRARIES "/Developpement/fftw-3.3.5-dll64") as the first line but it didn't change anything. How would you do ? Thx Yves Le 2020-01-20 13:45, Gregor Thalhammer a écrit :

I have gpyfft compiled and running on Windows with Anaconda Python 3.x, typically via python setup.py build python setup.py install It could be you have an older version of the clfft library or is found somewhere else on the path. You could try to use DependencyWalker to check for which libraries are actually loaded. Gregor -- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. Links: ------ [1] #45?email_source=notifications&amp;email_token=AFQM3HC2CADLH3IYOVIQAJLQ6WMHRA5CNFSM4KITMHU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJMQFYQ#issuecomment-576258786 [2] https://github.com/notifications/unsubscribe-auth/AFQM3HHOIQSOXEVUPTGDNPTQ6WMHRANCNFSM4KITMHUQ

[geggo]

Hi Yves,

fftw is optional for building clfft, it is only needed for validating results in tests. Building clFFT should gracefully proceed without fftw installed.
Did you try using DependencyWalker?

[yves-surrel]

Hi Gregor I tried DependencyWalker with FFT.dll, I got some errors, but apparently the same as when analyzing the original precompiled file from the clFFT web site. IMHO, the problem seems to be elsewhere. I can run the following code in python from ctypes import * libc = cdll.LoadLibrary('/Users/Yves/Anaconda3/Lib/site-packages/gpyfft/clFFT.dll') print(libc.clfftSetup) with no complain about the dll not being loaded. But if I try import gpyfft, I get File "C:\Users\Yves\Anaconda3\lib\site-packages\gpyfft\__init__.py", line 6, in <module> from .gpyfftlib import GpyFFT, GpyFFT_Error, Plan Could the problem be in gpyfftlib ? I do not know what kind of file it is. BTW, I ran cmake in a directory on a D: drive, not C:. Could this have had an influence? Best Yves Le 2020-01-20 19:41, Gregor Thalhammer a écrit :

…

Hi Yves, fftw is optional for building clfft, it is only needed for validating results in tests. Building clFFT should gracefully proceed without fftw installed. Did you try using DependencyWalker? -- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. Links: ------ [1] #45?email_source=notifications&amp;email_token=AFQM3HGSSCOCYJSXLZ5IV2LQ6XV5TA5CNFSM4KITMHU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJNQQNA#issuecomment-576391220 [2] https://github.com/notifications/unsubscribe-auth/AFQM3HBZWGAKMDYWC5RK333Q6XV5TANCNFSM4KITMHUQ

[geggo]

somewhere in your gpyfft module directory should be a gpyfftlib.XXX file, with the ending depending on your platform. Actually, it is dll file which you can analyze with DependencyWalker. It is linked to clFFT.dll, which needs to be found, i.e., it should be placed in the same folder as gpyfftlib or be found on the search path.
Possibly, clFFT.dll (and also statTimer.dll) is not found, or an incompatible version.

hope that helps
Gregor

[yves-surrel]

Hi Gregor It seems that there may be a problem with MacOSX and Python 3.7. I can run with no error python setup.py build python setup.py install in terminals open from Enthought Canopy (that I want to deprecate), using python 3.5, and from Anaconda, using Python 3.7 In the first case, I can run successfully the simple_example.py file. In the site-packages/gpyfft-0.7.0-py3.5-macosx-10.6-x86_64.egg/gpyfft there is a file gpyfftlib.cpython-35m-darwin.so In the second case, python 3.7 crashes with the message : Segmentation fault: 11 In the site-packages/gpyfft-0.7.0-py3.7-macosx-10.9-x86_64.egg/gpyfft there is a file gpyfftlib.cpython-37m-darwin.so Have you any idea on where it comes from and how this could be fixed ? Thx Yves

[yves-surrel]

Hi Gregor I confirm that the problem is related to using python 3.7 I created a new environment in Anaconda based on Python 3.5. I could build, install and run simple_example.py Yves

[yves-surrel]

Hi Gregor It also succeeds with python 3.6. Yves

[yves-surrel]

Hi Gregor This may help: conda-forge/python-feedstock#272 Yves

[geggo]

Hi Yves,

actually Anaconda Python 3.7 on MacOS is my primary development platform. I just pushed my local changes, including some modifications of setup.py to github. Perhaps this fixes your segfaults.
Otherwise I would recommend making a clean rebuild of gpyfft, and search for stale gpyfftlib*.so files, I remember this might cause troubles.
hopes that helps
Gregor

[yves-surrel]

Thx for the reply. I downloaded a fresh gpyfft-master directory from GitHub, and python setup.py build and install. Unfortunately, it didn't fix the problem. Does swig version matters ? Yves

…

Le 5 févr. 2020 à 12:10, Gregor Thalhammer ***@***.***> a écrit : Hi Yves, actually Anaconda Python 3.7 on MacOS is my primary development platform. I just pushed my local changes, including some modifications of setup.py to github. Perhaps this fixes your segfaults. Otherwise I would recommend making a clean rebuild of gpyfft, and search for stale gpyfftlib*.so files, I remember this might cause troubles. hopes that helps Gregor — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45?email_source=notifications&email_token=AFQM3HC6RMZ7ZBSCHK4MY5TRBKNDNA5CNFSM4KITMHU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK3BLBI#issuecomment-582358405>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQM3HFLK7FBFT3QFLSQBWLRBKNDNANCNFSM4KITMHUQ>.

[geggo]

Hi Yves,

I added a zipped wheel file of my recently built gpyfft, for Anaconda Python 3.7. Try using this, install it with
pip install

gpyfft.zip

If it crashes, try to figure where. Which version of clFFT are you actually using (you could check it with

otool -L gpyfftlib-XXX.so

Gregor

[yves-surrel]

Hi Gregor Thank you for your file. I had the error: ImportError: dlopen(/Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.3-py3.7-macosx-10.9-x86_64.egg/gpyfft/gpyfftlib.cpython-37m-darwin.so, 2): Symbol not found: _clfftInitSetupData Referenced from: /Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.3-py3.7-macosx-10.9-x86_64.egg/gpyfft/gpyfftlib.cpython-37m-darwin.so Expected in: /usr/local/lib/libclFFT.2.dylib in /Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.3-py3.7-macosx-10.9-x86_64.egg/gpyfft/gpyfftlib.cpython-37m-darwin.so So I tried (once again !) to start from scratch. I uninstalled clfft (brew uninstall clfft), downloaded the cleft-master package from GitHub, then: cd /Users/yves/clFFT-master/ mkdir build cmake ../src cmake --build . --config Release And then in setup.py of gpyfft: elif 'Darwin' in system: CLFFT_DIR = r'/Users/yves/clFFT-master' CLFFT_LIB_DIR = r'/Users/yves/clFFT-master/build/library' CLFFT_INCL_DIRS = [os.path.join(CLFFT_DIR, 'src', 'include'), os.path.join(CLFFT_DIR, 'build', 'include'),] CL_INCL_DIRS = [] EXTRA_COMPILE_ARGS = ['-stdlib=libc++'] EXTRA_LINK_ARGS = ['-stdlib=libc++'] Then cd /Users/yves/clFFT-master/build/library ls -al total 1856 drwxr-xr-x 8 yves staff 256 5 fév 19:00 . drwxr-xr-x 15 yves staff 480 5 fév 18:59 .. drwxr-xr-x 5 yves staff 160 5 fév 18:59 CMakeFiles -rw-r--r-- 1 yves staff 20572 5 fév 18:59 Makefile -rw-r--r-- 1 yves staff 1826 5 fév 18:59 cmake_install.cmake -rwxr-xr-x 1 yves staff 920592 5 fév 19:00 libclFFT.2.14.0.dylib lrwxr-xr-x 1 yves staff 21 5 fév 19:00 libclFFT.2.dylib -> libclFFT.2.14.0.dylib lrwxr-xr-x 1 yves staff 16 5 fév 19:00 libclFFT.dylib -> libclFFT.2.dylib Then cd /Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.0-py3.7-macosx-10.9-x86_64.egg/gpyfft otool -L gpyfftlib.cpython-37m-darwin.so gpyfftlib.cpython-37m-darwin.so: libclFFT.2.dylib (compatibility version 2.0.0, current version 2.14.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1) Then, trying to run the simple_example.py: ImportError: dlopen(/Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.0-py3.7-macosx-10.9-x86_64.egg/gpyfft/gpyfftlib.cpython-37m-darwin.so, 2): Library not loaded: libclFFT.2.dylib Referenced from: /Users/yves/opt/anaconda3/lib/python3.7/site-packages/gpyfft-0.7.0-py3.7-macosx-10.9-x86_64.egg/gpyfft/gpyfftlib.cpython-37m-darwin.so Reason: image not found Is there something else to do to install clFFT ? Should I do brew install clFFT again ??? Sorry for the questions, but I am clearly a user, and I hate having to compile all these things, with plenty of symlinks, caches, different ways of installing and messed up configurations… Windows binaries are so much better ( Yves

…

Le 5 févr. 2020 à 16:51, Gregor Thalhammer ***@***.***> a écrit : Hi Yves, I added a zipped wheel file of my recently built gpyfft, for Anaconda Python 3.7. Try using this, install it with pip install gpyfft.zip <https://github.com/geggo/gpyfft/files/4160418/gpyfft.zip> If it crashes, try to figure where. Which version of clFFT are you actually using (you could check it with otool -L gpyfftlib-XXX.so Gregor — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#45?email_source=notifications&email_token=AFQM3HEYRF7DEINGOZCOSVTRBLN7FA5CNFSM4KITMHU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK35NEI#issuecomment-582473361>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQM3HHBEDLSQG7YNXFY7RDRBLN7FANCNFSM4KITMHUQ>.

[geggo]

Hi Yves,

it seems you have clfft not installed, the library (libclFFT.dylib) is not on your search path and cannot be found.

I typically build and install clFFT by cloning latest version from github, and then

cd src
cmake -G "Unix Makefiles"
make
sudo make install

gpyfft is just a wrapper around the clFFT library and relies that it is properly installed. I do not have the resources to provide binaries for all the major platforms

Gregor

[yves-surrel]

Hi Gregor Thank you very much for the explanation about properly installing clFFT. I was definitely lacking the right commands. However, I am back to the initial problem (seg fault), both with your wheel and with what I compile. I have set up an Anaconda environment with Python 3.6, and I think I will stick to that for now, unless you have other hints (strange that you do not encounter the same problems; maybe a package discrepancy in the virtual environment -cython?) Thanks for your patience. Yves

…

Le 6 févr. 2020 à 11:26, Gregor Thalhammer ***@***.***> a écrit : I typically build and install clFFT by cloning latest version from github, and then cd src cmake -G "Unix Makefiles" make sudo make install gpyfft is just a wrapper around the clFFT library and relies that it is properly installed. I do not have the resources to provide binaries for all the major platforms