ImportError from dlopen on mac when installed via pypi (+ workaround)

[dtdannen]

I followed the instructions on the README to install from pypi via pip (not source), and kept running into this error where from nle_language_wrapper import NLELanguageWrapper would fail to dlopen libnethack.so . More details below, but a current workaround is just to install via pip install -e . after cloning the repo. This problem may be mac specific, I've only tested on apple silicon mac os.

Here's the error output:

Python 3.9.6 (default, Oct 18 2022, 12:41:40)
[Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import gym
import nle
from nle_language_wrapper import NLELanguageWrapper
Traceback (most recent call last):
File "", line 1, in
File "...venv/lib/python3.9/site-packages/nle_language_wrapper/init.py", line 1, in
from nle_language_wrapper.wrappers.nle_language_wrapper import NLELanguageWrapper
File "...venv/lib/python3.9/site-packages/nle_language_wrapper/wrappers/nle_language_wrapper.py", line 6, in
from nle_language_wrapper.nle_language_obsv import NLELanguageObsv
ImportError: dlopen(...venv/lib/python3.9/site-packages/nle_language_wrapper/nle_language_obsv.cpython-39-darwin.so, 0x0002): Library not loaded: @rpath/libnethack.so
Referenced from: <64AAEABE-1BD0-340E-9CF9-A5D9E26EDF46> ...venv/lib/python3.9/site-packages/nle_language_wrapper/nle_language_obsv.cpython-39-darwin.so
Reason: tried: '$ORIGIN/libnethack.so' (no such file), '/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '$ORIGIN/libnethack.so' (no such file), '/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS@rpath/libnethack.so' (no such file), '$ORIGIN/libnethack.so' (no such file), '/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '$ORIGIN/libnethack.so' (no such file), '/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/91/s0z1pjg57874l6shjrv719zh0000gn/T/pip-install-3z2poamf/nle-language-wrapper_415c2906ce2b43ee8b388cee2ab551e9/nle_language_wrapper/libnethack.so' (no such file), '/usr/local/lib/libnethack.so' (no such file), '/usr/lib/libnethack.so' (no such file, not in dyld cache)

[ngoodger]

Thanks for raising the issue and sharing the workaround. Can confirm I was able to reproduce. I think I originally tested on x86 mac. I'll investigate and see if I can see what is going wrong.

[saharad1]

The same issue was encountered using Mac OS silicon, and the workaround did not work. Can you explain it in more detail or describe an alternative way?

[ngoodger]

Hi @saharad1 , sorry for the inconvenience, I'm looking into it. I'm not exactly sure what the problem is but I think when the package is installed the nle-language-wrapper library is re-compiled for the current architecture but the nle library is not. The workaround should work as it recompiles both from scratch, but you have to remember to clone the submodules as well. Steps:

1. git clone https://github.com/ngoodger/nle-language-wrapper --recursive
2. pip install -e nle-language-wrapper

[saharad1]

Thank you! worked for me as well.