Usage in Conda

[vasilygrz]

Hi Nathaniel,

My working environment in Python consists of WSL2 + VS Code + Conda and I also use Dev Containers when Docker is needed. So far, I couldn't make R work in this environment, so I've been using rocker instead.

I've created the following Conda environment, which contains the minimum amount of libraries to give iglu-py a try:

name: iglu-env
channels:
  - conda-forge
dependencies:
  # Python
  - python=3.12.11
  - jupyter=1.1.1
  - pandas=2.2.3 py312*
  - pip=25.1.1
  - xz=5.8.1
  - zlib=1.3.1
  # R
  - r-base=4.4.3
  - rpy2=3.6.1
  - pip:
    - iglu-py==1.1.1

And, not surprisingly, it doesn't work. While the Installation section says that R is not needed, it's in fact required to use rpy2. For example, check this issue on iglu where another user has created a direct Python translation of the R library.

When I run import iglu-py two things happen:

1. libR.so cannot be found: Error importing in API mode: ImportError('libR.so: cannot open shared object file: No such file or directory'). It then proceeds to Trying to import in ABI mode without success. libR.so is located in $CONDA_PREFIX/lib/R/lib/ and iglu_py is in ~/miniconda3/envs/iglu-env/lib/python3.12/site-packages/. Adding the environment variable LD_LIBRARY_PATH in the Jupyter Notebook doesn't work, I've read that it has to be done on the shell before activating the Conda environment.
2. It doesn't find the iglu_4.1.7.tar.gz file, so it starts to download all dependencies from CRAN into lib/R/ and after 10 min it crashes.

Questions

1. I'd like to ask you whether it's feasible to adapt the library to make it work in a Conda environment.
2. Is it in your interest to do so?

Comments

1. We'd have to revisit and improve the Installation section.
2. The first thing I've thought about is to adapt both __init__.py and bridge.py to create a distinction between Conda and non-Conda environments (L84-85 cannot see the difference between lib/python3.12/ and lib/R/), but I've realised that it may not be a good solution.
3. For the time being, we can update __init__.py to check for where rpy2 looks for libR.so and raise an ImportError.
4. How about automating L12 so the latest iglu version from CRAN is passed in?

Note that I haven't created a library before, so my knowledge may be limited.

[Nathaniel-Fernandes]

Hi Vasily,

Thank you for reaching out about the package! I'm coming back from some time off - let me review what you wrote, and I'll get back to you.

[Nathaniel-Fernandes]

Hi @vasilygrz,

Thank you once again for reaching out. I've had some time to look into the issue, and here are my thoughts.

As a side note, I won't be able to commit too much time to this as I've started medical school, so if you could be the main driver, that would be awesome!

Thoughts

1. You are correct that rpy2 depends on R being installed on the machine, meaning the iglu-py package also depends on R being installed by extension. This was an oversight on my part, and a PR to update the iglu-py documentation should be made that states R >= 4.0 is needed.

2. I just saw the Iglu-Python Issue you mentioned. I am not sure what was decided in that case, but I agree with @irinagain that "Direct translations risk becoming unsynchronized with the main codebase, leading to divergence and maintenance issues." Personally, I'm in favor of having R remain the single source of truth, even if it means that the plotting functionality is not available in iglu-py.

3. From the rpy2 installation requirements, it states that "Running rpy2 on Windows is currently not supported although relative success was recently reported with the lastest in the 3.3.x series." Thus, I don't believe the problem is Conda per se, but that you're using WSL/Windows. For example, I am on a Mac and was able to use VS Code + Conda (Miniconda 25.5.1) + R 4.5.1 without issue:

$ conda create --name iglu-env python=3.8
$ conda activate iglu-env
$ pip install iglu_py
$ python
> import iglu_py as iglu
> print(iglu.mage(iglu.example_data_5_subject)) # yields correct output

• A. Could you check if your libR.so cannot be found error is raised from the rpy2 package (and potentially not from iglu-py)? That would inform the next steps.

• B. Could you go through the various versions of Python and see if the import breaks after version Python 3.8? Maybe later versions of Python present a problem.

One clarification

The iglu_4.1.7.tar.gz file only contains the iglu package, so the code would still need to "download all dependencies from CRAN into lib/R/" if these R dependencies are not already installed on the user's machine.

It is still strange, though, that your system crashes after 10 minutes. I'm not sure why that could be -- on my computer, it downloads all R dependencies in about 20 seconds to 1 minute from a fresh install.

On your comments

• On Comment #2: I'm not sure line 84-85 is an issue. By default, this line is trying to install the iglu_4.1.7.tar.gz package that comes bundled with iglu-py, which should be located in lib/python3.X/ not in lib/R/. If the user tries to pass in a relative file path, they'll just have to keep in mind the starting directory is in lib/python3.X/. Either way, I'm not sure the current behavior is incorrect, although we could clarify this better in the documentation.

• On Comment #3: I agree there should be better error handling to see if rpy2 fails to install libR.so. If this import fails, the rest of the code wouldn't work as ryp2 isn't loaded properly, so an error should be thrown.

• On Comment #4, automation would be helpful and would prevent iglu-py from lagging behind iglu-r! How were you thinking about automating it?