Dear FANTASIA-Team,
I tried to run FANTASIA in a Nextflow workflow that sometimes starts multiple FANTASIA processes at once and therefore they run in parallel.
When running multiple FANTASIA runs in parallel, inside a singularity container (I pulled the available docker://frapercan/fantasia:latest), the following error occurred:
2025-11-25 19:22:52,134 - EmbeddingLookUp - ERROR - process() failed: [Errno 2] Unable to synchronously open file (unable to open file: name = '/home/nas-hs/projs/fantasia-tiberius/Nextflow_Tiberius_FANTASIA/work/57/a303cea81c718ff89b195ad691c637/FANTASIA/fantasia/experiments/sample_20251125191923/embeddings.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
Traceback (most recent call last):
File "/app/fantasia/src/lookup.py", line 672, in process
with h5py.File(h5_path, "r") as h5:
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/h5py/_hl/files.py", line 566, in __init__
fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/h5py/_hl/files.py", line 241, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 104, in h5py.h5f.open
FileNotFoundError: [Errno 2] Unable to synchronously open file (unable to open file: name = '/home/nas-hs/projs/fantasia-tiberius/Nextflow_Tiberius_FANTASIA/work/57/a303cea81c718ff89b195ad691c637/FANTASIA/fantasia/experiments/sample_20251125191923/embeddings.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
This is due to the hard coded path to the WORKDIR:
That problem should be solved by adding a redefinition of the WORKDIR before running:
export WORKDIR=\$RUN_DIR
singularity exec --nv \\
--bind \$RUN_DIR:\$RUN_DIR \\
--bind \$PWD/fantasia:\$PWD/fantasia \\
--bind \$PWD:\$PWD \\
\$FT_IMG fantasia initialize --config \$CFG
singularity exec --nv \\
--bind \$RUN_DIR:\$RUN_DIR \\
--bind \$PWD/fantasia:\$PWD/fantasia \\
--bind \$PWD:\$PWD \\
\$FT_IMG fantasia run --config \$CFG
Even though I found a solution for this problem, I think it could be really helpful to change the default WORKDIR to a relative path. Thank you!
Dear FANTASIA-Team,
I tried to run FANTASIA in a Nextflow workflow that sometimes starts multiple FANTASIA processes at once and therefore they run in parallel.
When running multiple FANTASIA runs in parallel, inside a singularity container (I pulled the available
docker://frapercan/fantasia:latest), the following error occurred:This is due to the hard coded path to the WORKDIR:
FANTASIA/Dockerfile
Line 29 in 16c094d
That problem should be solved by adding a redefinition of the WORKDIR before running:
Even though I found a solution for this problem, I think it could be really helpful to change the default WORKDIR to a relative path. Thank you!