VRAM Requirements

[mthorwald]

What GPU do you recommend? I'm getting OOM for a ~950nt sequence with amber on a 5090 and 4090. Smaller sequences do work properly. I do have limited access to A40, A100

[mthorwald]

I just wanted to put this out there if anyone is curious: 950nt sequence works with an A100 80gb, but fails on L40s, A100 40gb, and A40 with out of memory errors. The failure happens at the inference stage even with reducing reserve memory with the suggested command. A small RNA (72nt) works but 700nt fails with a RTX 4090 and 5090. I've tried incorporating autocast to reduce memory which failed.

[mthorwald]

Final update: I now get this error while using an a100 which does not occur when folding the smaller RNA molecules.
[thorwald@discovery1 logs]$ cd /scratch1/thorwald/RhoFold/logs/ rhofold_apoe_a100_728080.err
-bash: cd: too many arguments
[thorwald@discovery1 logs]$ cd /scratch1/thorwald/RhoFold/logs/rhofold_apoe_a100_728080.err
-bash: cd: /scratch1/thorwald/RhoFold/logs/rhofold_apoe_a100_728080.err: Not a directory
[thorwald@discovery1 logs]$ cat rhofold_apoe_a100_728080.err
Warning: importing 'simtk.openmm' is deprecated. Import 'openmm' instead.
/scratch1/thorwald/RhoFold/rhofold/utils/rigid_utils.py:35: UserWarning: Using torch.cross without specifying the dim arg is deprecated.
Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ../aten/src/ATen/native/Cross.cpp:62.)
e3 = torch.cross(e1, e2)
Fetching 4 files: 100%|██████████| 4/4 [00:00<00:00, 36.86it/s]
Traceback (most recent call last):
File "/scratch1/thorwald/RhoFold/inference.py", line 145, in
main(args)
File "/home1/thorwald/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/scratch1/thorwald/RhoFold/inference.py", line 123, in main
amber_relax.process(unrelaxed_model, relaxed_model)
File "/scratch1/thorwald/RhoFold/rhofold/relax/relax.py", line 64, in process
self._run_amber_relax(pdbin_tmp, pdbout_tmp)
File "/scratch1/thorwald/RhoFold/rhofold/relax/relax.py", line 80, in _run_amber_relax
modeller.addHydrogens(forcefield)
File "/home1/thorwald/.conda/envs/rhofold/lib/python3.10/site-packages/openmm/app/modeller.py", line 944, in addHydrogens
delta = 0.1nanometer/norm(delta)
File "/home1/thorwald/.conda/envs/rhofold/lib/python3.10/site-packages/openmm/unit/quantity.py", line 408, in truediv
return (self/other._value) / other.unit
File "/home1/thorwald/.conda/envs/rhofold/lib/python3.10/site-packages/openmm/unit/quantity.py", line 411, in truediv
return self * pow(other, -1.0)
ZeroDivisionError: 0.0 cannot be raised to a negative power
[thorwald@discovery1 logs]$

[WangJiuming]

Hi @mthorwald,

Thanks for reporting this issue.

The model was primarily trained on structures up to 256 nt, so a sequence of around 950 nt is likely outside its optimal range.

In terms of hardware limits, sequences under 1000 nt should generally be runnable on an A100 (80 GB) GPU, while a 3090 (24 GB) GPU typically supports sequences up to around 500 nt.

Could you please share the RNA sequence that triggered the OOM error (as mentioned here)? This would help us reproduce and diagnose the issue more precisely.

[mthorwald]

@WangJiuming thanks for the response. The 4090/5090 give OOM errors when trying to fold human GPx4 (~700) and ApoE (~900+). Humanin (MDP) was the 72nt sequence that I folded on the 4090/5090/A100 successfully. Moving to the A100 on the HPC I no longer got the OOM error but the one above at the relaxation stage.

[WangJiuming]

Hi @mthorwald,

Thanks for the feedback.

I have tested the humanin sequence (72 nt) from https://www.ncbi.nlm.nih.gov/nuccore/AY029066.

In single-sequence mode, the relaxation step is expected to fail because the unrelaxed structure is significantly distorted. The predicted pLDDT score was 0.5029, suggesting low model confidence in the initial prediction.

When run with a searched MSA, the pLDDT score improved to 0.7467, and relaxation completed successfully.

I don't have access to an A100 within HPC at present, but I could run inference for the MSA mode on a 4090 GPU (24G) successfully without any error (including relaxation).

You can find more details about MSA search and pLDDT parsing in our complete protocol here: https://github.com/WangJiuming/rhofold_protocol/.

[mthorwald]

@WangJiuming Yes, I too was able to fold the Humanin on all of my cards in single sequence and MSA modes. I've tried folding SOD1 (465nt) on the RhoFold+ server with and without MSA - Both failed.

[WangJiuming]

Hi @mthorwald,

This is related to the issue reported in #7.

I was able to reproduce this issue using the coding sequence from https://www.ncbi.nlm.nih.gov/nuccore/AY438328.

The problem occurs because the sequence lacks an effective MSA, which leads to a highly distorted raw prediction that cannot be recovered by simple relaxation. As a result, the relaxation step takes an unexpectedly long time to complete. This is why the server failed in both modes. If you have a local deployment, you can try setting the relaxation steps to be 0 and check the raw predicted structure, which should verify it.