GPU device argument

[Barabaika]

Hello,

Thank you for developing an excellent tool!

In the configure_device function within configuration.py, the GPU selection is currently limited to device = torch.device("cuda") without the ability to specify a particular GPU index. It would be more convenient to add an argument for users to define the GPU index explicitly (e.g., cuda:0, cuda:1, etc.). Are there any arguments against implementing this?

Additional Question:
Regarding GPU mode in the inference pipeline: does the pipeline utilize multiple CPUs in GPU mode? The line print(f"Using device {str(device).upper()} with {ncpu} CPUs") (line 33 in configure_device) suggests multi-CPU usage, but I couldn't find evidence of this in the code. If multi-CPU usage doesn't occur in GPU mode, this print statement might be slightly misleading.

Thank you in advance

[ALGW71]

Hi @Barabaika - apologies for the delay, busy year.

Cuda:0, cuda:1

• We didn't consider this at all, but now that you have raised it I will set up a PR.
• Cannot think of any reason against this, other than an additional parameter, as long as the default remains "cuda".

CPUs in GPU mode:

• As far as I am aware from my local tests, the use of set_num_threads(ncpu) is relevant in GPU mode, as it controls how many threads are accessible for preprocessing operations.
• In the configure_cpus function the default is max number of available cpus.
• If users wanted to reduce this I cannot see very large reductions in speed, even when working with ~100K sequences. However I have not played a lot with this, so it may be worth some tests if you are really trying to get the most out of your hardware.

Thanks again for you kind words and I will flag the PR on this thread!

Best,
Alex