Search does not report when BLAST fails

[aringeri]

Hi,
I've been troubleshooting an issue I've encountered with dnabarcoder's search function when attempting to classify a relatively large number of full ITS sequences (~6000) against the UNITE-INSD 2024 database.
I've been using dnabarcoder v1.06.

What I've found is that the BLAST command runs out of memory and dnabarcoder continues as though BLAST command was successful.
The blastoutput file is empty or incomplete and the search.py command returns a .bestmatch file reporting BLAST scores of 0.

I think there are two issues at play here:

• dnabarcoder should detect whether the BLAST command fails
• the number of cpus (nproc) should be configurable. Currently this uses all threads on my system (on my cluster: 124 threads) and I believe is the source the excessive memory use.

I've manually edited the code on my system to use fewer threads and this temporarily alleviates memory the issue for me.

The first issue is a little trickier to deal with as dnabarcoder might need to check return codes from the BLAST process.
This might be better suited using python's subprocess module (https://docs.python.org/3/library/subprocess.html#), but just a suggestion.

Thank you!!

[vuthuyduong]

Hi Alex, Thank you very much for your troubleshooting and suggestions. They are greatly appreciated! I have added the option -ncpus to most of the scripts in dnabarcoder, allowing you to specify how many CPUs you would like to use for BLAST. Please use the latest version of dnabarcoder by running: git clone https://github.com/vuthuyduong/dnabarcoder.git To detect whether the BLAST command fails in search.py, I have added some code (lines 131-133) to check if BLAST is working properly and to give warnings to users. I hope this helps. Thank you again, Cheers Duong

…

On Tue, 15 Oct 2024 at 03:04, Alex Ringeri ***@***.***> wrote: Hi, I've been troubleshooting an issue I've encountered with dnabarcoder's search function when attempting to classify a relatively large number of full ITS sequences (~6000) against the UNITE-INSD 2024 database. I've been using dnabarcoder v1.06. What I've found is that the BLAST command runs out of memory and dnabarcoder continues as though BLAST command was successful. The blastoutput file is empty or incomplete and the search.py command returns a .bestmatch file reporting BLAST scores of 0. I think there are two issues at play here: - dnabarcoder should detect whether the BLAST command fails - the number of cpus (nproc) should be configurable. Currently this uses all threads on my system (on my cluster: 124 threads) and I believe is the source the excessive memory use. I've manually edited the code on my system to use fewer threads and this temporarily alleviates memory the issue for me. The first issue is a little trickier to deal with as dnabarcoder might need to check return codes from the BLAST process. This might be better suited using python's subprocess module ( https://docs.python.org/3/library/subprocess.html#), but just a suggestion. Thank you!! — Reply to this email directly, view it on GitHub <#4>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF6CZMWXAAVRRLV5DCS3DYDZ3RS2PAVCNFSM6AAAAABP6ASGLWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGU4DOMZXHE2TMMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[aringeri]

This is great! Thank you for you rapid response!