Issue with pip install apscale-blast

[onurdogan]

Issue with pip install apscale-blast

Hey Till,

When I attempted to run the installation commands for apscale-blast 1.0.1, I encountered an issue with the pip install apscale-blast command.

After checking the installation directory, I noticed that the package was installed under the name apscale_blast (with an underscore, not a hyphen).

The correct installation command for me was:

pip install apscale_blast

or

pip3 install apscale_blast

and also starting command for was:

apscale_blast blastn

[TillMacher]

Can you try the latest version? I updated to 1.0.2 this morning. For me I can install apscale blast both with an underscore and a hyphen. I don't know why PyPi keeps showing the "old" install command with a hyphen though.

[onurdogan]

Hi again,

When I attempt to upgrade apscale-blast using possible methods, I encounter following errors that prevent the update from completing (e.g., pip, pip3, etc.), but nothing seems to work.

Btw, the current version works fine, but when I try to build a custom database from a reference .fasta file, the resulting database file is created as 0 KB. However, when I use the files that you’ve shared in server, it works perfectly.

After creating the subsets, I encountered the openpyxl error once again and had to manually merge the subsets. Following this, I removed any OTU duplicates and manually verified the final .xlsx file, ensuring that the number of OTUs matched the OTU count in the read table, to successfully create taXon table for TTT.

To be honest, I'm having difficulty understanding the difference between the apscale_gui > local BLAST module and the apscale-blast program you developed. From what I can tell, they seem to serve the same purpose. Could you clarify the distinction between the two and provide guidance on how to set up a database from a reference FASTA file?

onurdogan@Onurs-iMac ~ % pip install --upgrade apscale-blast
zsh: /usr/local/bin/pip: bad interpreter: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: no such file or directory
ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale-blast (from versions: none)
ERROR: No matching distribution found for apscale-blast

onurdogan@Onurs-iMac ~ % pip install --upgrade apscale_blast
zsh: /usr/local/bin/pip: bad interpreter: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: no such file or directory
ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale_blast (from versions: none)
ERROR: No matching distribution found for apscale_blast

onurdogan@Onurs-iMac ~ % pip3 install --upgrade apscale_blast
ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale_blast (from versions: none)
ERROR: No matching distribution found for apscale_blast

onurdogan@Onurs-iMac ~ % pip install openpyxl --upgrade
zsh: /usr/local/bin/pip: bad interpreter: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: no such file or directory
Requirement already satisfied: openpyxl in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (3.0.10)
Collecting openpyxl
  Downloading openpyxl-3.1.5-py2.py3-none-any.whl.metadata (2.5 kB)
Requirement already satisfied: et-xmlfile in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from openpyxl) (1.1.0)
Downloading openpyxl-3.1.5-py2.py3-none-any.whl (250 kB)
Installing collected packages: openpyxl
  Attempting uninstall: openpyxl
    Found existing installation: openpyxl 3.0.10
    Uninstalling openpyxl-3.0.10:
      Successfully uninstalled openpyxl-3.0.10
Successfully installed openpyxl-3.1.5
onurdogan@Onurs-iMac ~ % apscale_blast filter         

    APSCALE blast command line tool - v1.0.1 - 06/09/2024
    Usage examples:
    $ apscale_blast blastn -h
    $ apscale_blast blastn -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -query_fasta ./12S_apscale_ESVs.fasta
    $ apscale_blast filter -h
    $ apscale_blast filter -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -blastn_folder ./12S_apscale_ESVs_blastn
    
Please enter PATH to database: /Users/onurdogan/0_APSCALE_projects/APSCALE_databases/pr2    
Please enter PATH to blastn folder: /Users/onurdogan/0_APSCALE_projects/Plankton_18S_time_series_apscale/9_lulu_filtering/otu_clustering/Plankton_18S_time_series_apscale_OTUs_filtered
20:56:11: Starting to filter blast results for '/Users/onurdogan/0_APSCALE_projects/Plankton_18S_time_series_apscale/9_lulu_filtering/otu_clustering/Plankton_18S_time_series_apscale_OTUs_filtered'
20:56:11: Your database: pr2
20:56:16: Finished filtering for subset 4/18.
20:56:16: Finished filtering for subset 11/18.
20:56:16: Finished filtering for subset 7/18.
20:56:16: Finished filtering for subset 10/18.
20:56:16: Finished filtering for subset 1/18.
20:56:16: Finished filtering for subset 9/18.
20:56:16: Finished filtering for subset 5/18.
20:56:16: Finished filtering for subset 6/18.
20:56:16: Finished filtering for subset 3/18.
20:56:16: Finished filtering for subset 8/18.
20:56:16: Finished filtering for subset 2/18.
20:56:19: Finished filtering for subset 13/18.
20:56:19: Finished filtering for subset 12/18.
20:56:19: Finished filtering for subset 16/18.
20:56:19: Finished filtering for subset 18/18.
20:56:19: Finished filtering for subset 14/18.
20:56:19: Finished filtering for subset 15/18.
20:56:19: Finished filtering for subset 17/18.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pandas/compat/_optional.py", line 135, in import_optional_dependency
    module = importlib.import_module(name)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'openpyxl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/apscale_blast", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/apscale_blast/__main__.py", line 641, in main
    blastn_filter(args.blastn_folder, args.database, thresholds, args.n_cores)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/apscale_blast/__main__.py", line 518, in blastn_filter
    df = pd.read_excel(file).fillna('')
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 495, in read_excel
    io = ExcelFile(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 1567, in __init__
    self._reader = self._engines[engine](
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pandas/io/excel/_openpyxl.py", line 552, in __init__
    import_optional_dependency("openpyxl")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pandas/compat/_optional.py", line 138, in import_optional_dependency
    raise ImportError(msg)
ImportError: Missing optional dependency 'openpyxl'.  Use pip or conda to install openpyxl.

and

onurdogan@Onurs-iMac ~ % pip3 install openpyxl==3.1.5
Collecting openpyxl==3.1.5
  Using cached openpyxl-3.1.5-py2.py3-none-any.whl.metadata (2.5 kB)
Requirement already satisfied: et-xmlfile in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from openpyxl==3.1.5) (1.1.0)
Using cached openpyxl-3.1.5-py2.py3-none-any.whl (250 kB)
Installing collected packages: openpyxl
  Attempting uninstall: openpyxl
    Found existing installation: openpyxl 3.1.0
    Uninstalling openpyxl-3.1.0:
      Successfully uninstalled openpyxl-3.1.0
Successfully installed openpyxl-3.1.5
onurdogan@Onurs-iMac ~ % apscale_blast blastn         

    APSCALE blast command line tool - v1.0.1 - 06/09/2024
    Usage examples:
    $ apscale_blast blastn -h
    $ apscale_blast blastn -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -query_fasta ./12S_apscale_ESVs.fasta
    $ apscale_blast filter -h
    $ apscale_blast filter -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -blastn_folder ./12S_apscale_ESVs_blastn
    
Please enter PATH to database: /Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023
Please enter PATH to query fasta: /Users/onurdogan/0_APSCALE_projects/eDNA_data_analysis_challange_apscale/9_lulu_filtering/otu_clustering/eDNA_data_analysis_challange_apscale_OTUs_filtered.fasta
/Users/onurdogan/0_APSCALE_projects/eDNA_data_analysis_challange_apscale/9_lulu_filtering/otu_clustering/eDNA_data_analysis_challange_apscale_OTUs_filtered.fasta
09:32:02 : Creating subset(s) from fasta file.
09:32:02 : Created 8 subset(s) from fasta file.
09:32:02: Starting blastn for 'eDNA_data_analysis_challange_apscale_OTUs_filtered'
09:32:02: Your database: Reference_library_COI_AllFish_Dez2023
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
BLAST Database error: No alias or index file found for nucleotide database [/Users/onurdogan/0_APSCALE_projects/APSCALE_databases/Reference_library_COI_AllFish_Dez2023/db] in search path [/Users/onurdogan::]
09:32:04: Finished blastn for subset 8/8.
09:32:04: Finished blastn for subset 4/8.
09:32:04: Finished blastn for subset 3/8.
09:32:04: Finished blastn for subset 6/8.
09:32:04: Finished blastn for subset 1/8.
09:32:04: Finished blastn for subset 2/8.
09:32:04: Finished blastn for subset 7/8.
09:32:04: Finished blastn for subset 5/8.
09:32:04: Finished blastn for 'eDNA_data_analysis_challange_apscale_OTUs_filtered'

[TillMacher]

Apscale-blast is basically an upgrade of the old "local blast" function from the GUI-version. But since PySimpleGUI is now not freely available anymore, I had to write a version without GUI (for now). Also, I made some updates so that the algorithm is much faster (through parallelisation) and also more stable.

Did you use the scripts that I uploaded here? The old databases do not work anymore. The taxonomy is now written in a more sophisticated format, which was not implemented in the "local blast" function. So please create new databases using these scripts. If you have troubles with the creation of custom databases, let me know. For now I will only sustain the official databases, but we can always work out individual databases :)

Apscale-blast is currently only supported for python 3.10 or higher. I see that you use 3.9, which causes some errors, probably also related to openpyxl.

[onurdogan]

Thank you for the clarification and the great work! I realized that I should have opened this issue on the apscale-blast GitHub page. I’ll go ahead and create the issue there with the same title and share the link here once it's posted.

I successfully tested the PR2 database you shared with apscale-blast, and it worked well. I also installed Python 3.10 but am encountering some issues switching between Python versions. After downloading apscale-blast in Python 3.9, I moved (copied and pasted) it to the Python 3.10 folder, which resolved the issue for that module.

Additionally, I'm having trouble figuring out how to activate the correct Python version I'm currently using, and I often struggle with this. I attempted to modify my .zshrc profile file as follows, but I couldn't apply the same solution for openpyxl. Maybe there is a point I missed that I can't see.

nano $HOME/.zshrc

# Setting PATH for Python 3.9
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.9/bin:${PATH}"
export PATH

# Setting PATH for Python 3.10
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.10/bin:${PATH}"
export PATH

# Setting PATH for Python 3.11
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.11/bin:${PATH}"
export PATH

#PATH for TTT
export PATH="/usr/local/bin:$PATH"
export PATH

Then, I ran the following commands:

source $HOME/.zshrc
source .zshrc

Any professional advice on resolving issues with switching between Python versions or managing the correct versions of cutadapt, pandas, and openpyxl etc. would be greatly appreciated. I typically copy and paste the downloaded package into the directory of the other Python version. At the end of the day, I somehow make it work but most of the time it happens without my understanding how.

For example, the apscale_blast blastn command works, but the upgrade command does not.

onurdogan@Onurs-iMac ~ % nano $HOME/.zshrc

onurdogan@Onurs-iMac ~ % pip install --upgrade apscale_blast
zsh: /usr/local/bin/pip: bad interpreter: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: no such file or directory

ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale_blast (from versions: none)
ERROR: No matching distribution found for apscale_blast

onurdogan@Onurs-iMac ~ % pip install --upgrade apscale_blast
zsh: /usr/local/bin/pip: bad interpreter: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: no such file or directory

ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale_blast (from versions: none)
ERROR: No matching distribution found for apscale_blast

onurdogan@Onurs-iMac ~ % pip3 install --upgrade apscale_blast
ERROR: Ignored the following versions that require a different python version: 0.1 Requires-Python >=3.10; 0.1.1 Requires-Python >=3.10; 0.1.2 Requires-Python >=3.10; 0.1.3 Requires-Python >=3.10; 0.1.4 Requires-Python >=3.10; 0.1.5 Requires-Python >=3.10; 1.0.0 Requires-Python >=3.10; 1.0.1 Requires-Python >=3.10; 1.0.2 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement apscale_blast (from versions: none)
ERROR: No matching distribution found for apscale_blast


onurdogan@Onurs-iMac ~ % apscale_blast blastn

    APSCALE blast command line tool - v1.0.1 - 06/09/2024
    Usage examples:
    $ apscale_blast blastn -h
    $ apscale_blast blastn -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -query_fasta ./12S_apscale_ESVs.fasta
    $ apscale_blast filter -h
    $ apscale_blast filter -database ./MIDORI2_UNIQ_NUC_GB259_srRNA_BLAST -blastn_folder ./12S_apscale_ESVs_blastn
    
Please enter PATH to database: