Allocation of 754647040 exceeds 10% of free system memory. #22
Description
Hello！ @swallow-design @GCS-ZHN
I have tried several times but each time I only get a result of 8001 sequences, but I submit 16000 sequences. Is there something wrong with my parameter settings, here is my code.
python -m annopro -i lh_over_30bp.fa -o lh-out --overwrite
Here is my log file
diamond v2.1.0.154 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)
#CPU threads: 4
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: lh-out
#Target sequences to report alignments for: 25
Opening the database... [0.053s]
Database: /home/liuli/.annopro/data/cafa4.dmnd (type: Diamond database, sequences: 87514, letters: 44798577)
Block size = 2000000000
Opening the input file... [0.01s]
Opening the output file... [0s]
Loading query sequences... [0.035s]
Masking queries... [0.381s]
Algorithm: Double-indexed
Building query histograms... [0.261s]
Loading reference sequences... [0.064s]
Masking reference... [2.06s]
Initializing temporary storage... [0.025s]
Building reference histograms... [1.422s]
Allocating buffers... [0s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 1/4.
Building reference seed array... [0.503s]
Building query seed array... [0.096s]
Computing hash join... [0.068s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.087s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 2/4.
Building reference seed array... [0.53s]
Building query seed array... [0.099s]
Computing hash join... [0.067s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.074s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 3/4.
Building reference seed array... [0.548s]
Building query seed array... [0.103s]
Computing hash join... [0.068s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.069s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 4/4.
Building reference seed array... [0.485s]
Building query seed array... [0.091s]
Computing hash join... [0.068s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.066s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 1/4.
Building reference seed array... [0.484s]
Building query seed array... [0.091s]
Computing hash join... [0.067s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.068s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 2/4.
Building reference seed array... [0.526s]
Building query seed array... [0.099s]
Computing hash join... [0.067s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.061s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 3/4.
Building reference seed array... [0.544s]
Building query seed array... [0.102s]
Computing hash join... [0.067s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.06s]
Deallocating memory... [0s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 4/4.
Building reference seed array... [0.481s]
Building query seed array... [0.09s]
Computing hash join... [0.066s]
Masking low complexity seeds... [0.006s]
Searching alignments... [0.058s]
Deallocating memory... [0s]
Deallocating buffers... [0.009s]
Clearing query masking... [0s]
Computing alignments... Loading trace points... [0.035s]
Sorting trace points... [0.012s]
Computing alignments... [10.428s]
Deallocating buffers... [0s]
Loading trace points... [0s]
[10.48s]
Deallocating reference... [0.003s]
Loading reference sequences... [0s]
Deallocating buffers... [0s]
Deallocating queries... [0s]
Loading query sequences... [0s]
Closing the input file... [0s]
Closing the output file... [0.016s]
Closing the database... [0.003s]
Cleaning up... [0s]
Total time = 20.936s
Reported 88062 pairwise alignments, 88062 HSPs.
8348 queries aligned.
2024-07-09 10:37:31.644732: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2024-07-09 10:37:31.654184: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: gpu03
2024-07-09 10:37:31.654200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: gpu03
2024-07-09 10:37:31.654323: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 550.54.15
2024-07-09 10:37:31.654390: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 550.54.15
2024-07-09 10:37:31.654402: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 550.54.15
2024-07-09 10:37:31.669962: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-07-09 10:37:31.808310: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 754647040 exceeds 10% of free system memory.
2024-07-09 10:37:32.854598: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 754647040 exceeds 10% of free system memory.
2024-07-09 10:37:33.089177: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 754647040 exceeds 10% of free system memory.
2024-07-09 10:37:34.509640: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 754647040 exceeds 10% of free system memory.
2024-07-09 10:37:36.304317: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
2024-07-09 10:48:23.850180: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 754647040 exceeds 10% of free system memory.Any help would be appreciated！