It runs without error if I run on .fastq. However, if I run with .fastq.gz, I have the following error.
bash-4.1$ snakemake --directory $HOME/software/miR-MaGiC/pipeline/ --snakefile $HOME/software/miR-MaGiC/pipeline/Snakefile --config outdir=$PWD fastq=$PWD/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz mirna=$HOME/references/mirbase21/mature_mmu.fa mirna_gp=$HOME/software/miR-MaGiC/resources/group_tables/miRBase_v21_group_by_core_ID_Mus_musculus.txt jar=$HOME/software/miR-MaGiC/pipeline/ k=20 plus_strand_only=True
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 all
1 collapse_counts_by_families
1 index_bam
1 kmer_match_to_reference
1 sort_bam
5
rule kmer_match_to_reference:
input: /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa
output: /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam
jobid: 4
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Creating index for target fasta /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Done creating index. Minimum k is 15. Maximum k is 20.
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Writing matches for reads in /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz to /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam...
Exception thrown while reading fastq file
Exception in thread "main" java.lang.NullPointerException
at mirmagic.PerfectKmerSearch.samRecordFirstKmerMatchEachTarget(PerfectKmerSearch.java:271)
at mirmagic.PerfectKmerSearch.writeFirstKmerMatchEachTarget(PerfectKmerSearch.java:320)
at mirmagic.PerfectKmerSearch.main(PerfectKmerSearch.java:713)
Error in job kmer_match_to_reference while creating output file /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam.
RuleException:
CalledProcessError in line 81 of /rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline/Snakefile:
Command 'java -jar /rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline//PerfectKmerSearch.jar -b /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam -fa /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa -fq /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz -k 20' returned non-zero exit status 1
File "/rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline/Snakefile", line 81, in __rule_kmer_match_to_reference
File "/risapps/rhel6/python/3.5/anaconda3/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job kmer_match_to_reference since they might be corrupted:
/rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam
Will exit after finishing currently running jobs.
Exiting because a job execution failed. Look above for error message
It runs without error if I run on .fastq. However, if I run with .fastq.gz, I have the following error.
bash-4.1$ snakemake --directory $HOME/software/miR-MaGiC/pipeline/ --snakefile $HOME/software/miR-MaGiC/pipeline/Snakefile --config outdir=$PWD fastq=$PWD/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz mirna=$HOME/references/mirbase21/mature_mmu.fa mirna_gp=$HOME/software/miR-MaGiC/resources/group_tables/miRBase_v21_group_by_core_ID_Mus_musculus.txt jar=$HOME/software/miR-MaGiC/pipeline/ k=20 plus_strand_only=True
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 all
1 collapse_counts_by_families
1 index_bam
1 kmer_match_to_reference
1 sort_bam
5
rule kmer_match_to_reference:
input: /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa
output: /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam
jobid: 4
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Creating index for target fasta /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Done creating index. Minimum k is 15. Maximum k is 20.
Reading sequences from fasta file /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa...
Got 1915 sequences.
Writing matches for reads in /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz to /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam...
Exception thrown while reading fastq file
Exception in thread "main" java.lang.NullPointerException
at mirmagic.PerfectKmerSearch.samRecordFirstKmerMatchEachTarget(PerfectKmerSearch.java:271)
at mirmagic.PerfectKmerSearch.writeFirstKmerMatchEachTarget(PerfectKmerSearch.java:320)
at mirmagic.PerfectKmerSearch.main(PerfectKmerSearch.java:713)
Error in job kmer_match_to_reference while creating output file /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam.
RuleException:
CalledProcessError in line 81 of /rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline/Snakefile:
Command 'java -jar /rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline//PerfectKmerSearch.jar -b /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam -fa /rsrch2/ccp_rsch/kchang3/references/mirbase21/mature_mmu.fa -fq /rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/source_data/Sample_smR28-11-R3/smR28-11-R3_S18_L002_R1_001.fastq.gz -k 20' returned non-zero exit status 1
File "/rsrch2/ccp_rsch/kchang3/software/miR-MaGiC/pipeline/Snakefile", line 81, in __rule_kmer_match_to_reference
File "/risapps/rhel6/python/3.5/anaconda3/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job kmer_match_to_reference since they might be corrupted:
/rsrch2/ccp_rsch/kchang3/naproxen_lynch/mirnaseq-mm10/kmer.smR28-11-R3_S18_L002_R1_001.gz.bam
Will exit after finishing currently running jobs.
Exiting because a job execution failed. Look above for error message