Skip to content

can’t create hard link #451

Open
Open
can’t create hard link#451
@llm98m

Description

@llm98m
llm98m
opened on Nov 11, 2024

Describe the bug

Hello,
I am seeking assistance with a problem I've encountered while utilizing a pipeline for analysis with a fastaq file. Once the file has been trimmed, the next step involves creating a hard link. Unfortunately, my server does not currently support the creation of hard links. Is there a method to circumvent this step, or could you guide me on how to alter the script so that it copies the file instead of creating a hard link? Enclosed is the specific error message I received when running the pipeline. I am eagerly awaiting your response.

OS/Platform

  • OS/Platform:
    LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
    Distributor ID: CentOS
    Description: CentOS Linux release 7.9.2009 (Core)
    Release: 7.9.2009
    Codename: Core
  • Conda version: conda 23.1.0
  • Pipeline version: [e.g. v1.8.0]
  • Caper version: caper --version 2.3.2

Caper configuration file

backend=local
local-loc-dir=/storage/public/home/lmli/software/Encode/ATAC
cromwell=/root/.caper/cromwell_jar/cromwell-82.jar
womtool=/root/.caper/womtool_jar/womtool-82.jar

Input JSON file

{
"atac.title" : "test (paired end)",
"atac.description" : "This is a template input JSON for paired ended sample.",

"atac.pipeline_type" : "FSH-OPU-16C-ATAC-34",
"atac.align_only" : false,
"atac.true_rep_only" : false,

"atac.genome_tsv" : "/storage/public/home/lmli/software/Encode/ATAC/genomes/bostau9/bostau9.tsv",

"atac.paired_end" : true,
"atac.fastqs_rep1_R1" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-3/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.fastq.gz"],
"atac.fastqs_rep1_R2" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-3/FSH-OPU-16C-ATAC-3_S1_L003_R2_001.fastq.gz"],
"atac.fastqs_rep2_R1" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.fastq.gz"],
"atac.fastqs_rep2_R2" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-4/FSH-OPU-16C-ATAC-4_S47_L002_R2_001.fastq.gz"],

"atac.auto_detect_adapter" : true,

"atac.multimapping" : 4

}

Troubleshooting result

==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=7438
START=2024-11-09T07:37:17.498Z, END=2024-11-09T15:39:33.298Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/non_mito/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=8941
START=2024-11-09T07:37:11.963Z, END=2024-11-09T23:15:35.005Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/non_mito/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=8455
START=2024-11-09T07:37:11.961Z, END=2024-11-10T01:01:13.459Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/non_mito/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=9115
START=2024-11-09T07:37:11.959Z, END=2024-11-10T02:49:55.187Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/non_mito/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc': Operation not permitted

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions