Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote checks on dependent steps not working #1502

Open
pgcudahy opened this issue Aug 16, 2022 · 1 comment
Open

Remote checks on dependent steps not working #1502

pgcudahy opened this issue Aug 16, 2022 · 1 comment
Assignees
Labels

Comments

@pgcudahy
Copy link
Contributor

Hello Bo, thanks again for providing this excellent tool. I have a workflow where I build a custom database for kraken and then have a subsequent step that uses kraken and the database on some samples with a dependency on the database build step. However, when I change the database build step, the subsequent steps do not re-run. Here's an example.

%sosrun depends_test -q yale_hpc_task_spooler \
-r yale_hpc_task_spooler mem="1G" cores=1 walltime="00:05:00" nodes=1

[touch_files]
output: f'/home/pgc29/scratch60/test.txt'
task: walltime='00:01:00', mem='1G', cores=1, nodes=1,
    workdir='/home/pgc29/scratch60/'

run: expand=True
    touch {_output}
    
[depends_test]
depends: sos_step('touch_files')
output: f'/home/pgc29/data/test2.txt'
task: walltime='00:01:00', mem='1G', cores=1, nodes=1,
    workdir='/home/pgc29/scratch60/'

run: expand=True
    touch {_output}

But if I then modify the first step to

[touch_files]
output: f'/home/pgc29/scratch60//test.txt'
task: walltime='00:01:00', mem='1G', cores=1, nodes=1,
    workdir='/home/pgc29/scratch60/'

run: expand=True
    echo "test" > {_output}

And run depends_test again, it re-runs touch_files but skips depends_test

INFO: Running touch_files: 
INFO: tce6f8d6f7553ab25 submitted to yale_hpc_task_spooler with job id 30325020
INFO: Waiting for the completion of 1 task.
INFO: tce6f8d6f7553ab25 received '/home/pgc29/scratch60/test.txt' from yale_hpc_task_spooler
INFO: touch_files output:   /home/pgc29/scratch60/test.txt
INFO: Running depends_test: 
INFO: depends_test (index=0) is ignored due to saved signature
INFO: depends_test (index=0) is ignored due to saved signature
INFO: depends_test output:   /home/pgc29/scratch60/test2.txt
INFO: Workflow depends_test (ID=w01db23b738dc700b) is executed successfully with 1 completed step, 1 ignored step and 1 completed task.
@BoPeng BoPeng self-assigned this Aug 18, 2022
@BoPeng
Copy link
Contributor

BoPeng commented Aug 18, 2022

It is a moderately complex setup. Let me try to reproduce it and see why signature does not work.

@BoPeng BoPeng added the bug label Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants