Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exact definition of --tmp-outdir-prefix and --tmpdir-prefix is ambiguous #3

Open
schultzmattd opened this issue Oct 17, 2015 · 0 comments

Comments

@schultzmattd
Copy link

I've hit what I think is an inconsistency in the expectation of these two arguments. When the arguments for the cwl-runner are first parsed, there are a couple of checks to make sure that the paths provided in these two arguments refer to actual directories:

cwltool/main.py:350

    if args.tmp_outdir_prefix != 'tmp':
        # Use user defined temp directory (if it exists)
        args.tmp_outdir_prefix = os.path.abspath(args.tmp_outdir_prefix)
        if not os.path.exists(args.tmp_outdir_prefix):
            _logger.error("Intermediate output directory prefix doesn't exist, reverting to default")
            return 1

    if args.tmpdir_prefix != 'tmp':
        # Use user defined prefix (if the folder exists)
        args.tmpdir_prefix = os.path.abspath(args.tmpdir_prefix)
        if not os.path.exists(args.tmpdir_prefix):
            _logger.error("Temporary directory prefix doesn't exist.")
            return 1

However, when these arguments are actually used, they aren't required to be, and in fact, will be used like the prefixes to directories that are created during the CWL run:

draft2tool.py:137

        if dockerReq and kwargs.get("use_container"):
            out_prefix = kwargs.get("tmp_outdir_prefix")
            j.outdir = kwargs.get("outdir") or tempfile.mkdtemp(prefix=out_prefix)
            tmpdir_prefix = kwargs.get('tmpdir_prefix')
            j.tmpdir = kwargs.get("tmpdir") or tempfile.mkdtemp(prefix=tmpdir_prefix)
        else:
            j.outdir = builder.outdir
            j.tmpdir = builder.tmpdir

Is it possible to either clarify in the documentation exactly how these arguments are expected to be used, or reconcile the usage in the code? My two cents is to make the first code block check for that the basedir of the argument exists, and then using them as prefixes to which random strings will be appended.

tetron added a commit that referenced this issue Dec 31, 2015
restore missing file from PyPI release
alexbarrera added a commit to alexbarrera/cwltool that referenced this issue Jul 27, 2016
lijiayong referenced this issue in lijiayong/cwltool Mar 17, 2017
misterbrandonwalker pushed a commit to misterbrandonwalker/cwltool that referenced this issue Mar 17, 2023
# This is the 1st commit message:

Add a file to each cachedir folder titled jobname with the job string in the file. This way users can maintain knowledge of which cachedir corresponds to which job.

# This is the commit message common-workflow-language#2:

Add back white spaces that were removed.

# This is the commit message common-workflow-language#3:

Remove extra white space added.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant