diff --git a/docs/source/cmdline/download.rst b/docs/source/cmdline/download.rst index 32407ac76..2ac097911 100644 --- a/docs/source/cmdline/download.rst +++ b/docs/source/cmdline/download.rst @@ -65,3 +65,23 @@ Options .. option:: --sync Delete local assets that do not exist on the server after downloading + +.. option:: --zarr FILTER + + Only download the entries of Zarr assets that match the given filter. The + filter is either the predefined name ``metadata``, which selects the Zarr + metadata files (``.zarray``, ``.zattrs``, ``.zgroup``, ``.zmetadata``, and + ``zarr.json``), or ``TYPE:PATTERN``, where ``TYPE`` is one of: + + - ``glob`` — ``PATTERN`` is a glob matched against the entry's path within + the Zarr, with ``**`` matching across directories (e.g., + ``glob:0/**/*``) + + - ``path`` — ``PATTERN`` is a path within the Zarr; the entry at that path + and all entries under it are downloaded + + - ``regex`` — ``PATTERN`` is a regular expression searched for in the + entry's path within the Zarr + + Can be specified multiple times, in which case an entry is downloaded if it + matches any of the filters. diff --git a/docs/source/cmdline/organize.rst b/docs/source/cmdline/organize.rst index 890a94ebe..16c1d3e5a 100644 --- a/docs/source/cmdline/organize.rst +++ b/docs/source/cmdline/organize.rst @@ -65,6 +65,11 @@ Options What to do if files without sufficient metadata are encountered [default: ``fail``] +.. option:: -J, --jobs N + + Number of parallel jobs to use while organizing, e.g., for extracting the + metadata from the files [default: one per CPU core] + .. option:: --media-files-mode [copy|move|symlink|hardlink] How to relocate video files referenced by NWB files [default: ``symlink``] diff --git a/docs/source/cmdline/upload.rst b/docs/source/cmdline/upload.rst index ac8934226..7f2898dd9 100644 --- a/docs/source/cmdline/upload.rst +++ b/docs/source/cmdline/upload.rst @@ -60,6 +60,15 @@ Options Data should pass validation before uploading. Use of this option is highly discouraged. +.. option:: --zarr-mode [full|patch] + + How to synchronize Zarr assets with the server: + + - ``full`` [default] — make the Zarr on the server identical to the local + one, deleting entries on the server that do not exist locally + - ``patch`` — upload new and changed entries only, without deleting + anything on the server + Development Options ------------------- diff --git a/docs/source/cmdline/validate-bids.rst b/docs/source/cmdline/validate-bids.rst new file mode 100644 index 000000000..80740b9aa --- /dev/null +++ b/docs/source/cmdline/validate-bids.rst @@ -0,0 +1,34 @@ +:program:`dandi validate-bids` +============================== + +:: + + dandi [] validate-bids [] [ ...] + +Validate BIDS paths. + +.. note:: + + This command is deprecated: :ref:`dandi validate ` + validates BIDS datasets along with everything else and should be used + instead. ``dandi validate-bids`` now merely invokes it with the given + paths and :option:`--grouping` (after emitting a deprecation warning). + +Options +------- + +.. option:: -g, --grouping [none|path] + + How to group the reported results [default: ``none``] + +.. option:: --report-path + + Accepted for backwards compatibility but ignored + +.. option:: -r, --report + + Accepted for backwards compatibility but ignored + +.. option:: --schema VERSION + + Accepted for backwards compatibility but ignored diff --git a/docs/source/cmdline/validate.rst b/docs/source/cmdline/validate.rst index 111ccd441..3a07d7e7a 100644 --- a/docs/source/cmdline/validate.rst +++ b/docs/source/cmdline/validate.rst @@ -1,30 +1,87 @@ +.. _dandi_validate: + :program:`dandi validate` ========================= :: - dandi [] validate [ ...] + dandi [] validate [] [ ...] Validate files for data standards compliance. Exits with non-zero exit code if any file is not compliant. +The validation results are automatically saved as a JSON Lines companion file +next to the dandi-cli log file (unless :option:`--output` is used or +:option:`--load` is active). Use :option:`--load` to re-render saved results +later with different grouping, filtering, or format options. + Options ------- -.. option:: -g, --grouping [none|path] +.. option:: -g, --grouping [none|path|severity|id|validator|standard|dandiset] - Set how to group reported errors & warnings: by path or not at all - (default) + How to group the reported results. Repeat the option for hierarchical + nesting, e.g., ``-g severity -g id``. [default: ``none``] .. option:: --ignore REGEX - Ignore any validation errors & warnings whose ID matches the given regular + Ignore any validation results whose ID matches the given regular expression -.. option:: --min-severity [HINT|WARNING|ERROR] +.. option:: --min-severity [INFO|HINT|WARNING|ERROR|CRITICAL] + + Only display results with severities at or above this level [default: + ``HINT``] + +.. option:: -f, --format [text|json|json_pp|json_lines|yaml] + + Output format [default: ``text``] + +.. option:: -o, --output + + Write the output to the given file instead of standard output. This + requires a structured :option:`--format`; if none is given, the format is + inferred from the file's extension (``.json``, ``.jsonl``, ``.yaml``, or + ``.yml``). :option:`--grouping` cannot be combined with the ``json_lines`` + format. + +.. option:: --summary, --no-summary + + Whether to show summary statistics (counts of results by severity, + validator, and standard) after the results [default: ``--no-summary``] + +.. option:: --max-per-group N + + Limit the number of results shown per group (or in total when not + grouping); the excess is replaced by a count of omitted results + +.. option:: --missing-file-content [error|only-non-data|skip] + + How to handle files whose content is unavailable, such as the broken + symbolic links of a DataLad_ dataset (a git-annex_ repository) whose + content has not been fetched: + + ``error`` + Emit a concise ``DANDI.FILE_CONTENT_MISSING`` error for each such file + (default) + + ``skip`` + Skip each such file, emitting a warning + + ``only-non-data`` + Skip content-dependent validators (pynwb, nwbinspector, ...) for each + such file but still validate its path layout + +.. option:: --load + + Instead of running validation, load previously saved results from the + given JSON Lines file (e.g., an automatically saved companion file) and + render them. Can be specified multiple times; cannot be combined with + paths. - Only display issues with severities above this level (HINT by default) +.. _DataLad: https://www.datalad.org +.. _git-annex: https://git-annex.branchable.com Development Options