Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/source/cmdline/download.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,23 @@ Options
.. option:: --sync

Delete local assets that do not exist on the server after downloading

.. option:: --zarr FILTER
Comment thread
CodyCBakerPhD marked this conversation as resolved.

Only download the entries of Zarr assets that match the given filter. The
filter is either the predefined name ``metadata``, which selects the Zarr
metadata files (``.zarray``, ``.zattrs``, ``.zgroup``, ``.zmetadata``, and
``zarr.json``), or ``TYPE:PATTERN``, where ``TYPE`` is one of:

- ``glob`` — ``PATTERN`` is a glob matched against the entry's path within
the Zarr, with ``**`` matching across directories (e.g.,
``glob:0/**/*``)

- ``path`` — ``PATTERN`` is a path within the Zarr; the entry at that path
and all entries under it are downloaded

- ``regex`` — ``PATTERN`` is a regular expression searched for in the
entry's path within the Zarr

Can be specified multiple times, in which case an entry is downloaded if it
matches any of the filters.
5 changes: 5 additions & 0 deletions docs/source/cmdline/organize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ Options
What to do if files without sufficient metadata are encountered [default:
``fail``]

.. option:: -J, --jobs N

Number of parallel jobs to use while organizing, e.g., for extracting the
metadata from the files [default: one per CPU core]

.. option:: --media-files-mode [copy|move|symlink|hardlink]

How to relocate video files referenced by NWB files [default: ``symlink``]
Expand Down
9 changes: 9 additions & 0 deletions docs/source/cmdline/upload.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,15 @@ Options
Data should pass validation before uploading. Use of this option is highly
discouraged.

.. option:: --zarr-mode [full|patch]

How to synchronize Zarr assets with the server:

- ``full`` [default] — make the Zarr on the server identical to the local
one, deleting entries on the server that do not exist locally
- ``patch`` — upload new and changed entries only, without deleting
anything on the server


Development Options
-------------------
Expand Down
34 changes: 34 additions & 0 deletions docs/source/cmdline/validate-bids.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
:program:`dandi validate-bids`
==============================

::

dandi [<global options>] validate-bids [<options>] [<path> ...]

Validate BIDS paths.

.. note::

This command is deprecated: :ref:`dandi validate <dandi_validate>`
validates BIDS datasets along with everything else and should be used
instead. ``dandi validate-bids`` now merely invokes it with the given
paths and :option:`--grouping` (after emitting a deprecation warning).

Options
-------

.. option:: -g, --grouping [none|path]

How to group the reported results [default: ``none``]

.. option:: --report-path <path>

Accepted for backwards compatibility but ignored

.. option:: -r, --report

Accepted for backwards compatibility but ignored

.. option:: --schema VERSION

Accepted for backwards compatibility but ignored
71 changes: 64 additions & 7 deletions docs/source/cmdline/validate.rst
Original file line number Diff line number Diff line change
@@ -1,30 +1,87 @@
.. _dandi_validate:

:program:`dandi validate`
=========================

::

dandi [<global options>] validate [<path> ...]
dandi [<global options>] validate [<options>] [<path> ...]

Validate files for data standards compliance.

Exits with non-zero exit code if any file is not compliant.

The validation results are automatically saved as a JSON Lines companion file
next to the dandi-cli log file (unless :option:`--output` is used or
:option:`--load` is active). Use :option:`--load` to re-render saved results
later with different grouping, filtering, or format options.

Options
-------

.. option:: -g, --grouping [none|path]
.. option:: -g, --grouping [none|path|severity|id|validator|standard|dandiset]

Set how to group reported errors & warnings: by path or not at all
(default)
How to group the reported results. Repeat the option for hierarchical
nesting, e.g., ``-g severity -g id``. [default: ``none``]

.. option:: --ignore REGEX

Ignore any validation errors & warnings whose ID matches the given regular
Ignore any validation results whose ID matches the given regular
expression

.. option:: --min-severity [HINT|WARNING|ERROR]
.. option:: --min-severity [INFO|HINT|WARNING|ERROR|CRITICAL]

Only display results with severities at or above this level [default:
``HINT``]

.. option:: -f, --format [text|json|json_pp|json_lines|yaml]

Output format [default: ``text``]

.. option:: -o, --output <file>

Write the output to the given file instead of standard output. This
requires a structured :option:`--format`; if none is given, the format is
inferred from the file's extension (``.json``, ``.jsonl``, ``.yaml``, or
``.yml``). :option:`--grouping` cannot be combined with the ``json_lines``
format.

.. option:: --summary, --no-summary

Whether to show summary statistics (counts of results by severity,
validator, and standard) after the results [default: ``--no-summary``]

.. option:: --max-per-group N

Limit the number of results shown per group (or in total when not
grouping); the excess is replaced by a count of omitted results

.. option:: --missing-file-content [error|only-non-data|skip]

How to handle files whose content is unavailable, such as the broken
symbolic links of a DataLad_ dataset (a git-annex_ repository) whose
content has not been fetched:

``error``
Emit a concise ``DANDI.FILE_CONTENT_MISSING`` error for each such file
(default)

``skip``
Skip each such file, emitting a warning

``only-non-data``
Skip content-dependent validators (pynwb, nwbinspector, ...) for each
such file but still validate its path layout

.. option:: --load <file>

Instead of running validation, load previously saved results from the
given JSON Lines file (e.g., an automatically saved companion file) and
render them. Can be specified multiple times; cannot be combined with
paths.

Only display issues with severities above this level (HINT by default)
.. _DataLad: https://www.datalad.org
.. _git-annex: https://git-annex.branchable.com


Development Options
Expand Down
Loading