feat: add numberOfSessions to AssetsSummary by bendichter · Pull Request #409 · dandi/dandi-schema

bendichter · 2026-05-14T00:16:24Z

Summary

Add optional numberOfSessions: int field to AssetsSummary, with a schema description documenting the counting semantics
Bump DANDI_SCHEMA_VERSION to 0.7.1, since 0.7.0 is already published in dandi/schema and the release workflow rejects modifications to published schemata
Aggregator counts unique (subject, session) pairs parsed from BIDS-style sub-* / ses-* tokens in asset paths. Keying on the pair avoids the undercount when two subjects share a session id like ses-1
Filename is parsed first (matching existing sub-/sample- handling); directory components are scanned as a fallback so ses-* tokens that appear only in the path (e.g. sub-X/ses-Y/foo_acq-Z_bold.nii.gz) are still counted

Caveats

Assets whose paths carry no sub-/ses- tokens won't contribute. Note that dandi organize produces sub-X/sub-X_ses-Y.nwb layouts, so organized NWB dandisets with session ids are counted; only datasets that follow neither BIDS nor the organized layout are missed.

Test plan

New test_aggregate_number_of_sessions covers: same subject across sessions, two subjects sharing a session id, ses- only in directory, ses- without any sub-, and the no-ses- case
Existing test_aggregate fixtures updated with expected numberOfSessions values
Existing test_aggregation_bids now asserts numberOfSessions == 2
Full suite: 268 passed, 17 skipped; tox -e lint,typing clean

🤖 Generated with Claude Code

Aggregator counts unique (subject, session) pairs parsed from BIDS-style sub-* and ses-* tokens in asset paths (filename first, then directory components when the session token is omitted from the filename). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

codecov · 2026-05-14T00:16:33Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.27%. Comparing base (3e8ddd7) to head (f73f95c).

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #409       +/-   ##
===========================================
+ Coverage   47.96%   97.27%   +49.31%     
===========================================
  Files          19       19               
  Lines        2427     2460       +33     
===========================================
+ Hits         1164     2393     +1229     
+ Misses       1263       67     -1196

Flag	Coverage Δ
unittests	`97.27% <100.00%> (+49.31%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

# Conflicts: # dandischema/metadata.py

yarikoptic · 2026-05-14T11:57:46Z

+        if not found.get("session") and part.startswith("ses-"):
+            found["session"] = part.split("ses-", 1)[1]
+    # Fallback: ses- tokens that appear only in directory components (e.g.
+    # `sub-X/ses-Y/foo_acq-Z_bold.nii.gz`) should still be counted. To form
+    # the (subject, session) pair we also accept a directory-only subject,
+    # but we do not add such a subject to stats["subjects"] — subject
+    # counting remains driven by the filename and wasAttributedTo, matching
+    # prior behavior.
+    if not found.get("session"):
+        dir_subject = found.get("subject")
+        dir_session: Optional[str] = None
+        for directory in asset_path.parts[:-1]:
+            for part in directory.split("_"):
+                if dir_subject is None and part.startswith("sub-"):
+                    dir_subject = part.split("sub-", 1)[1]
+                if dir_session is None and part.startswith("ses-"):
+                    dir_session = part.split("ses-", 1)[1]
+        if dir_subject is not None and dir_session is not None:
+            found.setdefault("subject", dir_subject)
+            found["session"] = dir_session
+    stats["sessions"] = stats.get("sessions", [])
+    if found.get("subject") and found.get("session"):
+        pair = (found["subject"], found["session"])
+        if pair not in stats["sessions"]:
+            stats["sessions"].append(pair)


too aisloppy -- easier to HI compose the logic but first potentially even generalize 2 prior blocks into a 'for' loop, then if no session was found in filename, go through path's parts and split on ses- and be done if any matches.

Done in cdb37ee — collapsed the sub-/sample-/ses- filename parsing into a single loop over an entities list, and simplified the session fallback to scanning asset_path.parts[:-1] for a ses-X directory and stopping at the first match. Dropped the directory-only subject fallback since BIDS files always carry sub- in the filename.

yarikoptic

THANK YOU for the PR. Needs a little a human touch ;)

Per review feedback: generalize the per-prefix blocks (sub-, sample-, ses-) into a single loop over an entity list, and simplify the session fallback to a direct scan of path parts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

candleindark · 2026-05-14T16:57:21Z

Is the minor label a bit too much? Since we don't have a major in our version number, patch has been carrying the semantics of a minor.

bendichter · 2026-05-14T16:59:27Z

yes, I think patch makes sense for this while major is 0

candleindark

LGTM as the targeted datasets are valid BIDS datasets. (See other comments for more details).

candleindark · 2026-05-14T19:04:49Z

    numberOfCells: Optional[int] = Field(None, json_schema_extra={"readOnly": True})
+    numberOfSessions: Optional[int] = Field(
+        None, json_schema_extra={"readOnly": True}
+    )  # BIDS ses-* tokens, counted as unique (subject, session) pairs


Possibly add to the comment that this count assumes the dataset is valid BIDS dataset. (For example, a session without subject will not be counted)

well, that's what I wonder -- since we do have sub-..._ses-... even in DANDI names, and we do count on those -- we better count sessions as well in non-BIDS names as well. but "AI logic" above is wrong for that... will comment there

actually my initial read was wrong -- there is a generic handling in the filename at https://github.com/dandi/dandi-schema/pull/409/changes#diff-196e8d7f9869422a0dba629b1f58a9972a300d67308aec783b2847cdb50ef5ffR541

but wouldn't it work for DANDI names (no folder name) as well and thus just a matter of

Suggested change

) # BIDS ses-* tokens, counted as unique (subject, session) pairs

) # BIDS or DANDI ses-* tokens (in file name or path), counted as unique (subject, session) pairs

?

Right, it works on DANDI names too. Changed the description to say BIDS or DANDI ses- tokens in file names or paths.

Coverage's `source = dandischema` resolves to the in-repo tree, but the package was installed (non-editable) into the tox venv, so the source modules actually executed from site-packages and fell outside the measured tree. As a result Codecov recorded 0% for every source module (metadata.py, models.py, ...) — only the test files, which run from the repo, were counted. This dragged patch coverage below the project threshold even for fully tested changes. Setting `usedevelop = true` makes the measured source and the executed source the same in-repo files, so metadata.py now reports ~90% instead of 0% and Codecov can map diff lines correctly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Schema 0.7.0 is already published in dandi/schema, so adding the numberOfSessions field to AssetsSummary requires a new schema version (the release workflow guards against modifying published schemata). Also add a schema description for the field documenting the (subject, session) pair counting, mark the new test as ai_generated, and cover the case where ses- appears without any sub- token. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

schema-0.8.0 was released while this branch was open, so the version this branch introduces is now 0.8.1 rather than 0.7.1, with 0.8.0 added to the allowed input and target lists the way master did for 0.7.0. master also removed TestValidateObjJson from test_metadata.py; the session-count tests this branch adds are kept and that class is dropped.

Per review: the token handling is not BIDS-specific, it works on DANDI-style file names too, so the field description should not say only BIDS. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWivE9AD3FkcAPr66oNnfH

Merge remote-tracking branch 'origin/master' into add-number-of-sessions

c7069f6

# Conflicts: # dandischema/metadata.py

bendichter requested review from candleindark and yarikoptic May 14, 2026 00:21

bendichter marked this pull request as ready for review May 14, 2026 00:29

yarikoptic reviewed May 14, 2026

View reviewed changes

yarikoptic requested changes May 14, 2026

View reviewed changes

yarikoptic added enhancement New feature or request minor Increment the minor version when merged labels May 14, 2026

bendichter mentioned this pull request May 14, 2026

AssetsSummary: numberOfSamples / numberOfCells under-count when the same id appears across subjects #411

Open

candleindark reviewed May 14, 2026

View reviewed changes

bendichter mentioned this pull request Jun 11, 2026

Add num_subjects: advanced-search operator dandi/dandi-archive#2827

Open

4 tasks done

bendichter and others added 7 commits July 7, 2026 09:10

Merge branch 'master' into add-number-of-sessions

6cba609

Merge branch 'master' into add-number-of-sessions

839f861

Merge branch 'master' into add-number-of-sessions

8b31bc6

bendichter requested a review from yarikoptic September 15, 2026 17:52

Navigation Menu

Repository navigation

feat: add numberOfSessions to AssetsSummary - #409

feat: add numberOfSessions to AssetsSummary#409
bendichter wants to merge 10 commits into
masterfrom
add-number-of-sessions

bendichter commented May 14, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 14, 2026 •

edited

Loading

Uh oh!

yarikoptic May 14, 2026

Uh oh!

bendichter May 14, 2026

Uh oh!

yarikoptic left a comment

Uh oh!

candleindark commented May 14, 2026

Uh oh!

bendichter commented May 14, 2026

Uh oh!

candleindark left a comment

Uh oh!

candleindark May 14, 2026

Uh oh!

yarikoptic Jul 20, 2026

Uh oh!

yarikoptic Jul 20, 2026

Uh oh!

bendichter Sep 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	) # BIDS ses-* tokens, counted as unique (subject, session) pairs
	) # BIDS or DANDI ses-* tokens (in file name or path), counted as unique (subject, session) pairs

Conversation

bendichter commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Caveats

Test plan

Uh oh!

codecov Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

yarikoptic May 14, 2026

Choose a reason for hiding this comment

Uh oh!

bendichter May 14, 2026

Choose a reason for hiding this comment

Uh oh!

yarikoptic left a comment

Choose a reason for hiding this comment

Uh oh!

candleindark commented May 14, 2026

Uh oh!

bendichter commented May 14, 2026

Uh oh!

candleindark left a comment

Choose a reason for hiding this comment

Uh oh!

candleindark May 14, 2026

Choose a reason for hiding this comment

Uh oh!

yarikoptic Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

yarikoptic Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

bendichter Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bendichter commented May 14, 2026 •

edited

Loading

codecov Bot commented May 14, 2026 •

edited

Loading