Skip to content

Conversation

@bendichter
Copy link
Contributor

@bendichter bendichter commented Oct 25, 2025

fix #1771

Add comprehensive support for audio and video recordings in behavioral
experiments:

- Add audio file extensions (mp3, wav) and video file extensions
  (mp4, mkv, avi) with corresponding _audio and _video suffixes
- Document usage of audio/video recordings in beh directory for
  capturing vocalizations, speech, facial expressions, and body movements
- Add metadata schema for audio/video device information and stream
  properties
- Include privacy warnings about personally identifiable information
  in human subject recordings
- Update behavioral experiments title to remove "with no neural
  recordings" restriction, clarifying data can be stored with or
  without neural recordings
- Add examples for file organization including multi-angle recordings
  and split files
- Define optional entities: task, acquisition, run, recording, split
@yarikoptic yarikoptic changed the title SCHEMA: Add audio video SCHEMA: Add audio video behavioral data support Oct 25, 2025
@yarikoptic yarikoptic added the schema Issues related to the YAML schema representation of the specification. Patch version release. label Oct 25, 2025
…ee macros

- Change section title from 'Behavioral experiments' to 'Behavioral recordings'
- Convert file tree examples to use MACROS___make_filetree_example for consistent rendering
- Address review comments from @yarikoptic in PR #2231
Copy link
Collaborator

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this makes sense to me. It would be good to get some feedback from contributors to related BEPs, such as eye-tracking (20), motion (29), stimuli (44) and physio (45). Even if this PR doesn't propose adding this as an associated file to those data types, the potential is there and it's worth getting opinions and identifying potential conflicts.

cc @bids-standard/bep029 @bids-standard/bep044
cc @mszinte @julia-pfarr @oesteban (BEP020)
cc @m-miedema @smoia @SouravKulkarni (?) (BEP045)

Comment on lines 146 to 150
"sub-01_ses-01_task-freeplay_run-01_video": {
"split-001.mp4": "",
"split-002.mp4": "",
"split-003.mp4": "",
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a substantial change in how much extensions, entities and suffixes can be reordered. I talked with @yarikoptic yesterday and I think this is an interesting future direction, but something that does not require any changes at the moment would be:

Suggested change
"sub-01_ses-01_task-freeplay_run-01_video": {
"split-001.mp4": "",
"split-002.mp4": "",
"split-003.mp4": "",
},
"sub-01_ses-01_task-freeplay_run-01_split-001_video.mp4": "",
"sub-01_ses-01_task-freeplay_run-01_split-002_video.mp4": "",
"sub-01_ses-01_task-freeplay_run-01_split-003_video.mp4": "",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yarikoptic 's concern here was that if we have hundreds or even thousands of video files this would get cumbersome. @talmo also preferred the subdirectory approach.

From my perspective, my philosophy is to only propose changes to the standard for expressibility or performance and this a large proposed change that seems merely for convenience, not expressibility or performance (correct me if I am wrong), so I am fine with reverting to the standard split organization (i.e. no subdirectory).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see I missed this discussion, apologies for jumping in late. I unresolved this so others can see this conversation, but reading through the previous discussion I think there's an 80/20 split here where it's better to work within the current BIDS constraints and then separately push for a generalization to _<suffix>/split-<label>.<ext> if that becomes a practical problem for people.

I would say it's a BEP-level change that would require design for the schema and validator and would significantly slow down this proposal to include it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

@effigies effigies changed the title SCHEMA: Add audio video behavioral data support [ENH] Add audio/video recordings to behavioral experiments Oct 28, 2025
@codecov
Copy link

codecov bot commented Oct 28, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.86%. Comparing base (7c06228) to head (aead0a4).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2231   +/-   ##
=======================================
  Coverage   82.86%   82.86%           
=======================================
  Files          20       20           
  Lines        1669     1669           
=======================================
  Hits         1383     1383           
  Misses        286      286           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

schema Issues related to the YAML schema representation of the specification. Patch version release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BEP for audio/video capture of behaving subjects

6 participants