@@ -6,29 +6,37 @@ ndx-pose is a standardized format for storing pose estimation data in NWB, such
66[ DeepLabCut] ( http://www.mackenziemathislab.org/deeplabcut ) and [ SLEAP] ( https://sleap.ai/ ) .
77Please post an issue or PR to suggest or add support for another pose estimation tool.
88
9- This extension consists of several new neurodata types:
9+ ## Data types overview
10+ This extension consists of several new neurodata types. They are divided into two main categories:
11+ 1 . ** Pose estimation data** : This includes the estimated positions of body parts (keypoints) over time, along with
12+ metadata about the pose estimation process.
13+ 2 . ** Training data** : This includes ground truth data for training pose estimation models, such as labeled images and
14+
15+ ## Pose estimation data types:
1016- ` Skeleton ` which stores the relationship between the body parts (nodes and edges).
1117- ` Skeletons ` which is a container that stores multiple ` Skeleton ` objects.
1218- ` PoseEstimationSeries ` which stores the estimated positions (x, y) or (x, y, z) of a body part over time as well as
1319the confidence/likelihood of the estimated positions.
1420- ` PoseEstimation ` which stores the estimated position data (` PoseEstimationSeries ` ) for multiple body parts,
1521computed from the same video(s) with the same tool/algorithm.
22+ ## Training data types:
1623- ` SkeletonInstance ` which stores the estimated positions and visibility of the body parts for a single frame.
1724- ` TrainingFrame ` which stores the ground truth data for a single frame. It contains ` SkeletonInstance ` objects and
1825references a frame of a source video (` ImageSeries ` ). The source videos can be stored internally as data arrays or
1926externally as files referenced by relative file path.
2027- ` TrainingFrames ` which is a container that stores multiple ` TrainingFrame ` objects.
2128- ` SourceVideos ` which is a container that stores multiple ` ImageSeries ` objects representing source videos used in training.
22- - ` PoseTraining ` which is a container thatstores the ground truth data (` TrainingFrames ` ) and source videos (` SourceVideos ` )
29+ - ` PoseTraining ` which is a container that stores the ground truth data (` TrainingFrames ` ) and source videos (` SourceVideos ` )
2330used to train the pose estimation model.
2431
2532It is recommended to place the ` Skeletons ` , ` PoseEstimation ` , and ` PoseTraining ` objects in an NWB processing module
2633named "behavior", as shown below.
2734
2835## Installation
2936
30- ` pip install ndx-pose `
31-
37+ ``` bash
38+ pip install " ndx-pose"
39+ ```
3240## Usage examples
3341
34421 . [ Example writing pose estimates (keypoints) to an NWB file] ( examples/write_pose_estimates_only.py ) .
@@ -42,8 +50,7 @@ As a result, ndx-pose was designed to store pose estimates from a single subject
4250Pose estimates data from different subjects should be stored in separate NWB files.
4351
4452Training images can involve multiple skeletons, however. These training images may be the same across subjects,
45- and therefore the same across NWB files. These training images should be duplicated between files, until
46- multi-subject support is added to NWB and ndx-pose. See https://github.com/rly/ndx-pose/pull/3
53+ and therefore the same across NWB files. These training images should be duplicated between files.
4754
4855## Resources
4956
@@ -61,8 +68,7 @@ Keypoint MoSeq: https://github.com/dattalab/keypoint-moseq
6168- Supports read of ` PoseEstimation ` objects from NWB files.
6269
6370NeuroConv: https://neuroconv.readthedocs.io/en/main/conversion_examples_gallery/conversion_example_gallery.html#behavior
64- - NeuroConv supports converting data from DeepLabCut (using ` dlc2nwb ` described above),
65- SLEAP (using ` sleap_io ` described above), FicTrac, and LightningPose to NWB. It supports appending pose estimation data to an existing NWB file.
71+ - NeuroConv supports converting data from DeepLabCut, SLEAP (using ` sleap_io ` described above), and LightningPose to NWB. It also supports appending pose estimation data to an existing NWB file.
6672
6773Ethome: Tools for machine learning of animal behavior: https://github.com/benlansdell/ethome
6874- Supports read of ` PoseEstimation ` objects from NWB files.
@@ -78,7 +84,7 @@ Several NWB datasets use ndx-pose 0.1.1:
7884Several [ open-source conversion scripts on GitHub] ( https://github.com/search?q=ndx-pose&type=code&p=1 )
7985also use ndx-pose.
8086
81- ## Diagram of non-training-related types
87+ ## Diagram of pose estimation types
8288
8389``` mermaid
8490%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', "primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%
@@ -121,6 +127,7 @@ classDiagram
121127 name : str
122128 nodes : array[str; dims [body part]]
123129 edges : array[uint; dims [edge, [node, node]]]
130+ subject: link (to pynwb.Subject), optional
124131 }
125132
126133 }
0 commit comments