Identity or track attribute for `SkeletonInstance`

[talmo]

The SkeletonInstance is intended to hold point data for ground truth annotations on individual images (in a TrainingFrame).

It would be great to support an attribute to store some notion of identity, i.e., what animal is which.

In temporally contiguous data, the identity is the same thing as the track, i.e., links between detections in individual frames.

For example, the PoseEstimationSeries supports the Subject attribute, in addition to the implicit association of multiple PoseEstimation objects being grouped within the same PoseEstimationSeries.

In temporally discontiguous data, like sparsely sampled images from different frames and different videos, identity is a bit trickier.

For subjects that look visually identical, it is not possible to simply look at a static image and be able to tell which subject is which. In these cases, the identity is derived from temporal contiguity from adjacent frames, but it might still be helpful to associate the Subject metadata if that is available at data creation time.

For subjects that don't look visually identical, storing the identity metadata can serve the purpose of supporting ReID-like models that use visual features to classify which identity the pose belongs to. This is done in SLEAP's supervised ID models, as well as idTracker.ai.

A minimal implementation of this would be support associating a Subject with a SkeletonInstance, so that we can say that a particular pose on a particular image is associated with a specific identity.

For example, in the flies13 dataset in SLEAP, we assign a Track with name "male" or "female" to each individual pose instance since you can tell the sex from the static appearance of the animal.

This brings up a different level of granularity which may or may not be of interest to you guys to explicitly represent. Here's a summary:

Granularity	Description	Example	SLEAP	NWB
Track	Poses that belong to the same track are temporally linked. Useful for timeseries analysis, or for training MOT models.	Pose 1 in frame 100 is the same subject as pose 2 in frame 101	Track	PoseEstimationSeries (implicit)
Identity class	Poses that belong to the same type of subject. You can't tell which individual from that class that it is, but by experimental design, there may be just one subject of that class present. Useful for training ReID or classification models that predict the identity from a static image.	Male vs female; resident vs intruder; adult vs juvenile; shaved vs unshaved; implanted animal vs naive animal; mutant vs wild type	Track (not ideal)	N/A?
Unique identity	The globally unique identity of the individual. This should span sessions and be permanent for any data associated with that individual. This is often treated as metadata outside of what's possible to infer from the data alone. A notable exception might be RFID implants, which are often used both in animal husbandry practice and in certain behavioral monitoring setups.	Subject XYZ	Track (not ideal)	Subject

As you can see, ultimately you just need a string to store various levels of identity, so we abuse the Track object in SLEAP to do this, but having explicit differentiation might be better to handle more diverse types of data and experiments.

Either way, this is the last missing attribute for the PoseTraining-related data structures to achieve parity with the SLP format.