Docs update (#993)

* Fix printing of auto-selected GPU free memory (#955)

* Fix printing of auto-selected GPU free memory

* Fix printing in inference

* Lint

* Fix add suggestions when target is current video (#956)

* Fix export cli arg parsing (#962)

* Fix cattr in Python 3.9 (#967)

* Remove structuring hook for forward reference and handle manually

* Handle missing key

* Add sample movies to docs (#992)

* Add initial examples

* Add clips for all datasets

* Add quick example clip

* Typo on instance count for flies13

Co-authored-by: Liezl Maree <[email protected]>
Co-authored-by: sheridana <[email protected]>

Author: 3 people

.github/workflows/website.yml

@@ -8,7 +8,7 @@ on:
       # 'main' triggers updates to 'sleap.ai', 'develop' to 'sleap.ai/develop'
       - main
       - develop
-      - liezl/siv-inference
+      - talmo/sample-movies
     paths:
       - "docs/**"
       - "README.rst"

docs/datasets.md

@@ -11,6 +11,10 @@ For the full set of labeled datasets and trained models reported in the SLEAP pa
 ([Pereira et al., Nature Methods, 2022](https://www.nature.com/articles/s41592-022-01426-1)),
 please see the [OSF repository](https://osf.io/36har/).
 
+````{hint}
+Need a quick testing clip? [Here's a video of a pair of mice in a standard home cage setting.](
+https://storage.googleapis.com/sleap-data/sandbox/sleap-mice-demo/mice.mp4)
+````
 
 ## `fly32`
 ![fly32](_static/example.fly32.jpg)
@@ -43,7 +47,9 @@ widths: 10 40
 * - Labels
   - 1500
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/random_split1/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/clips/072212_173836%400-3200.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/BermanFlies/clips/072212_173836%400-3200.slp)
 * - Credit
   - [Berman et al. (2014)](https://royalsocietypublishing.org/doi/10.1098/rsif.2014.0672), [Pereira et al. (2019)](https://www.nature.com/articles/s41592-018-0234-5), [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), Talmo Pereira, Gordon Berman, Joshua Shaevitz
 ```
@@ -77,9 +83,11 @@ widths: 10 40
 * - Identity
   - ✔
 * - Labels
-  - 2000 frames, 2000 instances
+  - 2000 frames, 4000 instances
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/tracking_split2/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/clips/talk_title_slide%4013150-14500.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/wt_gold.13pt/clips/talk_title_slide%4013150-14500.slp)
 * - Credit
   - [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), Junyu Li, Shruthi Ravindranath, Talmo Pereira, Mala Murthy
 ```
@@ -115,7 +123,9 @@ widths: 10 40
 * - Labels
   - 1000 frames, 2950 instances
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/labels.full/random_split1/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/clips/OFTsocial5mice-0000-00%4015488-18736.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/wang_4mice_john/clips/OFTsocial5mice-0000-00%4015488-18736.slp)
 * - Credit
   - [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), John D'Uva, Mikhail Kislin, Samuel S.-H. Wang
 ```
@@ -151,7 +161,9 @@ widths: 10 40
 * - Labels
   - 1474 frames, 2948 instances
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/random_split1/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/clips/20200111_USVpairs_court1_M1_F1_top-01112020145828-0000%400-2560.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/eleni_mice/clips/20200111_USVpairs_court1_M1_F1_top-01112020145828-0000%400-2560.slp)
 * - Credit
   - [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), Eleni Papadoyannis, Mala Murthy, Annegret Falkner
 ```
@@ -187,7 +199,9 @@ widths: 10 40
 * - Labels
   - 804 frames, 1604 instances
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/yan_bees/random_split1/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/yan_bees/clips/bees_demo%4021000-23000.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/yan_bees/clips/bees_demo%4021000-23000.slp)
 * - Credit
   - [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), Grace McKenzie-Smith, Z. Yan Wang, Joshua Shaevitz, Sarah Kocher
 ```
@@ -223,7 +237,9 @@ widths: 10 40
 * - Labels
   - 425 frames, 1588 instances
 * - Download
-  - [Train](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/train.pkg.slp) / [Validation](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/val.pkg.slp) / [Test](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/test.pkg.slp)
+  - [Train (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/train.pkg.slp) / [Validation (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/val.pkg.slp) / [Test (`.pkg.slp`)](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/cohort1_compressedTalmo_23vids_march_7_to_march_17/random_split1.day001/test.pkg.slp)
+* - Example
+  - [Clip (`.mp4`)](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/clips/2020-3-10_daytime_5mins_compressedTalmo%403200-5760.mp4) / [Tracking (`.slp`)](https://storage.googleapis.com/sleap-data/datasets/nyu-gerbils/clips/2020-3-10_daytime_5mins_compressedTalmo%403200-5760.slp)
 * - Credit
   - [Pereira et al. (2022)](https://www.nature.com/articles/s41592-022-01426-1), Catalin Mitelut, Marielisa Diez Castro, Dan H. Sanes
 ```

sleap/gui/commands.py

@@ -2413,11 +2413,20 @@ def do_action(cls, context: CommandContext, params: dict):
         else:
             params["videos"] = context.labels.videos
 
-        new_suggestions = VideoFrameSuggestions.suggest(
-            labels=context.labels, params=params
-        )
+        try:
+            new_suggestions = VideoFrameSuggestions.suggest(
+                labels=context.labels, params=params
+            )
 
-        context.labels.append_suggestions(new_suggestions)
+            context.labels.append_suggestions(new_suggestions)
+        except Exception as e:
+            win.hide()
+            QtWidgets.QMessageBox(
+                text=f"An error occurred while generating suggestions. "
+                "Your command line terminal may have more information about "
+                "the error."
+            ).exec_()
+            raise e
 
         win.hide()
 

sleap/gui/suggestions.py

@@ -66,8 +66,10 @@ def suggest(cls, params: dict, labels: "Labels" = None) -> List[SuggestionFrame]
         if method_functions.get(method, None) is not None:
             return method_functions[method](labels=labels, **params)
         else:
-            print(f"No {method} method found for generating suggestions.")
-            return []
+            raise ValueError(
+                f"No {'' if method == '_' else method + ' '}method found for "
+                "generating suggestions."
+            )
 
     # Functions corresponding to "method" param
 
@@ -82,7 +84,8 @@ def basic_sample_suggestion_method(
     ):
         """Method to generate suggestions randomly or by taking strides through video."""
         suggestions = []
-        sugg_idx_dict: Dict[Video, list] = {video: [] for video in videos}
+        sugg_idx_dict: Dict[Video, list] = {video: [] for video in labels.videos}
+
         for sugg in labels.suggestions:
             sugg_idx_dict[sugg.video].append(sugg.frame_idx)
 
@@ -287,7 +290,7 @@ def filter_unique_suggestions(
         proposed_suggestions: List[SuggestionFrame],
     ) -> List[SuggestionFrame]:
         # Create log of suggestions that already exist
-        sugg_idx_dict: Dict[Video, list] = {video: [] for video in videos}
+        sugg_idx_dict: Dict[Video, list] = {video: [] for video in labels.videos}
         for sugg in labels.suggestions:
             sugg_idx_dict[sugg.video].append(sugg.frame_idx)
 

sleap/instance.py

@@ -1232,6 +1232,13 @@ def structure_instances_list(x, type):
             if "score" in inst_data.keys():
                 inst = converter.structure(inst_data, PredictedInstance)
             else:
+                if (
+                    "from_predicted" in inst_data
+                    and inst_data["from_predicted"] is not None
+                ):
+                    inst_data["from_predicted"] = converter.structure(
+                        inst_data["from_predicted"], PredictedInstance
+                    )
                 inst = converter.structure(inst_data, Instance)
             inst_list.append(inst)
 
@@ -1243,14 +1250,13 @@ def structure_instances_list(x, type):
 
     # Structure forward reference for PredictedInstance for the Instance.from_predicted
     # attribute.
-    converter.register_structure_hook(
-        ForwardRef("PredictedInstance"),
-        lambda x, _: converter.structure(x, PredictedInstance),
+    converter.register_structure_hook_func(
+        lambda t: t.__class__ is ForwardRef,
+        lambda v, t: converter.structure(v, t.__forward_value__),
     )
-
     # converter.register_structure_hook(
-    #     PredictedInstance,
-    #     lambda x, type: converter.structure(x, PredictedInstance),
+    #     ForwardRef("PredictedInstance"),
+    #     lambda x, _: converter.structure(x, PredictedInstance),
     # )
 
     # We can register structure hooks for point arrays that do nothing

sleap/nn/inference.py

@@ -4238,7 +4238,7 @@ def export_cli():
     )
 
     args, _ = parser.parse_known_args()
-    export_model(args["models"], args["export_path"])
+    export_model(args.models, args.export_path)
 
 
 def _make_cli_parser() -> argparse.ArgumentParser:
@@ -4620,9 +4620,9 @@ def main(args: list = None):
                 free_gpu_memory = sleap.nn.system.get_gpu_memory()
                 if len(free_gpu_memory) > 0:
                     gpu_ind = np.argmax(free_gpu_memory)
+                    mem = free_gpu_memory[gpu_ind]
                     logger.info(
-                        f"Auto-selected GPU {gpu_ind} with {free_gpu_memory} MiB of "
-                        "free memory."
+                        f"Auto-selected GPU {gpu_ind} with {mem} MiB of free memory."
                     )
                 else:
                     logger.info(

sleap/nn/training.py

@@ -1947,9 +1947,9 @@ def main():
                 free_gpu_memory = sleap.nn.system.get_gpu_memory()
                 if len(free_gpu_memory) > 0:
                     gpu_ind = np.argmax(free_gpu_memory)
+                    mem = free_gpu_memory[gpu_ind]
                     logger.info(
-                        f"Auto-selected GPU {gpu_ind} with {free_gpu_memory} MiB of "
-                        "free memory."
+                        f"Auto-selected GPU {gpu_ind} with {mem} MiB of free memory."
                     )
                 else:
                     logger.info(

tests/gui/test_suggestions.py

@@ -81,11 +81,13 @@ def test_frame_increment(centered_pair_predictions: Labels):
     print(centered_pair_predictions.videos)
 
 
-def test_video_selection(centered_pair_predictions: Labels):
+def test_video_selection(
+    centered_pair_predictions: Labels, small_robot_3_frame_vid: Video
+):
     # Testing the functionality of choosing a specific video in a project and
     # only generating suggestions for the video
 
-    centered_pair_predictions.add_video(Video.from_filename(filename="test.mp4"))
+    centered_pair_predictions.add_video(small_robot_3_frame_vid)
     # Testing suggestion generation from Image Features
     suggestions = VideoFrameSuggestions.suggest(
         labels=centered_pair_predictions,
@@ -150,6 +152,18 @@ def test_video_selection(centered_pair_predictions: Labels):
         # Confirming every suggestion is only for the video that is chosen and no other videos
         assert suggestions[i].video == centered_pair_predictions.videos[0]
 
+    # Ensure video target works given suggestions from another video already exist
+    centered_pair_predictions.set_suggestions(suggestions)
+    suggestions = VideoFrameSuggestions.suggest(
+        labels=centered_pair_predictions,
+        params={
+            "videos": [centered_pair_predictions.videos[1]],
+            "method": "sample",
+            "per_video": 3,
+            "sampling_method": "random",
+        },
+    )
+
 
 def assert_suggestions_unique(labels: Labels, new_suggestions: List[SuggestionFrame]):
     for sugg in labels.suggestions: