deduplicate configuration and ensure it works on enumerate_camera_devices

[Lunarequest]

Linux

enumerate_camera_devices previously did not check if a camera configuration worked. This resulted in users being presented with a set of configurations, some of which did not work. enumerate_camera_devices now attempted to capture a frame. If this does not work the configuration is not returned.

This greatly reduces the number of "false"/non working configurations. However there are some configurations which do not track but are still appended, I am unsure how to test these without requiring the camera be pointed at a person on startup.

before:

after:

windows

On windows CAP_ANY returns 2 "entries" for every camera. Direct Show(DSHOW) and Media Foundation (MSMF) MSMF was introduced in windows vista and DSHOW has since been deprecated and its docs carry the warning it maybe ripped out in a future windows version. For future readiness only query by CAP_MSMF since that provides windows support

[Drikanis]

Windows testing

Before PR

CAP_ANY:700.0 is the working camera on my system, starts within seconds of selecting it, and doesn't drop any packets.

After PR:

First thing I noticed is that I get a version warning on loading the project, since SnekStudio is still on 4.4 but the project in the PR was last modified using 4.5. Not sure how much that matters though.

The virtual cams are missing, which should technically still work but aren't my main camera, so might not be an issue for most people?

Selecting the only camera available (MSMF:0.0) takes a really long time to start tracking, but it eventually starts after a few minutes. It's also constantly dropping packets whereas the main version does not.

[Lunarequest]

@Drikanis could you test with this commit

also with this patch applied

diff --git c/Mods/MediaPipe/_tracker/Project/new_tracker.py w/Mods/MediaPipe/_tracker/Project/new_tracker.py
index 2f0ebba..1bfc7f0 100644
--- c/Mods/MediaPipe/_tracker/Project/new_tracker.py
+++ w/Mods/MediaPipe/_tracker/Project/new_tracker.py
@@ -12,6 +12,8 @@ import threading
 import time
 import traceback
 
+os.environ["OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS"] = "0"
+
 import cv2
 import kiri_math
 import mediapipe
@@ -796,7 +798,7 @@ def enumerate_camera_devices():
         capture_api_preference = cv2.CAP_V4L2
 
     if sys.platform == "win32":
-        capture_api_preference = cv2.CAP_DSHOW
+        capture_api_preference = cv2.CAP_MSMF
 
     # On Linux, we sometimes see stuff showing up as just "video#", so
     # let's at least try to correlate paths and IDs from

[Drikanis]

commit 4312946

Tracking still takes a really long time to start (1-2 minutes), and appears to be dropping packets consistently once it starts.

after patch

Tracking starts immediately, but still appears to be dropping packets consistently.

[Lunarequest]

Tracking starts immediately, but still appears to be dropping packets consistently.

I'm not sure if this is actually an issue because this has been how things have been for me from day one. Kiri would need to confirm.

[Lunarequest]

After a little more research what I believe is happening is the following, without OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS being set to 0 causes the MSMF back end to stop using hardware accelerated transforms. I believe for some reason the HW transforms is slower then the software unaccelerated version mean the FPS from the camera exceeds what the hardware decoder provides? at least this is my theory but I have no way to test

[Drikanis]

Or CAP_ANY was slower than "direct" MSMF? If the dropped frames are benign (which it seems to be) then I guess they can be ignored.

Aside from the loss of DSHOW-exclusive devices and the version bump on the project file, it seems fine from a testing perspective.

[Lunarequest]

Oh that was my bad, I'll remove those changed files. If there's a similar issue of duplicate cameras on macos I can't fix it. Someone with a apple device would need fo. Other then that I believe this can be merged in?

[Lunarequest]

@ExpiredPopsicle can you look at this when you're free

[ExpiredPopsicle]

On my end this fails with...

E 0:00:06:219   KiriPythonWrapperInstance.gd:439 @ poll(): Call failed:
Traceback (most recent call last):
  File "/home/kiri/git/SnekStudio/Saved/_python_dist/packaged_scripts/addons/KiriPythonRPCWrapper/KiriPythonRPCWrapper/__init__.py", line 155, in <module>
    ret = func(*func_args)
          ^^^^^^^^^^^^^^^^
  File "/home/kiri/git/SnekStudio/Saved/_python_dist/packaged_scripts/Mods/MediaPipe/_tracker/Project/new_tracker.py", line 849, in enumerate_camera_devices
    from linuxpy.video.device import Device, VideoCapture
ModuleNotFoundError: No module named 'linuxpy'

  <C++ Source>  core/variant/variant_utility.cpp:1098 @ push_error()
  <Stack Trace> KiriPythonWrapperInstance.gd:439 @ poll()
                KiriPythonWrapperInstance.gd:373 @ call_rpc_sync()
                MediaPipeController.gd:359 @ _scan_video_devices()
                MediaPipeController.gd:100 @ _ready()
                Main.gd:505 @ deserialize_settings()
                Main.gd:621 @ load_settings()
                Main.gd:107 @ _ready()

[ExpiredPopsicle]

This also intentionally breaks DSHOW compatibility, which I would like to leave in solely because DSHOW works with WINE while MSMF did not, the last time I checked. I don't know who's running SnekStudio under WINE and why they would do it, but I don't like the idea of intentionally removing a functioning fallback.

[Lunarequest]

@ExpiredPopsicle messed up reverting a change and now there's a updated platform_status.json which should fix the issue you were having. I've also added a fallback for DSHOW. if we can find any cameras via MSMF we then enumerate via DSHOW. I am considering adding a ENV_VAR like "FORCE_CAP_DSHOW" and/or "FORCE_CAP_ANY" to allow advanced users to force using DSHOW or ANY based on their requirements.