YOLOv8 / YOLOv11 Segmentation Example?

[bluejaelly]

Thanks for providing this repo! It helped me install onnxruntime for Unity. I'm trying to combine this with ARFoundation's XRCpuImage so that I can run YOLOv8 / YOLOv11 segmentation model in AR.

I believe I am quite close, but the bounding boxes and segmentation masks are not aligning well with the target objects... I'm happy to share code, if you think that'll help.

I was wondering if you have any YOLO segmentation examples that I can take a look at. I saw an example for YOLOX and SAM, but I was hoping to use YOLOv8/v11-seg models in Unity, as it's easier to fine-tune via Ultralytics.

Please let me know, thank you!

[asus4]

Hi @bluejaelly, The YOLOv11 segmentation will be a good demo for demonstrating the ONXXRuntime.

I can help it if you open a minimum PR to the onnxruntime-unity-examples repository.

Please use the VirtualTextureSource instead of the ARFonudation XRCpuImage since I want to keep the example as simple as possible. The VirtualTextureSource also supports AR Foundation, so you can easily integrate AR into the example.

[bluejaelly]

Hi @asus4! Thanks! I'll make a minimum PR to the onnxruntime-unity-examples repository. I have a somewhat working code with ARFoundation XRCpuImage, but your example with VirtualTextureSource would help a lot :))

[bluejaelly]

Hi @asus4! Thanks again for responding to my request. I have made an issue in the onnxruntime-unity-examples repository instead, as I ended up translating ONNXRuntime YOLOv8 Python example into C#. This works, but has some performance issues like false positives... hence I thought it'd be best if you can take a look. My code does not use VirtualTextureSource... Just ARFoundation XRCpuImage. If you'd like me to attempt to follow your code more closely, let me know. Happy to help build this example :))