Camera Object Classification

[varunpat789]

Feature Description
We need a way to detect and classify objects for both navigation purposes but also for task requirements. In simple terms, I envision our perception stack using LiDAR info to generate a rough map and detect the potential presence of an object, but camera feed to actually confirm the existence of an obstacle and classify it.

Potential Paths

• Go through the 2024/2025 handbook and compile a list of obstacles we need to classify (at the very least, buoys, symbols on targets, and maybe the docks)
• Determine if any preprocessing is needed
• Still figuring out what we want outputted for this, but I envision some custom service msg representing obstacle type and coordinates/bounding boxes

Constraints/Additional context

• Models can take up space/slow, so look into potential optimizations. For instance, if we don't need to use models for everything, and instead use other ideas (ex. color for buoys). Likewise, we are running this on an Orin and have the option to utilize its CUDA cores to accelerate this (ex. Open-CV CUDA, CV CUDA)

Tasks

• Create a list of objects required to be detected.
• Collect images of the designated objects.
• Label, annotate and train a model.

Refrences
https://universe.roboflow.com/roboboat-soe01/buoys-dkb8e/images/0LxpCdFree1ZmSStKWeo

[shovan777]

@varunpat789 I would like to contribute on this feature. I have some prior experience with yolo. But minimal experience with ros and gazebo. How should I proceed?

[shovan777]

@Lthompson406
Let's triain our model here for now:
https://colab.research.google.com/drive/1Z-mOIFsE3-X3LVaSZex8-RBEHPD1Eyfm

[Lthompson406]

Class Labels: Port Marker Buoy(Red), Starboard Marker Buoy(Green), Gate Buoy.