Hailo Official integration #16906
Hailo Official integration #16906
Conversation
✅ Deploy Preview for frigate-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
Hi, thank you for the pull request! Please look at #10717, we will not be able to accept some of these changes |
frigate/detectors/detector_config.py
Outdated
| @@ -38,6 +38,7 @@ class ModelTypeEnum(str, Enum): | |||
| yolov9 = "yolov9" | |||
| yolonas = "yolonas" | |||
| dfine = "dfine" | |||
| hailoyolo = "hailo-yolo" | |||
There was a problem hiding this comment.
We don't want these to be detector specific, I'd suggest just using yolo 9 as we may change that to be called yolo-generic or something in the future
There was a problem hiding this comment.
Hey Nick,
We've made this change to support all the models in our model zoo, not just YOLO. Any model, regardless of its configuration, can be run in Frigate as long as its post-processing is done on Hailo (which happens during the model compilation).
Regarding the default SSD MobileNet, it uses a 300x300 input size. Resizing from 320x320 to 300x300 takes more time than simply moving from 320x320 to 640x640. To optimize this, I've updated the detector to perform the resizing (so you can run Frigate with a 320x320 configuration) and use YOLOv6n with a 640x640 input size.
In summary, we now use the default 320x320 configuration for most models, and we only resize and run YOLOv6n with a 640x640 input size.
There was a problem hiding this comment.
Right, but this is a mixing of concerns, because now we have detectors that rely on the model type and model type that relies on the detector.
We either need the different model types to be listed out, or just have the Hailo detector ignore the configured model type when using a model from the Hailo zoo
There was a problem hiding this comment.
did all the required changes , and improved performance by 3x , we need just to change something that is not correct and will re push
|
|
||
| The inference time for the Hailo-8L chip at time of writing is around 17-21 ms for the SSD MobileNet Version 1 model. |
There was a problem hiding this comment.
We should leave this in here, perhaps create a table like the other hardware sections so users know what to expect.
I am happy to do that separately if you can provide some inference times for the yolov6 model used here
There was a problem hiding this comment.
Can you explain how the detector inference speed and Detector CPU usage is measured in your system metrics page?
We removed the table because we saw a mismatch between the inference speed measured in your metrics and the one we measure.
There was a problem hiding this comment.
It is the total time of inference including image preprocessing and detection post processing
|
Looks like ruff needs to be run |
| class HailoDetectorConfig(BaseDetectorConfig): | ||
| type: Literal[DETECTOR_KEY] | ||
| device: str = Field(default="PCIe", title="Device Type") | ||
| #url: Optional[str] = Field(default=None, title="Custom Model URL") |
There was a problem hiding this comment.
This commented line can just be removed if it's unused.
We are now standardizing on asynchronous inference to improve performance and responsiveness, while also upgrading our detection models to the latest YOLO variants—YOLOv8s for Hailo-8 and YOLOv6n for Hailo-8L—both optimized for 640x640 processing. In addition, documentation and configuration guidance have been updated to reflect these changes.
Key Changes:
Async Inference Integration:
Migrated our inference pipeline to use asynchronous inference as the new standard.
Improved throughput and reduced latency by decoupling input preprocessing from model execution.
Model Upgrades:
Updated defaults to use YOLOv8s on Hailo-8 and YOLOv6n on Hailo-8L.
These models are optimized for 640x640 input dimensions, delivering high accuracy and performance (320 FPS for YOLOv8s and ~300 FPS for YOLOv6n).
Detection Post-Processing Enhancements:
Revised the detect_raw function to leverage the outer loop index to determine the class label, aligning with our standard detection extraction method.
`