I’d like to share some impressions about different Yolo flavors and HAILO8 (Rpi+HAILO8 26TOPS version)
I compiled all the five trained yolo11*.tp (downloaded from utralytics.com) flavors with DeGirum Cloud Compiler and the compilation was ok.
Then I started comparing performance (hailo monitor) and accurancy (expectetions and score in image overlay/meta)
From yolo11n to yolo11s I got little bit more accurancy and the performances are ok with 10 fps from rtsp.
With yolo11s 10 fps and 16% Hailo cpu is OK to run also other cameras
With yolo11m 10 fps and 30% Hailo cpu is also OK but I stepped down to 5 fps to stay with 15% Hailo CPU.
With yolo11m I got better accurancy and less false detection then yolo11s.
Until this point is pretty clear for me the curve accurancy and performance, better accurancy and more CPU.
Starting with yolo11l and yolo11x the curve changes.
With yolo11x 50% Hailo cpu and 5 fps seems ok but starting with yolo11l and yolo11x the accurancy is worst with false detection e different scores then yolo11m.
I will stick with yolo11m for now, performance and accurancy are coerent then large and x flavors
Are there some tuning to apply for yolo11 large and x or they are failing models?
Glad to see you could compile all the 5 models. Also, thanks for sharing your observations on the performance. The peal performance of yolo11n and yolo11s should be much higher than 10FPS, but it looks like you want to keep utilization low.
Regarding yolo11l and yolo11x: it is known that larger models (m,l and x) need finetuning to retain accuracy after quantization. If you really want to use those models, you can get the precompiled hefs from hailo model zoo where they already finetuned the larger models. Meanwhile, we will internally benchmark mAP of these models and let you know if they are actually worse than the smaller models. @lawrence and @nikita Can you please take a look?
We compiled the same checkpoints from the official Ultralytics repository using the Cloud Compiler, with the default suggested settings. Below are the accuracy results we get for yolo11 models starting with m:
Model Architecture
mAP 50-95 on Hailo-8
yolo11m
49.0
yolo11l
51.2
yolo11x
52.6
From this data, we see that accuracy increases as model size increases.
When comparing the visual results on a specific image, yolo11l and yolo11x detect most of the same objects that yolo11m detect, and also detects some objects that yolo11m does not. The scores of these detections for yolo11x and yolo11l are not always smaller than those of yolo11m.
