Evaluation of YOLO-OBB Model Fails in degirum_tools

General
, ,
1

Hi ,

I’m trying to evaluate a YOLO-OBB (oriented bounding box) model:

  • Model: hrsid-obb--640x640_quant_hailort_hailo8_3

  • Label format: class cx cy w h angle

  • Converted annotations to COCO format

I can run batch_predict successfully, but when I try:

import degirum as dg
from degirum_tools.detection_eval import ObjectDetectionModelEvaluator

model = dg.load_model(
    model_name="hrsid-obb--640x640_quant_hailort_hailo8_3",
    inference_host_address="@cloud",
    zoo_url="sivagnanam_maheshwaran/Hailo8_Deployyment",
    token="<token>"
)

evaluator = ObjectDetectionModelEvaluator(model)
results = evaluator.evaluate(
    "extracted_content/hundred_labelled_images/corresponding_images",
    "ground_truth_coco.json",
    max_images=100
)

I get:

Exception: Model loaded for evaluation is not a Detection Model

The model config shows "OutputPostprocessType": "DetectionYoloV8OBB", which is not recognized by ObjectDetectionModelEvaluator (expects YOLOv5/YOLOv8 axis-aligned or segmentation models).

Questions:

  1. Is there a way to evaluate YOLO-OBB models natively with degirum_tools?

  2. If not, what is the recommended workflow for computing metrics like mAP / precision / recall

Thanks!

2

Hi @Mahesh

Thanks for bringing this to our attention and catching one more corner case :slight_smile: . We are aware of this and are working on a fix. We will keep you posted. In the meantime, I will see if we can provide you a script that can help.

1 Like
3

Hi @shashi may I ask if the script is ready or where I could find the documentation to make my own one?

4

Hi @Mahesh

Apologies for the delay. We will share the code shortly.

5

Hi @Mahesh,

You can find the evaluation code with an example here in our ultralytics fork.

Note that the annotation format is custom as COCO does not have the OBB task.