Failure to deploy first custom model: DeGirum exceptions in postprocessing, tensor shape condition not met

General
,
1

Hey guys, firstly id like to commend you all because Degirum is definetly a step in the right direction when it comes to AI deployment, simplifying the code down to a dozen lines.

So i have been learning AI for a bit now, and for me the Raspberry Pi and Hailo 8L are the focus of my studies, as i have been developing a few computer vision and such apps, a few of which i am open to sharing as tutorials on here to help inspire a few more tools and projects.

The current hurdle on my journey is the following error:

degirum.exceptions.DegirumException: [ERROR]Execution failed
Condition 'input_tensor->shape()[ 1 ] == 4 + m_OutputNumClasses' is not met: input_tensor->shape()[ 1 ] is 1, 4 + m_OutputNumClasses is 5
dg_postprocess_detection.cpp: 1307 [DG::DetectionPostprocessYoloV8::inputDataProcessBaseline]
When running model 'custom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/swt/MVP1.2.0/AI_Vision.py", line 232, in update_camera_view
    for result in self.model.predict_batch(self.frame_gen()):
  File "/home/swt/MVP1.2.0/venv/lib/python3.11/site-packages/degirum/model.py", line 289, in predict_batch
    for res in self._predict_impl(source):
  File "/home/swt/MVP1.2.0/venv/lib/python3.11/site-packages/degirum/model.py", line 1206, in _predict_impl
    raise DegirumException(
degirum.exceptions.DegirumException: Failed to perform model 'custom' inference: [ERROR]Execution failed
Condition 'input_tensor->shape()[ 1 ] == 4 + m_OutputNumClasses' is not met: input_tensor->shape()[ 1 ] is 1, 4 + m_OutputNumClasses is 5
dg_postprocess_detection.cpp: 1307 [DG::DetectionPostprocessYoloV8::inputDataProcessBaseline]
When running model 'custom'
./MyScript.sh: line 21:  4764 Aborted                 sudo /home/swt/MVP1.2.0/venv/bin/python entry_point.py

For the context of this i trained a custom Yolov8n model on my pc, converted it to onnx, and converted it to hef following this tutorial, my model only detects a single class, misplaced components on a defective pcb, and it was trained in the standard 640x640, on images i took from my HQ Raspberry pi Camera, and i am running it on my RPI 5 with 8GB of ram, and the hailo8l hat.

Software details also include a hailo 4.19.0 version and a degirum PySDK 0.15.0

My JSON file is as follows:

{
  "ConfigVersion": 10,
  "DEVICE": [
    {
      "DeviceType": "HAILO8L",
      "RuntimeAgent": "HAILORT",
      "SupportedDeviceTypes": "HAILORT/HAILO8L"
    }
  ],
  "PRE_PROCESS": [
    {
      "InputType": "Image",
      "ImageBackend": "opencv",
      "InputPadMethod": "letterbox",
      "InputResizeMethod": "bilinear",
      "InputN": 1,
      "InputH": 640,
      "InputW": 640,
      "InputC": 3,
      "InputQuantEn": true
    }
  ],
  "MODEL_PARAMETERS": [
    {
      "ModelPath": "custom.hef"
    }
  ],
  "POST_PROCESS": [
    {
      "OutputPostprocessType": "DetectionYoloV8",
      "OutputTopK": 1,
      "OutputNumClasses": 1,
      "OutputClassIDAdjustment": 1,
      "LabelsPath": "labels.json"
    }
  ]
}

Running both

hailortcli run custom.hef

and

hailortcli parse-hef custom.hef

works correctly, however when attempting to run it with the mvp code, using a real time image from the hq camera is when i get the error posted.

I have posted the files to the following git in case there are more tests you can run which i am unaware of.

Thank you for your help, and i am sorry if this is user error, i am still rather new to this, and it seems so is Degirum as far as troubleshooting.

2

Hi @DAlexSanc
Thanks for reaching out. We are happy to assist you in this process.
It looks like the issue may be related to the compilation of the Hailo HEF file. Specifically, if NMS is included in the HEF compilation and handled by the Hailo8/8L chip, then DetectionYoloV8 should not be used. Instead, a Python postprocessor should be implemented.
Here is the code for HailoDetectionYolo.py


import json


# Post-processor class, must have fixed name 'PostProcessor'
class PostProcessor:
    def __init__(self, json_config):
        """
        Initialize the post-processor with configuration settings.

        Parameters:
            json_config (str): JSON string containing post-processing configuration.
        """
        # Parse the JSON configuration
        self._json_config = json.loads(json_config)

        # Extract configuration parameters
        self._num_classes = int(
            self._json_config["POST_PROCESS"][0]["OutputNumClasses"]
        )
        self._label_json_path = self._json_config["POST_PROCESS"][0]["LabelsPath"]
        self._input_height = int(self._json_config["PRE_PROCESS"][0]["InputH"])
        self._input_width = int(self._json_config["PRE_PROCESS"][0]["InputW"])

        # Load label dictionary from JSON file
        with open(self._label_json_path, "r") as json_file:
            self._label_dictionary = json.load(json_file)

        # Extract confidence threshold
        self._output_conf_threshold = float(
            self._json_config["POST_PROCESS"][0].get("OutputConfThreshold", 0.0)
        )

    def forward(self, tensor_list, details_list):
        """
        Process the raw output tensor to produce formatted JSON results.

        Parameters:
            tensor_list (list): List of tensors from the model.
            details_list (list): Additional details (unused in this example).

        Returns:
            str: JSON-formatted string containing detection results.
        """
        # Initialize results list
        new_inference_results = []

        # Extract and reshape the raw output tensor
        output_array = tensor_list[0].reshape(-1)

        # Index to parse the array
        index = 0

        # Iterate over classes and parse results
        for class_id in range(self._num_classes):
            # Number of detections for this class
            num_detections = int(output_array[index])
            index += 1  # Move to the next entry

            # Skip if no detections for this class
            if num_detections == 0:
                continue

            # Process each detection for this class
            for _ in range(num_detections):
                if index + 5 > len(output_array):
                    # Safeguard against unexpected array end
                    break

                # Extract score and bounding box in x_center, y_center, width, height format
                score = float(output_array[index + 4])
                y_min, x_min, y_max, x_max = map(float, output_array[index : index + 4])
                index += 5  # Move to the next detection

                # Skip detections below the confidence threshold
                if score < self._output_conf_threshold:
                    continue

                # Convert to x_min, y_min, x_max, y_max format
                x_min = x_min * self._input_width
                y_min = y_min * self._input_height
                x_max = x_max * self._input_width
                y_max = y_max * self._input_height

                # Format the detection result
                result = {
                    "bbox": [x_min, y_min, x_max, y_max], 
                    "score": score,
                    "category_id": class_id,
                    "label": self._label_dictionary.get(
                        str(class_id), f"class_{class_id}"
                    ),
                }
                new_inference_results.append(result)

            # Stop processing if padded zeros are reached
            if index >= len(output_array) or all(v == 0 for v in output_array[index:]):
                break

        # Return results as JSON string
        return json.dumps(new_inference_results)

You need to create a python file “HailoDetectionYolo.py” and put this file in the the same folder as hef and JSON.
The new JSON will look like this

{
  "ConfigVersion": 10,
  "DEVICE": [
    {
      "DeviceType": "HAILO8L",
      "RuntimeAgent": "HAILORT",
      "SupportedDeviceTypes": "HAILORT/HAILO8L"
    }
  ],
  "PRE_PROCESS": [
    {
      "InputType": "Image",
      "ImageBackend": "opencv",
      "InputPadMethod": "letterbox",
      "InputResizeMethod": "bilinear",
      "InputN": 1,
      "InputH": 640,
      "InputW": 640,
      "InputC": 3,
      "InputQuantEn": true
    }
  ],
  "MODEL_PARAMETERS": [
    {
      "ModelPath": "custom.hef"
    }
  ],
  "POST_PROCESS": [
    {
      "OutputPostprocessType": "DetectionYoloV8",
      "PythonFile": "HailoDetectionYolo.py",
      "OutputNumClasses": 1,
      "LabelsPath": "labels.json"
    }
  ]
}

Additionally, we are in the process of releasing a web-based cloud compiler tool. This tool will support the compilation of various runtimes, including Hailo, directly into the DeGirum model zoo.

Let me know if this helped.

3

Hi @DAlexSanc

Hey guys, firstly id like to commend you all because Degirum is definetly a step in the right direction when it comes to AI deployment, simplifying the code down to a dozen lines.

Thank you for your kind words.

So i have been learning AI for a bit now, and for me the Raspberry Pi and Hailo 8L are the focus of my studies, as i have been developing a few computer vision and such apps, a few of which i am open to sharing as tutorials on here to help inspire a few more tools and projects.

We are so happy to hear that you want to share tutorials. Let us know how we can help.

4

I created the folder with the changes mentioned in the previous response.
model folder

Here is the cloud inference result on the image in your shared repository.

Screenshot 2025-02-20 at 9.38.46 PM
Screenshot 2025-02-20 at 9.38.46 PM1204×1170 70.1 KB

5

I see, i will make a note of this thank you @khatami.mehrdad for your help!