Live detection optimization

General
, ,
1

How Can I Make My Pi Camera Object Detection Code More Responsive?

Hi everyone,

I’m working on a real-time object detection project using the DeGirum model with a Raspberry Pi camera and I’m looking to make the code more responsive particularly in terms of processing speed and display fluidity. Here’s my current code:

import degirum as dg
import degirum_tools
from picamera2 import Picamera2
import cv2

# Initialize Picamera2
camera = Picamera2()
camera.configure(camera.create_video_configuration(main={"format": "RGB888"}))
camera.start()

model = dg.load_model(
    model_name="yolo11n_silu_coco--640x640_quant_hailort_hailo8_1",
    inference_host_address="@local",
    zoo_url="/home/rohan/degirum_project/models"
)

with degirum_tools.Display("Object Detection Camera") as output_display:
    while True:
        frame = camera.capture_array("main")
        result = model.predict(frame)
        result._show_probabilities = True
        output_display.show(result)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

cv2.destroyAllWindows()

Any advice or suggestions would be greatly appreciated!

Thanks in advance.

1 Like
2

Hi @rohanpatrick568 ,

To greatly speedup the inference you need to use streaming prediction API: model.predict_batch, because single-frame synchronous prediction API model.predict() always waits for result, so no pipelining happens, which reduces FPS.

This streaming API accepts the frame iterator as argument, so we need to wrap the camera frame capturing code into a generator function source(), see code below.

Few more suggestions:

  1. I put overlay_show_probabilities = True into load_model call to enable showing probabilities for all results.
  2. I removed cv2.destroyAllWindows() call because with degirum_tools.Display will close its window.
  3. I removed if cv2.waitKey(1) & 0xFF == ord('q') call because output_display.show does exactly that and it throws KeyboardInterrupt exception on pressing q or x. This exception is then caught in degirum_tools.Display.__exit__ so it will not be visible to you.
def source():
    camera = Picamera2()
    camera.configure(camera.create_video_configuration(main={"format": "RGB888"}))
    camera.start()
    while True:
        frame = camera.capture_array("main")
        yield frame

model = dg.load_model(
    model_name="yolo11n_silu_coco--640x640_quant_hailort_hailo8_1",
    inference_host_address="@local",
    zoo_url="/home/rohan/degirum_project/models",
    overlay_show_probabilities = True
)

with degirum_tools.Display("Object Detection Camera") as output_display:
    for result in model.predict_batch(source()):
        output_display.show(result)
3 Likes