Inferencce time on Raspberry PI5

General
1

There are 2 ways of my inference the model on HAILO 8L with Raspberry PI5.

1 - using Raspberry PI as degirum server - all logic and all preparations are done on my PC. In this case I have inference speed 44-45 fps.

2 - using Raspberry for running all code. In this case the inference speed is 24-25 fps.

I belive this is due PRE and POST processes, running on RAspberry.

In my case I have video from IP camera, I transform this video via ffmpeg to grayscale and resize it to 640x640.

My question is - may I avoid PRE and POST processes by prepearing video by ffmpeg to appropreate format. For my work I need just Object ID and its bbox (on each frame).
My goal is to increase fps.

As the ffmpeg is the nessesary step - I want to do by this step all works for image preparation - to make inference task as light and fast as possible.

Thank you for advance!

2

Hi @zoomrenew

Can you share your code where you do the IP camera to video conversion and resizing?

3 
 command_bw = (f'c:/ffmpeg/bin/ffmpeg   -i {in_folder}\\{file_name}   -vf "scale=640:640, fps=25, format=gray" -an -c:v libx264rgb -b:v 2000k  -ss {t_start} -to {t_stop} {out_folder_bw}\\{fn_part[0]}_stp{num}.mp4')
4

Hi @zoomrenew

Thanks for sharing. Can you explain a little more about your setup? When you say you have a video from a IP camera, is it a stream or an actual video file? If you already have a video file, you can directly pass it to predict_streamfunction.

5

I need to count passangeers in city bus. The bus equippedby IP camera. The stream from camera is written to a disk. When stop occurs, I fix start_stop_time and end_stop_time.

Then I cut out a part of main video file from start_stop_time - 3 sec to end_stop_time + 3 sec.

This vedeo file now is ready to be processed.

The code, I’ve shared - is code for split and transform video file.

As I need as INPUT 640 x 640 resolution, I resize video to meet this. For reduceing the diffeence between day time and night time, I convert video to GRAYSCALE.

So. as an INPUT I have short video files, transformed as you can see.

for inference_result in predict_stream(model_hailo, video_source):

            frame = inference_result.image_overlay
            cv2.polylines(frame, np.int32([lines[0]]), isClosed=False, color=(0, 255, 255), thickness=2)
            cv2.polylines(frame, np.int32([lines[1]]), isClosed=False, color=(0, 155, 255), thickness=2)
            cv2.polylines(frame, np.int32([lines[2]]), isClosed=False, color=(255, 255, 0), thickness=2)
            cv2.polylines(frame, np.int32([lines[3]]), isClosed=False, color=(255, 150, 150), thickness=2)
            output_display.show(frame)
            wr.write(frame)
            
            for item in  inference_result.results:
                if 'bbox' and 'track_id' in item:
                    bbox,  track_id = item["bbox"], item["track_id"]
                    track = track_history[track_id]
                    center_x = bbox[0] + (bbox[2] - bbox[0]) / 2
                    center_y = bbox[1] + (bbox[3] - bbox[1]) / 2
                    track.append((float(center_x), float(center_y)))  # x, y center point
                    if cv2.waitKey(1) & 0xFF == ord("q"):
                        break

This is my inference code.

After that I do my own - for counting.

6

@zoomrenew , there is (almost) drop-in replacement for RPi5: Radxa Rock5C (Radxa ROCK 5C). It is more powerful AND less expensive. Actually, any system based on RK3588 will outperform RPi5. And that RK3588 has built-in NPU, which is supported by PySDK and has decent performance, so you will not need any extra accelerator.

Also, in your code, frame = inference_result.image_overlay call is expensive: it creates new image and draws AI annotations. cv2.polylines calls are also expensive. If you do not need any live video preview, better to disable it.

Regarding pre-processing. If you provide your images already resized to the model input size, PySDK pre-processing will skip resizing, which is the most time-consuming step.

If AI server runs on the same host as client, you may also skip conversion to JPEG and back. To do it, you set model.input_image_format = "RAW".

7

Thank you very much!

Your answer was very helpful for me!

1 Like
8

@zoomrenew we’re glad to hear that our team was able to help you! Could you mark the solution in the thread (icon with the checkbox) so others could easily reference it?