Hi everyone,
I’m currently fine-tuning the parameters involved in box fusion when using tiled inference with the DeGirum SDK. I’m running into an issue where overlapping boxes are not being merged correctly, especially near tile borders.
I’m using the following setup:
(…your setup here…)
# detection model
model = dg.load_model(
model_name,
hw_location,
model_zoo_url,
"",
overlay_show_labels=True,
overlay_show_probabilities=True,
overlay_line_width=1,
output_confidence_threshold=.10,
output_class_set={"person", "car", "truck", "bus", "motorcycle", "bicycle"}
)
# tiling (inside)
tile_extractor = TileExtractorPseudoModel(
cols=3,
rows=1,
overlap_percent=0.05,
model2=model,
global_tile=True, # required for the BoxFusion
)
tiled_detector = BoxFusionLocalGlobalTileModel(
model1=tile_extractor,
model2=model,
nms_options=NmsOptions(
threshold=0.50,
box_select=NmsBoxSelectionPolicy.MERGE,
),
edge_threshold=0.02,
fusion_threshold=0.95,
large_object_threshold=0.02,
add_model1_results=True,
)
# extract only the region of interest
roi_pseudo = RegionExtractionPseudoModel(
roi_list=[bounding_extent(rois)],
model2=tiled_detector,
)
# crop to the region of interest and use tiling inside that region
tiled_model = CroppingAndDetectingCompoundModel(
roi_pseudo,
tiled_detector,
add_model1_results=True,
)
tiled_ai_model = dgstreams.AiSimpleGizmo(tiled_model, stream_depth=stream_depth, allow_drop=allow_drop)
Problem Description
In the image, several detections should clearly be fused, but aren’t:
➀ & ➁ Duplicate detections Local and Global tiles
Bullets ➀ and ➁ show that the same vehicle is detected twice: once as a truck and once as a car.
I suspect one detection comes from a local tile, and the other from the global tile (or vice-versa).
The IoU between these two boxes is almost ~1.0, so they should be merged.
➂ Small partial tile detection should be merged
Bullet ➂ shows a tiny bounding box from a neighboring tile that falls completely inside the larger box from bullets ➀ or ➁.
This box is not being fused either, while logically it should merge into the parent box.
➃ Sheriff car: green box should merge with orange box ➄
In bullet ➃, the bright green box (local tile) and the orange box ➄ (another tile/global) are clear duplicates.
Again, these should be fused given their near-total overlap.
My question
How should I tune the box-fusion parameters so that:
- The local global duplicate detections in 1 and 2 are merged properly (high IoU ~1.0).
- The small inner box in 3 becomes fused into the larger parent box.
- The two sheriff-car boxes in 4 and 5 also merge into a single box.
Any help or best practices for optimizing tile-aware box fusion would be greatly appreciated!
