Occasionally, you will run into errors in quantization where the dynamic range of a quantized layer will not fit the input range for the layer. Normally, the compiler will try to automatically adjust to fit these ranges. However, sometimes the values will have large outliers which expand the range such that they will not fit the quantization parameters. It is possible that these large input ranges are generated from a non representative dataset.
By default, our cloud compiler uses a small subset of the COCO dataset to generate quantization statistics. To upload a representative dataset (choose images from your training set), click the advanced options in the cloud compiler and upload 64 to 100 images. If this doesn’t work, it is possible it is an inherent property of your checkpoint. This can occur if you do not use regularization while training (e.g. weight decay).