Model porting guide: Hosted and local compiler user guide

Guides PySDK Basics
1

Compilation of PyTorch checkpoints using DeGirum hosted compiler UI

Since PyTorch checkpoints do not contain the model structure, they cannot be compiled directly. To address this, we have enabled direct compilation from PyTorch checkpoints for models generated from commonly implemented repositories such as Ultralytics and YOLOv5.

Steps to compile PyTorch checkpoints

  1. Visit the AI Hub: Go to hub.degirum.com and log in with your credentials.
  2. Create a new model zoo: Click on “New Model Zoo” to create your model zoo.
  3. Navigate to Models >> Compiler: Go to the Cloud Compiler section in the UI.
  4. Upload Your PyTorch checkpoint:
    • Choose the appropriate compiler (YOLOv5 or YOLOv8 (Ultralytics)) based on your model architecture.
    • Upload your model checkpoint along with the necessary compiler input options.
  5. Select output model zoo: Choose the model zoo you created in Step 2 as the destination for the compiled model.
  6. Press Compile: Click Compile and wait a few minutes for the process to complete.

Once the compilation process is finished, your model will be available in your model zoo. You can test it out on hosted inference or use it through the PySDK. For examples of how this can be done, visit our PySDK Examples.

Compilation of ONNX models using our local compiler

Compiling ONNX models is done using the provided Docker on your local machine. For detailed instructions on using this Docker, refer to the link below:

Once compiled, you can upload the model (zip file) directly to your hosted model zoo.

1 Like
2

I was able to quantize the model using cloud compiler on degerium ai hub. But the model does not perform well.Is there any piece of documentation that allows us to tweak the parameters during compilation. like when i wa susing hailo_sdk_client to quantize the mode, it would go something like:

alls = “”“normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])change_output_activation(conv74, sigmoid)change_output_activation(conv90, sigmoid)change_output_activation(conv105, sigmoid)nms_postprocess(“/local/shared_with_docker/nms_config/nms_layer_config_yolo11m.json”, meta_arch=yolov8, engine=cpu)model_optimization_config(calibration, batch_size=1)performance_param(compiler_optimization_level=max)“””

performance_param(compiler_optimization_level=max) - To achieve optimal performance, set the compiler_optimization_level to “max”

post_quantization_optimization(finetune, policy=enabled, learning_rate=0.00001)

resources_param(max_apu_utilization=0.8, max_compute_16bit_utilization=0.8, max_compute_utilization=0.8, max_control_utilization=0.8, max_input_aligner_utilization=0.8, max_memory_utilization=0.8, max_utilization=0.0)

model_optimization_flavor(optimization_level=0)

runner.load_model_script(alls)runner.optimize_full_precision()

How can i debug compilation using degerium. does pysdk for it support it?

3

Hi @muhammadhamzaj2001, it looks like @shashi was able to respond in the thread here: How to debug compilation on degerium ai hub if accuracy is not good - #4 by shashi