How to accelerate inference for lung nodule detection

[zhuofalinHJh]

How to accelerate inference for lung nodule detection

1. I used the code of this project to complete the training of the lung nodule detection model. Now I plan to convert it to an onnx model for inference acceleration, but I don’t know how to do it?
2. How can I accelerate the inference of lung nodule detection when the hardware remains unchanged?

---

My development environment

1. Windows 11 pro
2. nvidia info

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 576.52                 Driver Version: 576.52         CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4070 ...  WDDM  |   00000000:01:00.0  On |                  N/A |
| N/A   46C    P8              2W /  140W |     629MiB /   8188MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

3. monai: 1.4.0

[zhuofalinHJh]

@ericspod @tvercaut, can you help me? thanks

[ericspod]

Hi @zhuofalinHJh These are rather open ended questions and there's going to be a lot of factors going into improving performance. The training scripts for this project produce Torchscript model outputs, so if you haven't already take a lookt at the documentation on ONNX and conversion to see how to convert these or convert from Python objects in a running script with the loaded weights. For inference with this model, there are going to be a lot of factors again about performance but it depends on your data. If you do inference on only one loaded input at a time it will be slow and use little memory as you see, so you may have to adapt what you're doing to accelerate that with batching. There is a testing script to look at to see how inference works in the pipeline, but otherwise I'm not sure what other specific advice here I can give other than to seek out other tutorials on inference acceleration as well as look at the acceleration tutorials we have here. Perhaps @Can-Zhao @mingxin-zheng might have further insight.