CPU Memory Consumption Reduction

[narendasan]

[cehongwang]

In Qwen, TRT builder uses 1x to build live engine.
Sometimes TRT builder uses more than 2x even for building live engine: FLUX, GPT-OSS

[cehongwang]

INetworkDefinition does not take any memory actually, it is the lowered graph and constand folding takes the up to 1x (0-1x) memory. Code here:

TensorRT/py/torch_tensorrt/dynamo/lowering/passes/constant_folding.py

Line 39 in 763fd08

replace_node_with_constant(

INetwork is just hold the reference to the weights in the lowered graph