gpu memory and latency time

[StephanPan]

During the model warm-up, especially in the initial steps, there is significant fluctuation in GPU memory, and the process takes a considerable amount of time, reaching around twenty seconds. The GPU memory and the latency time is not that stable. Is there any suggestions please? thx.

[StephanPan]

Normally, GPU usage is around 2400 M, but occasionally, it may exceed 10 G.

[Ttayu]

Does this UserWarning appear?
You can turn it off below

UserWarning: operator() profile_node %385 : int[] = prim::profile_ivalue(%383) does not have profile information (Triggered internally at ../third_party/nvfuser/csrc/graph_fuser.cpp:104.)

torch._C._jit_set_profiling_executor(False)
torch._C._jit_set_profiling_mode(False)

[StephanPan]

Does this UserWarning appear? You can turn it off below

UserWarning: operator() profile_node %385 : int[] = prim::profile_ivalue(%383) does not have profile information (Triggered internally at ../third_party/nvfuser/csrc/graph_fuser.cpp:104.)

torch._C._jit_set_profiling_executor(False)
torch._C._jit_set_profiling_mode(False)

thanks，it works.