[Usage] running problem:when i set ratio =1,problme disappear! however,when i set other ratio value,such as 0.1,problme appear

[Zhan0924]

Describe the issue

Issue: running problem:when i set ratio =1,problme disappear! however,when i set other ratio value,such as 0.1,problme appear

Command:

bash scripts/v1_5/eval/textvqa.sh

Log:

....
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [122,0,0], thread: [123,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [122,0,0], thread: [124,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [122,0,0], thread: [125,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [122,0,0], thread: [126,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [122,0,0], thread: [127,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
Traceback (most recent call last):
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/zqk/AdaptPrune-main/llava/eval/model_vqa_loader.py", line 144, in <module>
    eval_model(args)
  File "/home/zqk/AdaptPrune-main/llava/eval/model_vqa_loader.py", line 105, in eval_model
    output_ids = model.generate(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/zqk/AdaptPrune-main/llava/model/language_model/llava_llama.py", line 146, in generate
    return super().generate(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/transformers/generation/utils.py", line 1479, in generate
    return self.greedy_search(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/transformers/generation/utils.py", line 2340, in greedy_search
    outputs = self(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/zqk/AdaptPrune-main/llava/model/language_model/llava_llama.py", line 100, in forward
    return super().forward(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1183, in forward
    outputs = self.model(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/zqk/AdaptPrune-main/llava/model/language_model/adaptprune.py", line 242, in forward
    layer_outputs = decoder_layer(
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 795, in forward
    hidden_states = self.input_layernorm(hidden_states)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/data/zqk/miniconda3/envs/adaptprune/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 115, in forward
    variance = hidden_states.pow(2).mean(-1, keepdim=True)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Screenshots: