ERC-SVD: Error-Controlled SVD for Large Language Model Compression

Haolei Bai^1,2, Siyong Jian^1,4, Tuo Liang³, Yu Yin³, Huan Wang^1,*

¹Westlake University, ²Nanyang Technological University, ³Case Western Reserve University, ⁴Nanjing University

🚀 Usage

1. Install dependencies

conda create -n erc-svd python=3.10
conda activate erc-svd
pip install -r requirements.txt

2. Perform SVD compression

# loss analysis (compression ratio=20% for example)
python ERC-SVD_compression.py --model /yout/path/to/llama-2-7b-hf/ --step 0 --ratio 0.2 --whitening_nsamples 256 --dataset wikitext2 --seed 3 --model_seq_len 2048 --layer_num 16

python ERC-SVD_compression.py --model /yout/path/to/llama-2-7b-hf/ --step 1 --ratio 0.2 --whitening_nsamples 256 --dataset wikitext2 --seed 3 --model_seq_len 2048 --layer_num 16 --save_path /your/save/path/

3. Evaluation

3.1 Perplexity

python ERC-SVD_compression.py --step 2 --model_path /your/path/to/compressed_model.pt

3.2 Zero-shot accuracy

python evaluation_zero_acc.py --model_id /your/path/to/compressed_model.pt --pt_flag

🙌 Acknowledgement

We are grateful to the ASVD and SVD-LLM for releasing their code publicly, which greatly facilitated our work.

📖 Citation

If you find ERC-SVD useful for your research or projects, please consider citing our work:

@inproceedings{bai2026ercsvd,
  title={ERC-SVD: Error-Controlled SVD for Large Language Model Compression},
  author={Bai, Haolei and Jian, Siyong and Liang, Tuo and Yin, Yu and Wang, Huan},
  booktitle={CPAL},
  year={2026}
}
@article{bai2025ressvd,
  title={Ressvd: Residual compensated svd for large language model compression},
  author={Bai, Haolei and Jian, Siyong and Liang, Tuo and Yin, Yu and Wang, Huan},
  journal={arXiv preprint arXiv:2505.20112},
  year={2025}
}