School of Computing Science

Our 2nd ICML paper: 'Advancing SVD-based LLM Compression via Layer-Wise Error Model Search' has been accepted in ICML 2026, Seoul, South Korea July 6th - 11th, 2026

This paper advances SVD-based LLM compression by introducing LEMS, a calibrated layer-wise rank-allocation method, and KFAC-SVD, a token-wise Fisher-based decomposition that avoids rank collapse. Across Mistral, Qwen3, and Llama models, it reports stronger compression quality, with up to 4.8 percentage-point zero-shot accuracy gains and around 15% average perplexity improvement over baselines.

First published: 7 May 2026