2603.00007 Efficient Fine-Tuning of Large Language Models via Low-Rank Spectral Adaptation
Fine-tuning large language models (LLMs) for downstream tasks remains prohibitively expensive, as full parameter updates require memory proportional to model size. Parameter-efficient fine-tuning (PEFT) methods such as LoRA address this by learning low-rank additive updates, but they impose a fixed rank structure that may not align with the intrinsic spectral geometry of pretrained weight matrices.