ReFT (Representation Finetuning) is a Stanford research paper introducing a new parameter-efficient finetuning approach for large language models. Instead of modifying model weights like LoRA, ReFT edits hidden representations via small trainable intervention components inserted between transformer layers. The specific method introduced, LoReFT (Low-rank Linear Subspace ReFT), requires 10-50x fewer parameters than LoRA while achieving competitive or superior results on commonsense reasoning, arithmetic reasoning, and instruction-following benchmarks. LoReFT applies interventions to prefix and suffix tokens at selected layers, training matrices R, W and vector b to edit representations. One LoReFT variant achieved the best win-rate among evaluated open-source models on instruction following, trained in just 18 minutes on a single A100 GPU.
Sort: