LoRA and QLoRA: Finetuning with less memory
Why do we need LoRA and QLoRA? Say I have a pretrained language model and want to adapt it to my own dataset. Maybe I want it to answer questions in a particular domain or follow a specific response format. I can fine-tune it, but how much GPU memory will that take? With full fine-tuning, we update all the model’s weights. Those weights are only part of the memory bill: training also needs gradients, optimizer states, and intermediate activations. A model that fits on my GPU for inference may still be too large to fine-tune on it. ...