##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
Full fine-tuning of a B model needs GB+ of VRAM. Most practitioners don't have access to that hardware.
LoRA (Low-Rank Adaptation) solves this by freezing the original model and training small adapter matrices instead. You get %+ of full fine-tuning quality at a fraction of the memory cost. A B LoRA fine-tune fits on a GB consumer GPU.
