##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
Modern LLMs typically go through:
Pre-training on raw text (learn language)
Supervised fine-tuning on instructions (learn to follow directions)
Preference alignment (learn to respond well)
Each stage builds on the previous. You can apply alignment to any SFT model. The techniques in this section are that third stage.
