##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
Quantization reduces the precision of weights. Instead of bits per value, you use bits.
-bit: possible values per weight -bit: possible values per weight
This sounds like massive information loss, but careful quantization preserves most model quality. The key is choosing which values best represent the original distribution.
