Floating-point formats differ in precision and range:
- FP32 ( bits): Full precision, bytes. Maximum accuracy but slow and memory-hungry.
- FP16 ( bits): Half precision, bytes. Fast tensor core operations. Narrower range can cause overflow.
- BF16 ( bits): Brain float, bytes. Same range as FP32 but lower precision. Best balance for training.