Common issues and fixes:
Model not learning:
- Increase rank or alpha
- Check that target modules are correct for your model
Training unstable:
- Reduce learning rate
- Enable gradient clipping
Out of memory:
- Reduce batch size
- Enable gradient checkpointing
- Switch to QLoRA