Decision guide:
- Simple task, limited data: Prompt tuning or IA3
- Standard fine-tuning needs: LoRA or DoRA
- Memory constrained: QLoRA
- Multi-task deployment: LoRA with adapter swapping
- Maximum quality: DoRA with higher rank
- Need to merge models: Train with TIES/DARE merging in mind
LoRA/QLoRA is the safe default. Branch out based on specific needs.