Most PEFT research focuses on decoder-only transformers. Encoder-decoder models (T5) and encoder-only models (BERT) also support PEFT.
The concepts transfer:
- Apply LoRA to attention layers
- Use same hyperparameters as starting points
- May need architecture-specific tuning
Check your framework's documentation for architecture-specific guidance.