IA3 (Infused Adapter by Inhibiting and Amplifying Inner Activations) multiplies activations by learned vectors.
Instead of adding new computations, IA3 rescales existing ones. It's highly parameter efficient, training only vectors per layer.
IA3 works well with few-shot learning and in-context learning enhancement. It's less powerful than LoRA but uses far fewer parameters.