When you need better results than simple averaging, use TIES or DARE.
TIES (Trim, Elect Sign, Merge):
Trim low-magnitude updates
Resolve sign conflicts between adapters
Average remaining updates
DARE (Drop And REscale) randomly zeros adapter weights before merging, then rescales to compensate. This reduces interference between adapters. Combined, TIES-DARE gives you better merged models from diverse adapters than naive approaches.