Zubnet AILearnWiki › Model Merging
Training

Model Merging

TIES, DARE, SLERP, Frankenmerge
Combining the weights of multiple fine-tuned models into a single model without any additional training. If model A is great at coding and model B is great at creative writing, merging them can produce a model that's good at both. Popular merging methods include SLERP (spherical interpolation), TIES (resolving sign conflicts), and DARE (randomly dropping parameters before merging).

Why it matters

Model merging is the open-source community's secret weapon. It costs zero compute (just math on weight tensors) and can produce models that outperform their components. Many top models on the Open LLM Leaderboard are merges. It's also how practitioners combine multiple LoRA fine-tunes into a single versatile model. Understanding merging unlocks a powerful, free capability for anyone working with open models.

Deep Dive

The simplest merge: linear interpolation. New_weight = α · A_weight + (1−α) · B_weight, where α controls the balance. This works surprisingly well when the models share the same base model (e.g., two different Llama fine-tunes). The merged model interpolates between the behaviors of both sources. SLERP (Spherical Linear Interpolation) interpolates along the hypersphere surface rather than linearly, often producing slightly better results.

Task Arithmetic

A more principled approach: compute "task vectors" (the difference between a fine-tuned model and the base model), then add task vectors to the base model. This lets you compose capabilities: base + coding_vector + writing_vector = a model with both skills. TIES improves on this by resolving sign conflicts between task vectors (when two tasks want to move the same weight in opposite directions). DARE improves it by randomly dropping most of the task vector entries, reducing interference.

Why It Works (and When It Doesn't)

Merging works because fine-tuning typically modifies a small subset of the model's behavior while preserving most of its general capabilities. The modifications from different fine-tunes often occupy different "regions" of parameter space with minimal conflict. It fails when fine-tunes conflict directly (two models trained to behave oppositely), when the base models are too different (can't merge a Llama with a Mistral), or when one component's modifications are so large that they dominate the merge.

Related Concepts

← All Terms
← Model Collapse Model Serving →