The root cause is weight sharing: the same parameters encode multiple capabilities, and updating them for a new task disrupts the existing encodings. In a large neural network, knowledge isn't stored in dedicated neurons — it's distributed across the weights in complex, overlapping patterns (superposition). Modifying those weights for new knowledge inevitably disturbs old knowledge.
Several techniques reduce forgetting. Low learning rates during fine-tuning minimize weight changes. LoRA adds new trainable parameters while keeping the original weights frozen. Elastic Weight Consolidation (EWC) identifies which weights are important for old tasks and penalizes changes to them. Replay methods mix old task data into new task training. None fully solve the problem — there's always a trade-off between plasticity (learning new things) and stability (retaining old things).
Continual learning (also called lifelong learning) is the research goal of building models that can keep learning from new data without forgetting old capabilities — the way humans do. Current LLMs sidestep this by training once on a massive dataset and then fine-tuning carefully. True continual learning remains an open problem and would be transformative: imagine a model that keeps learning from every conversation without degrading.