The dominant approach (ROME/MEMIT): identify which feedforward network weights encode a specific fact by tracing the causal effect of neurons on the model's prediction, then modify those weights to change the stored association. For example, to update "The Eiffel Tower is in Paris" to "The Eiffel Tower is in London," you find the weights that map "Eiffel Tower" → "Paris" in the FFN layers and redirect them to "London."
Editing "The Eiffel Tower is in London" should also change answers to "What country is the Eiffel Tower in?" (UK, not France) and "What landmarks are in Paris?" (no longer the Eiffel Tower). Current editing methods often fail at this: they change the direct fact but leave related inferences inconsistent. This "ripple effect" problem suggests that knowledge in LLMs is more interconnected than the surgical editing metaphor implies.
A few edits work reasonably well. Hundreds of edits start to degrade model quality — the edited weights accumulate changes that interfere with each other and with unedited knowledge. This limits knowledge editing's practical use: it's fine for a few corrections but can't serve as a general model update mechanism. For staying current, RAG (providing updated information at inference time) remains more practical than editing the model's weights.