Evidence that LLMs may build world models: they can play chess (requiring spatial reasoning), solve novel physics problems, generate working code for described algorithms (requiring causal reasoning about program execution), and navigate text-based worlds consistently. Research by Li et al. (2023) showed that a model trained only on Othello game transcripts developed an internal representation of the board state — a literal world model emerging from sequence prediction.
LLMs make errors that suggest pattern matching rather than understanding: they struggle with spatial reasoning ("I walk north, then east, then south — where am I relative to the start?"), fail at novel physical reasoning (situations not in training data), and can be tripped up by simple modifications to familiar problems (changing numbers in a math problem they solved correctly in standard form). These failures suggest the model learned surface patterns, not underlying mechanisms.
The emerging view: LLMs build partial, approximate world models that work well for common situations but break down at the edges. They learn useful representations of how the world works — good enough for most text generation tasks — but these representations are incomplete, inconsistent, and not grounded in actual physical experience. Whether this constitutes "understanding" depends on your definition. What's practical: LLM world models are useful but shouldn't be trusted for safety-critical physical reasoning without verification.