State space models borrow their mathematical framework from control theory, where SSMs have been used for decades to model dynamical systems. The core idea is a linear recurrence: the model maintains a hidden state that gets updated at each timestep by a learned linear transformation, then mixed with the current input. In continuous time, this is a differential equation (dx/dt = Ax + Bu, y = Cx + Du). Discretizing it gives you a recurrence that can process sequences token by token, updating a fixed-size state at each step. The elegant part is that during training, this recurrence can be unrolled into a convolution, making it parallelizable on GPUs just like attention. During inference, you switch back to the recurrence form and process tokens one at a time with constant memory — no growing KV cache.
Mamba (Albert Gu and Tri Dao, 2023) was the breakthrough that made SSMs competitive with Transformers on language. Earlier SSMs like S4 and H3 used fixed state transition matrices, which limited their ability to perform content-based reasoning — the model could not change how it processed a token based on what that token was. Mamba introduced selective state spaces, where the A, B, and C matrices are functions of the input. This lets the model decide, at each token, how much to remember and how much to forget. Think of it like a learned, differentiable gating mechanism, but operating through the lens of linear recurrence rather than attention. Mamba-2 later reformulated this as structured state space duality (SSD), revealing that selective SSMs and linear attention are mathematically related, and enabling even faster GPU implementations via matrix-multiply-based algorithms.
The practical advantages are real and measurable. During inference, a Transformer must store key-value pairs for every token in the context — that KV cache grows linearly with sequence length and is the primary bottleneck for long-context serving. An SSM maintains a fixed-size state regardless of how many tokens it has seen. For a model with a 128K context window, this difference is enormous: the SSM uses the same memory generating token 128,001 as it did generating token 1. Training throughput also benefits at long sequences because the parallel scan or convolution mode scales linearly with sequence length, versus the quadratic scaling of full attention. These efficiency gains are why SSMs are particularly attractive for applications that need long-range context: document analysis, code generation across large repositories, and real-time streaming where tokens arrive continuously.
That said, SSMs have real limitations that the hype tends to gloss over. Pure SSMs can struggle with tasks that require precise retrieval from earlier in the context — the "needle in a haystack" problem. A Transformer can, in principle, attend directly to any past token through its attention weights. An SSM must have compressed the relevant information into its fixed-size state, and if it did not prioritize the right things when it first processed that token, the information is gone. This is why hybrid architectures — interleaving SSM layers with a few attention layers — are gaining traction. Jamba (from AI21) and various research hybrids have shown that you can get most of the efficiency of SSMs with the retrieval precision of attention by using attention sparingly at strategic points in the network.
Mamba-3, the latest generation, pushes the architecture further with a multi-input multi-output (MIMO) formulation and complex-valued states via rotary position encodings. The recurrence uses a trapezoidal integration rule for better numerical stability, and the architecture drops the causal convolution layer that earlier versions used as a short-range mixing mechanism. These are not incremental tweaks — they change the computational profile enough that custom Triton kernels are needed to get full performance, and the standard mamba-ssm PyPI package does not include them yet. If you are building on SSMs today, expect to be working closer to the metal than you would with a mature Transformer stack. The tooling is catching up, but it is still early days for production SSM deployment.