Zyphra released Zamba2-VL, an open vision-language model family in three sizes, 1.2B, 2.7B, and 7B parameters, under Apache 2.0 with weights, code, and a paper all public. On the perception side it is conventional: a pre-trained Qwen2.5-VL vision-transformer encoder turns image patches into features, a two-layer MLP adapter projects them into the language model's space, and the language model processes the interleaved vision and text tokens with single- and multi-image grounding. The unconventional part is the language backbone, and Zyphra's claim for it is the headline: this is the first open VLM family built on a fully open hybrid state-space-plus-transformer model.
The architecture splits the labor on purpose. The Zamba2 backbone alternates Mamba2 state-space layers, which carry the bulk of the computation cheaply and scale near-linearly with sequence length, with shared transformer attention blocks, each given its own LoRA adapter, that preserve the in-context retrieval pure-SSM models tend to give up. The reason this matters is latency on long inputs, and vision makes inputs long fast: a single high-resolution image can add on the order of 3,400 vision tokens, so a few images or a multi-page document blow past the point where attention's quadratic cost and its growing KV cache dominate the prefill. Zamba2 keeps a fixed-size recurrent state instead, and on a 32k-token prefill Zyphra reports roughly an order-of-magnitude lower time-to-first-token than the closest Transformer baseline. The models were trained on 100 billion tokens of vision-text and text data with the Mistral v0.1 tokenizer.
The benchmarks tell a two-sided story, and the honest read is in the second side. At 2.7B, Zamba2-VL is strong where the job is perception: it posts 90.9 on DocVQA for document understanding and 82.5 on PixMoCount for visual counting, the latter far above same-size Transformer baselines, with competitive general perception. It is behind where the job is knowledge and reasoning: 37.7 on MMMU and 51.0 on MathVista trail same-size Transformer VLMs, and OCR lags too. So the order-of-magnitude latency win is not free. You trade some reasoning and OCR quality for much faster first-token on long visual context, and the use cases Zyphra names line up exactly with that shape: document and form extraction, retail counting and inventory, multi-page PDFs, and on-device assistants, where the edge-tuned 1.2B model and low time-to-first-token matter more than maximum reasoning.
For the architecture-diversity thread we keep tracking, the slow accumulation of evidence that attention is not the only way to build a capable model, this is a clean and checkable data point: an open hybrid SSM-transformer VLM with a real, measured latency advantage and an honestly-bounded quality cost, code and weights in the open. The caveats are stated plainly: it needs a CUDA GPU for the optimized kernels, with CPU paths slow; there is no managed API, so you self-host; and it ships as a research artifact, not a production service. But the shape is the point. As multimodal inputs keep getting longer, more pages, higher-resolution images, more video frames, the quadratic cost of attention on the prefill is exactly where a near-linear state-space backbone earns its keep, and Zamba2-VL is the first open family that lets builders measure that tradeoff on their own long-context visual workloads instead of taking it on faith.