Gemma
Why it matters
Deep Dive
Gemma arrived in 2024 as Google's entry into the open-weight model race that Meta's Llama had kicked off a year earlier. Built by Google DeepMind from the same research and technology used to create the Gemini models, it has moved through three generations: the original Gemma (2B and 7B parameters), Gemma 2 (9B and 27B), and Gemma 3 (1B, 4B, 12B, and 27B), which added image understanding, a 128k-token context window, and support for more than 140 languages. All Gemma models are decoder-only transformers trained on Google's TPUs, and all are released as downloadable weights under a custom license that permits commercial use. The pitch is simple: take the quality of a frontier lab's training pipeline and shrink it into something you can actually run yourself.
Picking a Size
The number in the name tells you the parameter count, and each tier targets a different machine. The 1B model runs comfortably on phones and even in browsers, the 4B is a good fit for laptops and small edge boxes, the 12B is a popular base for fine-tuning on a single consumer GPU, and the 27B flagship wants a workstation GPU or a small server but delivers quality that sits close to proprietary models many times its size. Because every size in a generation shares the same architecture, tokenizer, and training recipe, moving up or down the ladder later is a configuration change, not a rebuild. The real decision is quality per request against hardware budget — and, for on-device use, battery and memory.
Small Enough to Run Anywhere
Most Gemma usage happens outside data centers. Quantized to 4 bits, the 4B model shrinks to a few gigabytes, which means it runs on a modern laptop or a flagship phone at readable speeds, and community tooling does the heavy lifting: llama.cpp and Ollama ship ready-to-run GGUF builds for every Gemma size. This is the small language model thesis made concrete — a model you control, that works offline, keeps user data on the device, and costs nothing per request. For privacy-sensitive apps, offline tools, and embedded assistants, that combination often matters more than the last few points of benchmark quality. Google reinforces the story with its own mobile and browser runtime support, which is unusually polished compared with other open-weight families.
Fine-Tuning Is the Main Event
Because the weights are downloadable, Gemma has become one of the most fine-tuned model families in the ecosystem. Parameter-efficient methods like LoRA let a practitioner adapt a 4B or 12B Gemma to a niche job — a support-bot voice, a legal summarizer, a specialized classifier — on a single GPU in hours, and Hugging Face hosts thousands of community fine-tunes to start from. Google ships its own derivatives too: CodeGemma for code, PaliGemma for vision-language tasks, and ShieldGemma for safety classification. Part of the base models' strength also comes from distillation, where a larger teacher's outputs help train the smaller model — one reason Gemma consistently performs above what its parameter count would suggest.
Open Weights Isn't Open Source
A common misconception is that Gemma is open source. It is open weights, which is a different thing: Google releases the files under the Gemma Terms of Use, a custom license that allows commercial use, modification, and redistribution, but attaches a prohibited-use policy and reserves the right to restrict usage that violates it. In day-to-day practice this rarely blocks anyone — plenty of startups ship commercial products on Gemma without asking permission — but it is not the freedom of Apache 2.0, the license Qwen uses for most of its models, and the distinction matters if your legal team cares about license purity. The same nuance applies to Meta's Llama license: open weights describes what you can download, not the terms attached. When people say 'open model,' it is worth asking which of these two things they mean.
Where Gemma Fits
Gemma competes in a crowded bracket with the smaller Llama models, Qwen, and Microsoft's Phi, and the honest summary is that these families leapfrog each other with every release. Gemma's distinguishing strengths are its training pedigree — few small models come out of a frontier lab's full pipeline — and first-party support for on-device deployment. Its limits are the flip side of its size: nothing in the family challenges frontier models on the hardest reasoning problems, and like any small model it holds less world knowledge and hallucinates more readily than its bigger relatives. Teams typically reach for Gemma when they want a capable, cheap-to-run base model they can own end to end, and reach for a hosted API model when the task needs everything a frontier system has.