Llama
Why it matters
Deep Dive
Llama is not one model but a lineage of releases from Meta AI, and understanding the family means tracking two threads at once: the technical line that runs from Llama 1 through Llama 4, and the licensing line that turned a research artifact into the industry's shared substrate. Every release ships as downloadable weights plus a tokenizer and reference code, so anyone with enough GPU memory — or, for the smaller sizes, an ordinary laptop — can run the model locally, fine-tune it, quantize it, or embed it in a product. That combination of credible quality and total local control is what made Llama the reference point of the open-weights ecosystem: when a new model from Mistral, Qwen, or DeepSeek appears, the first question practitioners ask is how it stacks up against the equivalent Llama size.
From Research Leak to Product Family
The original LLaMA, released in February 2023, was a research-only affair: sizes from 7B to 65B parameters, gated behind an application form, licensed for non-commercial use. The full weights leaked within days, which accidentally demonstrated how much demand existed for a capable model people could run themselves. Meta leaned into that demand with Llama 2 in July 2023, shipping 7B, 13B, and 70B sizes under a license permitting most commercial use, plus chat variants tuned with RLHF. The Llama 3 generation in 2024 raised the bar again, trained on roughly 15 trillion tokens: 8B and 70B sizes first, then the 405B flagship in Llama 3.1, small 1B and 3B text models plus vision-capable 11B and 90B versions in 3.2, and an efficiency-focused 70B in 3.3. Llama 4 arrived in 2025 with a mixture-of-experts architecture: Scout and Maverick as the released models, with the much larger Behemoth described as a teacher model still in training.
The Architecture That Became the Default
Architecturally, Llama is a decoder-only transformer, and each generation is evolutionary rather than radical: pre-normalization with RMSNorm, rotary positional embeddings, SwiGLU feed-forward layers, and, from Llama 2's 70B onward, grouped-query attention to shrink the KV cache at inference time. None of these choices were invented at Meta, but Llama's popularity made them the de facto recipe, and people now casually describe other models as Llama-architecture when they follow the same template. That standardization has real practical value: inference stacks like vLLM, SGLang, and llama.cpp are optimized first and hardest for Llama-shaped models, and some labs deliberately keep their releases Llama-compatible so the tooling just works. Llama 3's 128k-token vocabulary and the 128k-token context window introduced with 3.1 similarly became reference points other open models are measured against.
Open Weights Is Not Open Source
A persistent misconception is that Llama is open source. It is not, at least by the Open Source Initiative's definition: Meta's Llama Community License grants broad rights to use, modify, and redistribute the models, but it carries field-of-use restrictions that OSI's criteria disallow. Companies with more than 700 million monthly active users need a separate license from Meta, the Acceptable Use Policy restricts certain applications, and derivative models must include Llama in the name and display Built with Llama attribution. Some releases add regional quirks — the multimodal Llama 3.2 models, for example, are not licensed to companies domiciled in the EU. For an individual developer or a small startup none of this usually matters in practice, but it is exactly the kind of fine print legal teams care about, and it is the sharpest illustration of the open-versus-closed spectrum: downloadable weights are not the same thing as an Apache 2.0 or MIT license.
The Ecosystem Around the Weights
The weights themselves are only half the story; the tooling and derivatives built around them are arguably Llama's biggest contribution. Within weeks of the Llama 1 leak, projects like Alpaca and Vicuna showed that a cheap instruction-tuning run on synthetic data could turn the base model into a usable chatbot, kicking off the modern fine-tuning wave. Georgi Gerganov's llama.cpp made it possible to run quantized Llama models on consumer CPUs and laptops, spawning the GGUF format and a whole local-inference stack that includes Ollama, while Hugging Face now hosts tens of thousands of Llama fine-tunes, merges, and quantizations. Thanks to distillation and model merging, many supposedly new open models are, under the hood, Llama derivatives. This network effect is self-reinforcing: because everyone builds for Llama first, choosing Llama means the best tooling support, which in turn keeps it the default even when a competitor posts slightly better benchmark numbers.