Apple Silicon
Why it matters
Unified memory made Macs one of the most popular ways to run large models locally. A MacBook Pro with 128 GB of memory can load a quantized 70B-parameter model that would need multiple high-end NVIDIA cards on a PC. High-memory Mac Studios became a common choice for running very large open models at home or in small offices, quietly and at low power.
Apple Silicon is also the hardware behind Apple Intelligence's on-device models, and a key target for the Local AI path: tools like Ollama, LM Studio, llama.cpp and MLX all have first-class Mac support.
Deep Dive
In 2020, Apple began replacing Intel processors in Macs with its own chips, starting with the M1. Each chip combines CPU cores, a GPU and a Neural Engine on one piece of silicon, except Ultra chips and the M5 Pro and Max, which are two dies joined together. The key feature for AI is unified memory: instead of the GPU having its own separate memory (VRAM), all parts of the chip share one large pool of fast memory.
On a typical PC, a model must fit in the graphics card's VRAM, which is 8 to 32 GB on consumer cards. On a Mac, the GPU can use most of the system's memory, which can be 64 GB, 128 GB or even up to 512 GB at the top.
Memory Size Versus Memory Speed
Two numbers matter for local LLMs. Memory capacity decides how big a model you can load. Memory bandwidth decides how fast it generates text, because producing each token means reading through the model's weights. Bandwidth varies a lot by chip tier: base chips offer roughly 100 to 150 GB/s, Pro chips about twice that, Max chips up to 614 GB/s on the M5 Max, and Ultra chips over 800 GB/s. A dedicated high-end NVIDIA GPU still has more, often 1,000 to 1,800 GB/s, but its memory is far smaller.
A rough rule: tokens per second ≈ bandwidth ÷ size of the model in memory. A 40 GB model on a chip with 400 GB/s tops out near 10 tokens per second.
Where Macs Lag
Macs are strong at generating tokens for their price and power, but slower at prompt processing (reading a long input before answering), which depends on raw compute. Long documents or large codebases can take a noticeable time to "read in". Training and fine-tuning large models is also far slower than on NVIDIA hardware. Apple's M5 generation (2025) added dedicated matrix units inside the GPU, which targets exactly this gap.
Software
- MLX: Apple's own machine learning framework, designed around unified memory.
- llama.cpp / Ollama / LM Studio: run GGUF models using Apple's Metal GPU framework. LM Studio can also run MLX models.
- PyTorch: supports Mac GPUs through its MPS backend, useful for experimentation.
- Core ML: Apple's framework for shipping models inside apps.
Practical Tips
By default, macOS limits how much memory the GPU may use (roughly two-thirds to three-quarters of the total), and this can be raised with a system setting for dedicated AI machines. Leave room for the KV cache, since long contexts need several extra gigabytes. When buying, memory capacity is the spec you can't upgrade later.