MLX
Why it matters
MLX became one of the fastest ways to run LLMs on a Mac, often generating text faster than other options on the same hardware. A large community converts new open models to MLX format within hours or days of release, and the mlx-community organization on Hugging Face hosts thousands of ready-to-use, pre-quantized models. LM Studio uses MLX as one of its engines on Macs, so many people use MLX without knowing it.
It also made local fine-tuning practical on a laptop. With MLX-LM you can run LoRA or QLoRA on a 7B to 8B model on a MacBook with enough memory, which used to require an NVIDIA GPU.
Deep Dive
MLX was released by Apple's machine learning research team in December 2023. It looks and feels a lot like NumPy and PyTorch, so it's familiar to most Python developers, but it's designed from the ground up for Apple's chips and their unified memory. Arrays live in memory shared by the CPU and GPU, so there's no copying data back and forth between them.
Alongside the core framework, Apple maintains MLX-LM, a package for running and fine-tuning language models with a single command, and there are companion projects for vision and audio models. MLX also has Swift, C and C++ APIs for building apps.
What Makes It Different
- Unified memory: an array is usable by CPU and GPU without transfers, which simplifies code and saves memory.
- Lazy evaluation: operations build up a computation and only run when a result is needed, letting MLX optimize and fuse them.
- Composable function transforms: like JAX, it offers
grad,vmapandcompileto transform functions, which makes training code concise. - Familiar APIs:
mlx.coremirrors NumPy, andmlx.nnandmlx.optimizersmirror PyTorch's style.
Using MLX-LM
Typical commands look like this:
pip install mlx-lm
mlx_lm.generate --model mlx-community/<model-name> --prompt "Hello"
mlx_lm.convert --hf-path <huggingface-model> -q # convert and quantize
mlx_lm.lora --model <model> --train --data ./data # fine-tune with LoRA
It also includes a local server with an OpenAI-compatible API, so you can point existing tools at your Mac.
MLX Versus GGUF and llama.cpp
Both run well on Macs. llama.cpp (and tools built on it, such as Ollama) uses the GGUF format, runs on almost every platform and has a vast ecosystem. MLX is Mac-only, uses its own safetensors-based format, and is often faster for generation and better for fine-tuning on Apple hardware. Many Mac users keep both: GGUF for portability, MLX for speed.
Limitations
MLX only runs on Apple Silicon (with a CUDA backend in development for some uses), so it's not a choice for cloud servers. Model support depends on the community porting each new architecture, and it's not aimed at large-scale distributed training. For local inference and experiments on a Mac, though, it's often the best tool available.