Why it matters
JAX is the main framework at Google DeepMind and across much of Google's research. Models like Gemini and AlphaFold were built with it. It's also used outside Google: xAI has said it uses JAX in its training stack, and it's popular in scientific computing, physics simulation and probabilistic programming, where its composable transformations shine.
PyTorch remains far more common in the wider community. But if you read research code from Google, or you need things like per-example gradients or highly optimized code for TPUs, you will meet JAX.
Deep Dive
JAX is an open-source library from Google, first released in 2018. On the surface it looks like NumPy: you write jax.numpy code with arrays and math functions. Underneath, it adds a set of function transformations that turn plain Python functions into fast, differentiable, parallel programs that run on GPUs and TPUs.
It isn't a deep learning framework by itself. It has no built-in layers or optimizers. Those come from libraries built on top, such as Flax and Equinox for neural networks and Optax for optimizers. JAX is the engine beneath them.
The Four Transformations
gradtakes a function and returns a new function that computes its gradient. Apply it twice for second derivatives.jitcompiles a function with the XLA compiler into optimized code for CPU, GPU or TPU.vmapautomatically vectorizes a function written for one example so that it runs on a whole batch.pmapand newer sharding tools spread computation across multiple devices.
Their power is that they compose:
import jax
import jax.numpy as jnp
def loss(w, x, y):
return jnp.mean((x @ w - y) ** 2)
# a compiled gradient function
grad_fn = jax.jit(jax.grad(loss))
# one gradient per example instead of one for the batch
per_example = jax.vmap(jax.grad(loss), in_axes=(None, 0, 0))
Functional by Design
JAX requires pure functions: the output depends only on the inputs, and nothing is modified in place. This has consequences that surprise newcomers.
- Arrays are immutable. Instead of
x[0] = 1, you writex = x.at[0].set(1). - Model parameters live outside the model. They're passed into functions as nested dictionaries or other "pytrees" and returned updated, instead of being hidden inside objects.
- Random numbers are explicit. There is no global seed. You create a key and split it each time you need randomness, which keeps results reproducible even in parallel code.
- Python control flow is limited under
jit. A compiled function is traced once, so anifthat depends on an array's value must usejax.lax.condor a similar construct.
These rules are what let the compiler optimize aggressively, but they make JAX harder to learn and to debug than PyTorch.
The Ecosystem
- Flax and Equinox: neural network libraries.
- Optax: optimizers and learning rate schedules.
- Keras 3: runs on JAX, PyTorch or TensorFlow as interchangeable backends.
- Reference code: Google often publishes model code in JAX, as it did for AlphaFold and Gemma.
JAX's ideas have spread. PyTorch added torch.func, with JAX-style vmap and grad, and Apple's MLX borrows its composable transformations.
JAX or PyTorch?
Choose PyTorch by default: more tutorials, more pretrained models, more libraries, easier debugging and the largest community. Choose JAX when you want maximum performance from compiled code, when you run on TPUs, when your work is as much scientific computing as deep learning, or when you need transformations such as per-example gradients, Jacobians or Hessians to be easy.