Dynamo Snapshot uses CRIU + cuda-checkpoint for sub-5s K8s inference start

NVIDIA shipped Dynamo Snapshot v1.1.1 today, a Kubernetes checkpoint/restore system for AI inference workloads, available at github.com/ai-dynamo/dynamo with the snapshot-agent container on NGC. The architecture pairs two tools: CRIU (Checkpoint/Restore in Userspace), the long-standing Linux process freeze/thaw mechanism, with cuda-checkpoint, a newer capability in the CUDA driver that dumps GPU device state (contexts, streams, device memory, virtual mappings) into CPU memory. The order matters: cuda-checkpoint moves GPU state to CPU memory first, then CRIU serializes CPU memory, threads, file descriptors, and namespaces to storage. Restore reverses the chain. Cold start latency for inference workloads on Kubernetes has historically been minutes, and Dynamo Snapshot reports gpt-oss-120b initialization dropping from multiple minutes to under five seconds (21x speedup), with Qwen3-8B restore going from 24 seconds (upstream CRIU) to 4.7 seconds (5.1x) after optimization.

The mechanism worth pausing on is the KV cache size optimization. Inference workloads carry large KV caches that would otherwise dominate the snapshot artifact. Dynamo Snapshot preserves the virtual addresses but releases the physical memory via cuMemUnmap and cuMemRelease before capture, dropping artifact size from 190 GiB to 6 GiB for Qwen3-0.6B on B200. On restore, the addresses are reconstituted and the cache is re-warmed. The K8s integration uses three custom resources: a privileged snapshot-agent DaemonSet, a DynamoCheckpoint CR that defines the model config to snapshot, and a DynamoGraphDeployment CR that references the checkpoint for restore. Quiesce and resume hooks fire between engine initialization and distributed runtime startup via signal files. Hardware requirements are x86_64 GPU nodes with NVIDIA driver 580.xx or newer (590.xx or newer for multi-GPU) and ReadWriteMany storage for cross-node restore. The limitations matter and NVIDIA discloses them: vLLM backend only in this limited preview, single-GPU checkpoints only with multi-GPU planned, GMS (GPU Memory Service for concurrent weight restoration) is PoC-only, the CRIU optimizations using AIO and parallel memfd are pending upstream merge rather than already shipped, restore is sensitive to live TCP socket state, and the privileged DaemonSet requirement is non-trivial for some cluster security postures.

Two ecosystem threads worth tracking. First, cold start has been the structural argument against autoscaling AI inference on Kubernetes for as long as people have been doing it. Spinning up a new pod takes minutes; demand spikes do not wait minutes; SLA violations follow. The standard mitigations have been over-provisioning (pay for GPU idle), warm pools (still pay for GPU idle), and model streaming (faster but still 30 to 90 seconds). Sub-5-second restore changes that math materially. If reproducible on your workload, this is the first credible solution to the elastic-AI-inference-on-Kubernetes problem rather than a workaround. Second, the limitations are the substantive read on adoption timing. vLLM-only and single-GPU-only mean Dynamo Snapshot is currently a fit for a specific deployment shape (single-GPU vLLM serving) rather than a general K8s capability. TensorRT-LLM integration, multimodal workers, embedding and diffusion workers, multi-node, multi-GPU are all on the roadmap but not shipped. The CRIU upstream merges (AIO and parallel memfd) are the path to widening adoption beyond NVIDIA's fork. For builders evaluating this for production, the question is whether your stack matches the current limitations or you wait for the next quarter.

Monday morning, if you are running vLLM on Kubernetes with single-GPU inference and have been paying for warm-pool GPU idle to absorb demand spikes: try Dynamo Snapshot on a non-production cluster against your actual workload. The 21x for gpt-oss-120b is the upper bound; your real-world number depends on storage IO (ReadWriteMany matters), model size, and how aggressively you exercise the KV cache unmap path. If you are on TensorRT-LLM or multi-GPU today, monitor the roadmap and the CRIU upstream merges rather than betting on Dynamo Snapshot as a current option. If you are building inference platforms more broadly, the two-tool architecture (cuda-checkpoint for GPU state, CRIU for CPU and process state, with KV cache virtual address preservation) is the engineering pattern worth studying regardless of whether you adopt Dynamo Snapshot itself. And if you operate clusters where privileged DaemonSets are a non-starter for security reasons, this is not the solution for you yet; the architecture currently requires the privileged container to drive cuda-checkpoint and CRIU at the host level.