Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

NVIDIA released Snapshot, an open-source checkpoint-and-restore system for GPU workloads on Kubernetes. It captures a fully initialized inference workload, CPU and GPU memory included, and restores it on a node with the same GPU model and driver. That skips weight loading, kernel compilation and CUDA graph capture.

In NVIDIA's tests on B200 GPUs, restores took 3.6 to 42.7 seconds, 8x to 36x faster than cold starts. The largest mixture-of-experts models, with checkpoints up to about 1 TB across eight GPUs, came back in under 43 seconds versus cold starts of 6 to 19 minutes.

We think this is one of the more practically useful releases of the day. When a replica fails or traffic spikes, idle GPUs aren't capacity until the model is loaded and warmed up. Minutes of dead time push operators to over-provision just in case. If restores take seconds, they can run leaner and scale from zero more aggressively, which lowers cost and wastes less power.

The numbers are vendor numbers, so read them that way. They exclude container image pulls and infrastructure provisioning, and they were run on fast VAST NFS storage. Real clusters with slower storage will see less. Still, the methodology is laid out by phase, the reproduction tools are public, and two design partners report similar gains: Crusoe cut GLM-5.3 readiness from roughly eight minutes to 31 seconds, and Saturn Cloud brought vLLM startup from two to eight and a half minutes down to six to 34 seconds.

One detail we liked: in every tested setup, checkpoint time plus a restore was shorter than one cold start. That means the first restore pays for itself.

The catch is portability. Restores require the same GPU model and driver, which ties snapshots to specific fleets and rewards uniform, single-vendor hardware. Checkpoint artifacts are also big, up to about a terabyte, and they hold live memory state. Who can read those artifacts, and how they're protected at rest, is a question operators should ask before treating them as routine files.

Snapshot started as part of NVIDIA Dynamo and is now part of the DSX AI Factory Platform. It works through Kubernetes custom resources and supports vLLM, SGLang and TensorRT LLM, including multi-GPU parallelism strategies for large MoE models. NVIDIA's breakdown shows moving GPU state dominates large restores, at 18 to 29 seconds for the eight-GPU MoE cases.