NVIDIA’s November 10 report describes open-source Snapshot restoring initialized inference servers, including workloads using multiple GPUs on one machine.

The big change

  • Snapshot saves prepared CPU and GPU state. Restoring it avoids repeating initialization, rather than merely reloading a model file.
  • NVIDIA’s tests exclude some setup costs. Its repository also advises against production-critical use, so the results need careful interpretation.
  • A useful evaluation should measure when a service becomes usable, not just one fast step. Operators should compare that result with the work required to maintain the recovery system.

Why saving initialization matters

NVIDIA’s May account explains the earlier experimental design. Starting another inference server meant loading weights and warming up execution while GPUs waited to serve requests. Its checkpoint moved GPU state into CPU memory, then saved the host process to disk. Restoration reversed that sequence so execution could resume where it stopped.

The documented workflow waited for initialization, saved an artifact to shared storage and restored it into a placeholder Kubernetes pod—a unit that runs containers. This illustrates why a checkpoint differs from a weights file: it preserves prepared execution, not just the model’s parameters.

The earlier design also required cooperation from the workload. Open connections could not simply travel with the saved state; a restored pod could have a different network address. The workload therefore needed to pause safely and recreate external resources. May’s experiment supported single-GPU vLLM and SGLang servers, with broader support still on its roadmap.

What the newer tests establish

The November comparison reports 3.6–42.7-second restores on B200 GPUs with VAST NFS storage. Cold starts took minutes. Container-image pulls and runtime setup were excluded; cached attention state was cleared before capture. Checkpoint creation separately took 15 seconds–5.5 minutes. BIG CHANGE did not reproduce these measurements.

The current contract excludes multi-node workloads. Restores require matching GPU models and drivers. Workloads must pause safely before capture. The repository requires shared storage and a privileged node agent; APIs may change. A privileged agent has elevated system access.

Before adopting any recovery system, an operator should test representative failures, restrict access to saved artifacts and budget for maintaining them. A reproducible comparison needs the same starting point and readiness definition on both sides. Those checks turn a speed claim into a decision about a particular service; they are not evidence that a deployment has already passed.