Faster LLM Serving Startup with fastsafetensors

As models grow, startup time has become a bottleneck for PyTorch inference systems like vLLM. A large model can take minutes to load, which slows down experimentation, deployment, autoscaling, and failure recovery.

This talk presents fastsafetensors, an open-source library that accelerates safetensors checkpoint loading. fastsafetensors treats loading as a data-movement problem: it removes per-tensor copies, coalesces fragmented I/O, and skips host staging. The result is 4.8x to 7.5x faster model loading and up to 28 GB/s of NVMe read throughput. It already plugs into existing LLM serving systems, for example:

vllm serve Qwen/Qwen3-0.6B --load-format fastsafetensors

We will highlight community contributions that have transformed fastsafetensors from its original NVIDIA GPU-specific loader into a portable framework: parallel loading, 3FS integration, ROCm support, a universal wheel with runtime CUDA/ROCm detection, Windows DirectStorage exploration, and unified-memory support.

Attendees will learn how fastsafetensors works and how to make model loading fast, portable, and practical.