How to Read Data into GPU Memory in Python with NVIDIA GPUDirect Storage

Nvidia published a hands-on guide to reading files straight into GPU memory from Python using KvikIO, a library that sits on top of GPUDirect Storage and its cuFile C API. GPUDirect Storage lets supported storage move data directly into GPU memory instead of staging it through the CPU, which cuts CPU load and helps keep expensive accelerators fed.

The practical value is verification. The post shows that a successful read doesn't prove the accelerated path was used, and it walks through how to check. As datasets and compute keep scaling, starved GPUs are wasted money, and this is the kind of detail that decides whether a cluster earns its keep.

We don't usually spend air on tutorials, but this one carries a lesson worth amplifying. There are two separate decision points. KvikIO decides whether to call cuFile at all, and cuFile then decides whether to use the accelerated nvidia-fs kernel path or a CPU-staged compatibility path. So a read that returns the right number of bytes might have quietly gone the slow way. That's the kind of silent fallback that burns budgets without anyone noticing.

The fix Nvidia lays out is simple and, frankly, should be standard practice. Turn on per-process cuFile statistics, inspect them with gds_stats, and look at the counters: operations under nvfs went the accelerated route, operations under posix did not. In Nvidia's test on an H100 with a local NVMe drive, a 1 GiB read split into 256 four-megabyte operations, all 256 on the fast path. Then the guide deliberately forces each fallback so you can see the difference. That's good engineering pedagogy.

The deeper section on how KvikIO and cuFile each split a big request, task sizes, thread pools, minimum I/O thresholds and bounce buffers, is a reminder that performance tuning lives in layers that interact. Change one knob and the other layer may re-split the work in ways you didn't expect.

Our caveat: the tutorial setup is idealized. It ran with IOMMU and ACS disabled to preserve peer-to-peer transfers, on a tight GPU-to-NVMe topology. Disabling those protections is a real security and isolation trade-off in shared or multi-tenant environments, and teams shouldn't copy that configuration blindly. The ecosystem also tilts further toward one vendor's stack each time a guide like this becomes the default path.

Still, we're for it. Efficient data movement is unglamorous useful AI infrastructure: less wasted energy, fewer idle accelerators, faster pipelines in fields like digital pathology and scientific imaging where KvikIO is already in use.

KvikIO is part of Nvidia's CUDA-X data science libraries, formerly RAPIDS, and grew out of GPUDirect Storage work in the cuDF DataFrame library for faster Parquet and ORC ingestion. Nvidia says it has since been used in cuCIM digital pathology, the cuPhoton xDataReader for scientific images, and GPU-accelerated Presto analytics. The tutorial was tested with CUDA 13.3 and cuFile 1.18.