NVIDIA Nsight Python Tutorial: How to Profile GPU Kernels

A detailed tutorial walks through Nsight Python, NVIDIA's open-source Python interface to its kernel profilers. Instead of running a script under a separate command-line tool and digging through a report, developers annotate the code they care about, decorate a benchmark function, and get a pandas table of hardware metrics back. Since version 1.0 it attaches to the running Python process, so it works in Jupyter and Colab. The upcoming 1.1 release adds an experimental lighter-weight CUPTI backend and custom data collectors.

GPU time is the most expensive resource in AI right now. Better, easier measurement means less waste.

We're fans of tooling that makes good engineering habits the easy path, and this is that. Profiling has long been a separate ritual that many machine learning engineers skip. Putting it a decorator away, inside the notebook where the work already happens, means more people will actually measure before they optimize. The tutorial's demonstration of measuring kernel fusion in bytes moved, not just time, is a nice example of the deeper questions this unlocks.

The more useful lesson, though, is about defaults. The tutorial shows that Nsight Python flushes caches before each replay and leaves GPU clocks unlocked by default, which is the opposite of how the plain command-line profiler behaves. It also notes that the clock column reports the card's rating rather than the clock the kernel actually ran at, and that normalization only applies to the average value. None of these are bugs. All of them can quietly change what a number means.

That's where I'd put the caution flag. When measurement gets this convenient, numbers get copied into slides and papers without anyone remembering which settings produced them. Two teams using the same tool with different defaults can reach opposite conclusions about the same kernel. Easy tools raise the bar for documenting methods, not lower it.

The CUPTI backend is a sensible trade: lower overhead, but only duration measurements, no replay, cache flushing or clock control. Good for quick checks, not for deep diagnosis. Our read: worth adopting, with settings written down every time.

Nsight Compute is NVIDIA's established kernel profiler. Nsight Python requires its command-line tool at version 2026.2.1 or later, newer than what Colab ships, and must be imported before CUDA initializes. The tutorial ran on a free Colab T4 GPU.