How NVIDIA Minerva Turns Telemetry Chaos Into Peak GPU Yield

NVIDIA has detailed Minerva, an AIOps platform that collects telemetry from GPUs, switches, network cards, storage, schedulers and workloads, maps it into a live knowledge graph, and uses machine learning to find faults that standard threshold alerts miss. It also includes an AI agent called Sherlock that operators can question in plain language, plus an MCP server so outside agent tools can query the same data.

This matters because in big synchronized training runs, a job moves only as fast as its slowest component. One GPU running hot or one network link losing bandwidth can drag down thousands of healthy chips. With compute this expensive, finding those faults in minutes instead of days is real money and real energy.

This is AI we're comfortable cheering. Gray failures, components that keep working but work badly, are exactly where fixed rules break down and pattern-matching against peers helps. NVIDIA's examples make the case concretely: one degrading NVLink connection found among roughly 400,000 interfaces, a rare PCIe fault on a SuperNIC caught from a single log line before customer jobs landed on the node, and a failing NVMe drive whose fix saved an estimated 5,530 hours of job runtime.

The alert-storm example is the one operators will recognize. A switch restart set off congestion across the fabric and produced 81 separate events. Minerva grouped them into one incident, and NVIDIA says that saved about 54 hours of cluster uptime. Anyone who has been paged 81 times for one problem knows why that matters.

Now the caveats. These are NVIDIA's own case studies and internal testing, described with phrases like significant reductions without hard numbers for most claims. We'd want independent operators reporting their results. And while NVIDIA says parts of Minerva, starting with the data collector, will become open source within DSX OS, the full picture is NVIDIA software watching NVIDIA hardware on NVIDIA networks. Better tools for running AI factories also make it harder to run them on anything else.

The Sherlock agent is useful but deserves care. NVIDIA frames it as giving less experienced operators access to specialist knowledge, and says recommendations trace back to supporting telemetry. That traceability is the part to insist on. An agent that confidently gives the wrong reason for a fault at this scale wastes as much as the fault itself. NVIDIA notes that mitigation can stay human-approved or be automated by local policy, and I'm watching how quickly operators hand over the keys.

Minerva is being aligned with NVIDIA's DSX OS, its software layer for operating AI factories, and reuses some DSX OS health and resilience components. It arrived the same day as NVIDIA's Mission Control 2.5 update, which extends automated job and hardware recovery to Kubernetes.