NVIDIA Releases Nemotron 3 Diarization: A 100M-Parameter Open-Weight Model That Tracks 8 Speakers in Real Time

ASR gives you the words. It does not tell you who said them. Without speaker attribution, meeting tools, call analytics, podcasts, and voice-agent memory cannot assign commitments or objections to a person.

Nemotron 3 Diarization doubles NVIDIA’s prior Streaming Sortformer limit from four speakers to eight, and ships one checkpoint that covers both offline recordings and real-time streams. That matters anywhere multi-party audio is messy and overlapping.

We’re treating this as useful infrastructure for spoken AI, not a vanity parameter count.

The model is 100M parameters, open-weight on Hugging Face under OpenMDW License 1.1, which MarkTechPost reports permits commercial use. It runs through NVIDIA NeMo on Ampere, Ada Lovelace, Hopper, or Blackwell GPUs. Architecture-wise it’s a Sortformer-style stack: Mel features at 10 ms, stacked to 80 ms encoder frames, a 31-layer Transformer with RoPE, then upsampling back to 10 ms with a [T, 8] activity tensor — overlap handled by activating multiple channels in the same frame.

Latency knobs are concrete. Offline-style buffering at 30.4 seconds posts 12.73% DER on DIHARD III with 15,113× RTFx at batch-32 compiled. Ultra-low 0.32-second buffering lands at 13.55% DER and 292× RTFx. Voice Arena’s initial Diarization-Bench put it first among 12 systems at 14.72% DER versus 19.3% for the next system — NVIDIA notes those results may move when Version 1 finishes.

Against the four-speaker baseline at 1.04-second latency, relative DER reductions averaged 41.0% across eight conditions, with one small regression on two-speaker CALLHOME at the long buffer. Training mixed about 10,000 hours of real conversation with 82,611 hours of simulated multi-talker audio spanning 21 languages in licensed source material.

Useful AI loves open, commercial-ready speech plumbing. Who-spoke-when is table stakes for trustworthy meeting summaries and contact-center agents. The downside is the hard eight-speaker ceiling — more voices mean missed or misassigned speech — plus noise, reverb, and far-field capture still raise error rates. Labels are anonymous; identity mapping is downstream. And the ranking is an initial Voice Arena cut, not a settled crown.

I’m watching production pairing with Parakeet TDT and whether on-device Argmax Pro SDK 3 paths close the gap for edge deployments that cannot ship every call to a GPU box.

MarkTechPost coverage by Asif Razzaq, Sep 23, 2026, of NVIDIA’s Nemotron 3 Diarization release on Hugging Face, including latency table, Voice Arena ranking, and NeMo getting-started path.