With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents
Aug 24, 2026, 8:00 AM · NVIDIA Blog
NVIDIA is splitting long-context inference between Rubin GPUs for context work and Groq LPX for decode, treating token latency as a rack-level problem.
Why it matters
NVIDIA says Groq 3 LPX is in full production and reports 3,400 output tokens per second in a Gemma 4 31B, 100,000-token benchmark, four times the nearest alternative. Rubin handles context while LPX targets latency-sensitive generation.
The announcement also describes Spectrum-X Multiplane and BlueField-4, and names Nebius, CoreWeave and SpaceXAI as adopters or planned users.
From the desk
This is NVIDIA’s own announcement, so benchmark and partner-adoption claims are vendor-reported. It is best read as a bundle of platform announcements, not one isolated launch.
Signal Desk's read: specializing decode fits agent loops, where small delays compound across tool calls. But a fast decoder cannot rescue poor prefill scheduling, cache locality or idle accelerators.
The strategic move is making the rack the unit of inference competition. That may improve responsiveness while making cloud comparisons harder.
Context
NVIDIA describes 256 LP30 accelerators in a rack and says Multiplane can scale to 512,000 GPUs. An eight-plane failure is claimed to retain about 90% bandwidth.
Who feels it
- Inference providers
- Decode specialization may improve responsiveness.
- Cloud customers
- Benefits depend on pricing and workload mix.
- Schedulers
- Prefill/decode separation adds placement decisions.
What to watch
- Independent long-context testing
- Nebius exposure of LPX capacity
- Balance between Rubin and LPX under changing traffic
Companies: NVIDIA