Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory

HBM is the contested tier in every AI factory. From GB300 to Rubin, capacity held at 288 GB while bandwidth rose to 19.2 TB/s — and SemiAnalysis AgentX shows Vera Rubin NVL72 delivering up to 30× throughput per megawatt versus GB300 at 160 tokens per second per user.

NVIDIA's technical post breaks the gain into five stack moves: NVFP4 shrinks weights and KV, wide expert parallelism spreads MoE experts across the NVLink domain, Dynamo treats KV as a fleet resource, Vera CPU parks idle context during tool calls, and CMX adds a pod-level G3.5 storage tier for shared cache.

This is the engineering companion to the marketing memory-codesign story — with enough specifics to audit.

We're treating this as the receipts post. Capacity alone cannot explain a 30× megawatt efficiency jump on the same 288 GB. The levers have to stack.

NVFP4's E2M1 elements with block and tensor scales cut weight footprint about 3.5× versus FP16 and 1.8× versus FP8, and NVFP4 KV cache halves footprint versus FP8 while holding over 99% of BF16 scores on cited DeepSeek-v4-Pro benches. Wide-EP in TensorRT-LLM puts DeepSeek-R1 style expert sets across eight or more GPUs so each device keeps more HBM for cache and raises arithmetic intensity; NVL72's fabric is what makes remote experts nearly local. Dynamo adds KV-aware routing, disaggregated prefill/decode, and a KV Cache Runner that tiers blocks across GPU, CPU, and storage via NIXL — with vendor demos from VAST and WEKA on transfer rates. Vera's 88 Olympus cores and up to 1.5 TB LPDDR5X over 1.8 TB/s NVLink-C2C shorten tool-call gaps and stage paused KV. CMX, built on BlueField-4 STX and Spectrum-X, is the shared flash tier for multi-turn, multi-node reuse.

I want operators to copy the measurement mindset even if they do not buy the whole rack. The downside if every efficiency claim stays inside NVIDIA's AgentX setup: buyers cannot tell co-design from benchmark selection. Publish the harness assumptions. Useful factories still win on tokens per megawatt under real agent gaps — not on GB of HBM in the SKU table.

Companion posts cover Rubin architecture, NVFP4 KV cache tuning, and Wide-EP on NVL72. The five-ways piece is aimed at practitioners implementing the stack rather than executives buying the metaphor.