Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes

At GTC Berlin, NVIDIA announced official support for eight-node DGX Spark clusters, scaling unified memory to 1 TB. That's enough to run the full GLM 5.3 model from Z.ai, at 753 billion parameters, locally. Support for eight-node clusters in NVIDIA's Sync setup tool is due by the end of the year.

For teams that can't or won't send sensitive data to a cloud, this widens what is possible on hardware they control, from serving big open models to training small ones from scratch.

We think local AI is worth rooting for. Running a frontier-class open model on your own hardware means your data stays put, your costs are predictable, and you aren't one pricing change away from rewriting your stack. NVIDIA's numbers make the case credible. Decode throughput on a dense Qwen3.8-27B model rose from about 12 tokens per second on one node to roughly 52 on eight, using a switch and a new Spark-optimized communication backend called RoCEnante. Pre-training a small nanochat model hit a 7.79x speedup on eight nodes at 97.4 percent scaling efficiency.

The honest picture is more mixed than the headline. The full GLM 5.3 model ran at about 31 tokens per second for a single user, dropping to 18 per user at a concurrency of eight. That's usable for development, not a production service for many people. Prefill throughput barely scaled with node count in NVIDIA's dense-model test, reaching under 2x on eight nodes. And RoCEnante only helps on switch-connected clusters at low concurrency; NVIDIA itself says its standard NCCL library is better as clusters and traffic grow.

The most interesting idea here may be the least flashy. NVIDIA's Switchyard prototype routes easy requests to a small model and hard ones to a larger one. On a 92-case tool-use benchmark, the routed setup matched the large-model-only score while raising cluster throughput up to 3x when most traffic went to the small model. That's the right direction for the whole industry: stop paying frontier prices for simple tasks.

The tradeoff to name is cost and concentration. Eight Sparks plus networking is a serious purchase, and this is NVIDIA's ecosystem top to bottom. Local doesn't mean independent if every layer comes from one vendor. I'm watching pricing and whether open tooling keeps pace.

A single DGX Spark has 128 GB of unified memory; four nodes reach 512 GB. NVIDIA says a switchless ring is the cheapest and simplest option for four-node inference, while a switch is needed to scale to eight nodes, split the cluster into groups, or run all-to-all workloads. The RoCEnante backend is merged into FlashInfer and falls back to NCCL if its startup test fails.