Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared

Open Mixture-of-Experts models share one idea and disagree on almost every routing detail. Expert counts now run from 8 to 384. Top-k spans 1 to 8. Some keep an always-on shared expert; others drop it. A comparison of ten models, checked against each Hugging Face config.json, model card, or technical report on November 7, 2026, runs from Qwen3.6-35B-A3B — 35 billion total parameters, 3 billion active — to DeepSeek-V4-Pro at 1.6 trillion total and 49 billion active.

That spread is not academic. gpt-oss-120b fits on a single 80 GB GPU with MXFP4 weights. DeepSeek-V4-Pro and Kimi K2 need multi-GPU expert parallelism. Routed sparsity stretches from 4× in Mixtral, which uses 8 experts and top-2, to 128× in Llama 4 Maverick, which uses 128 experts and top-1. Configs can make routing fully auditable. Marketing names like “17B active” hide very different routers.

We're treating this as a serving story, not a beauty contest. Every model in the set uses token-choice routing: each token picks its own experts. The five knobs that actually matter are expert count, experts per token, shared experts, the scoring function, and the load-balancing method. Those fields decide memory, all-to-all traffic, and whether a shop can run the model at all. There is no published quality winner here, and inventing one from the configs would be dishonest.

The DeepSeek family is the house style everyone else is circling. DeepSeek-V3 scores 256 routed experts with a sigmoid, then normalizes the selected scores. A per-expert bias steers top-8 selection without touching the mixing weights. Experts sit in eight groups, and the router keeps four groups first, so each token reaches at most four nodes, according to the technical report. One shared expert runs for every token. DeepSeek-V4 raises Pro to 384 routed experts, drops top-k from 8 to 6, switches the score function to sqrtsoftplus, and puts hash routing — a fixed token-to-expert map — in the first three MoE layers. Bias-based noaux_tc balancing stays. Kimi K2 ships with the DeepseekV3ForCausalLM architecture class and the same noaux_tc router, raising routed experts to 384 at the same top-8. Its report found sparsity 48 needed 1.69× fewer FLOPs than sparsity 8 at equal validation loss, and that higher sparsity lowered loss at fixed active compute. GLM-4.5 went the other way: 160 experts and more layers, favoring depth over width.

The non-DeepSeek designs are not minor variants. Both original Qwen3 MoE models use 128 experts and route each token to 8. Qwen3 dropped the shared experts of Qwen2.5-MoE and adopted a global-batch balancing loss. Qwen3.6-35B-A3B changed course again: 256 experts, eight routed plus one shared, per its model card. Llama 4 Maverick alternates dense and MoE layers. In each MoE layer a token goes to one shared expert and one of 128 routed experts — top-1, echoing the Switch Transformer — and still claims 17 billion active parameters against 400 billion total. gpt-oss-120b uses a linear router over 128 experts and keeps the top 4. Its config lists no shared expert, and MXFP4 MoE weights let the 117 billion-parameter model fit on one 80 GB GPU.

I'm glad someone sat down and read the configs. Useful serving work starts with knowing the router, not the brochure. The downside is already visible. Llama 4 Maverick and gpt-oss do not publish their gating or balancing method in config.json. Configs and cards for Qwen3.6 and Llama 4 omit gating; gpt-oss and Llama 4 omit the balancing method. Fat expert counts raise resident memory and network cost. A lower top-k, as in DeepSeek-V4, trims per-token compute and communication. Shared experts add a dense path every token pays for. Seven of the ten models keep one shared expert. Mixtral, gpt-oss, and Qwen3 do not. Expert counts have grown from Mixtral's 8 to 384 in Kimi K2 and DeepSeek-V4-Pro, and top-k now spans Llama 4's 1 to the top-8 used by DeepSeek-V3, Qwen3, Kimi K2, and GLM-4.5. If this scales as a marketing habit, teams will buy “17B active” and discover they sized GPUs for the wrong machine. The useful habit is to read the five knobs and distrust the name.

Sigmoid or sqrtsoftplus scoring with bias balancing is the DeepSeek-family standard. That pattern is spreading through architecture class names, not through a shared open specification, which is why unpublished gating and balancing details still matter.