Every Mixture-of-Experts (MoE) model repeats one decision for every token: which experts should process it. That decision belongs to the router. MoE routing strategies shape quality, load balance and serving cost in every major open model. This guide explains the 7 routing strategies used in production, and the models that use each one.
TL;DR
- Size: Routers choose from 8 experts (Mixtral) to 384 (DeepSeek-V4-Pro), activating 1 to 8 per token.
- Runs on: Routing cost shows up in multi-GPU expert parallelism, where each choice becomes all-to-all network traffic.
- Performance: Sigmoid scoring with aux-loss-free bias balancing is the dominant 2026 recipe for frontier open MoE models.
- Best: Loss-free balancing cut MaxVio from 0.72 to 0.04 at 1B, with lower perplexity.
- Worst: Expert choice leaks future tokens in causal models; one test showed a roughly 10% abnormal loss drop.
- Bottom line: Best thing: bias-based balancing adds no interfering gradients. Worst thing: no router removes the need for serving-time load fixes.
What is MoE routing?
MoE routing is the mechanism that sends each token to a small subset of expert networks. A router scores every expert, keeps the top candidates and weights their outputs. Only the selected experts run. Mixtral 8x7B, for example, gives each token access to 47B parameters but uses 13B active parameters.
The 7 MoE routing strategies used in production
1. Softmax top-k token choice
Each token scores all experts with a linear layer and keeps the k highest. A softmax over those k scores sets the mixing weights. Mixtral picks 2 of 8 experts. gpt-oss-120b picks 4 of 128. Qwen3-235B-A22B picks 8 of 128. An auxiliary loss usually keeps expert load even.
Used by: Mixtral, gpt-oss, Qwen3.
2. Top-1 switch routing with a capacity factor
Switch Transformer showed that routing each token to 1 expert preserves quality and cuts router compute. Each expert gets a fixed buffer: tokens per batch divided by experts, times a capacity factor. Overflow tokens skip the layer through the residual connection. Llama 4 Maverick also sends each token to 1 routed expert, plus a shared expert.
Used by: Switch Transformer, Llama 4 Maverick (with a shared expert).
3. Shared plus routed experts
DeepSeekMoE splits each layer into always-on shared experts and sparsely routed experts. The shared expert captures common knowledge, so routed experts can specialize. DeepSeek-V3 uses 1 shared and 256 routed experts, activating 8. Kimi K2, GLM-4.5 and Qwen3.6-35B-A3B follow the same pattern. Qwen3 went the other way and removed shared experts.
Used by: DeepSeek-V3 and V4, Kimi K2, GLM-4.5, Qwen3.6, Llama 4.
4. Sigmoid gating with aux-loss-free bias
DeepSeek-V3 scores experts with a sigmoid instead of a softmax. It adds a per-expert bias only when choosing the top-k. After each step, the bias drops for overloaded experts and rises for idle ones. Mixing weights still come from the raw scores. GLM-4.5 uses loss-free balance routing with sigmoid gates. DeepSeek-V4 switches the score to sqrtsoftplus but keeps bias balancing.
Used by: DeepSeek-V3, DeepSeek-V4, GLM-4.5, Kimi K2.
5. Group-limited and node-limited routing
Sending one token to experts on many nodes is expensive. DeepSeek-V3 caps each token at 4 nodes, chosen by their highest affinity scores. Its config splits experts into 8 groups and keeps 4 before the final top-8 pick. DeepSeek says this cap lets training nearly fully overlap computation and communication. Kimi K2 drops grouping entirely, with n_group set to 1.
Used by: DeepSeek-V2 and V3.
6. Expert choice routing
Google’s expert choice routing flips the direction. Each expert picks its top-k tokens, so every expert gets a fixed bucket size. Tokens can receive a variable number of experts. The paper reports more than 2x faster training convergence. The catch is causality: a token’s expert can depend on later tokens. DeepSeek researchers measured an abnormal loss drop of about 10% from this leakage.
Used by: Research and non-causal settings. None of the 10 models in our companion table uses it.
7. Hash routing and dynamic top-k
Hash routing assigns experts from the token ID, with no learned router. DeepSeek-V4 uses hash routing in its first 3 MoE layers. Dynamic top-k lets the number of active experts vary per token. SMIDT built it into SGLang and improved P99 time-to-first-token by 1.21 to 1.26x.
Used by: DeepSeek-V4 (hash routing), research serving systems (dynamic top-k).
MoE routing strategies compared
| Strategy | Who chooses | How load is balanced | Main risk | Used by |
|---|---|---|---|---|
| Softmax top-k | Token picks k experts | Auxiliary loss | Loss trades off quality | Mixtral, gpt-oss, Qwen3 |
| Top-1 switch | Token picks 1 expert | Aux loss + capacity factor | Dropped overflow tokens | Switch, Llama 4 Maverick |
| Shared + routed | Token picks routed; shared always runs | Inherits routed method | Extra always-on compute | DeepSeek, Kimi K2, GLM-4.5, Qwen3.6 |
| Sigmoid + bias | Token picks via biased score | Per-expert bias, no loss gradient | Imbalance inside one sequence | DeepSeek-V3/V4, GLM-4.5, Kimi K2 |
| Node-limited | Token picks within capped nodes | Inherits routed method | Restricts expert choice | DeepSeek-V2/V3 |
| Expert choice | Expert picks k tokens | Fixed by construction | Future-token leakage | Research models |
| Hash / dynamic | Token ID or variable k | Fixed mapping or adaptive k | Less learned specialization | DeepSeek-V4, SMIDT |
‘;var ex=nd.querySelector(‘.ex’);for(var j=0;j<4;j++){var e=n*4+j;var c=document.createElement(‘div’);c.className=”e”;c.id=’r1e’+e;c.innerHTML=’E’+(e+1)+’0‘;ex.appendChild(c)}nw.appendChild(nd)}
function post(){try{parent.postMessage({mtpEmbed:’r1′,h:root.offsetHeight+40},’*’)}catch(x){}}
function reset(){timers.forEach(clearTimeout);timers=[];busy=false;load=[];for(var e=0;e
$(‘r1shared’).className=”shared”+(s.sh?’ on’:”);$(‘r1shared’).style.display=s.sh?’flex’:’none’;
$(‘r1s2l’).textContent=s.node?’Max nodes / token’:(s.ec?’Tokens with 0 experts’:’Experts used’);
$(‘r1s3l’).textContent=s.cap?’Dropped tokens’:(s.bias?’Bias on E1 to E4′:’Expert slots’);reset();post()}
function topk(arr,k,idx){idx=idx||arr.map(function(_,i){return i});return idx.slice().sort(function(a,b){return arr[b]-arr[a]}).slice(0,k)}
function plan(){var s=S[cur],P=[],drop=[],nodesMax=0;
if(s.ec){var bucket=Math.round(T*2/E);var got=[];for(var t=0;t
return{P:P,drop:drop}}
function go(){if(busy)return;reset();busy=true;var s=S[cur],pl=plan(),P=pl.P,step=0;
P.forEach(function(ex,t){timers.push(setTimeout(function(){
for(var q=0;q
if(s.sh){var sh=$(‘r1shared’);sh.classList.add(‘pulse’);setTimeout(function(){sh.classList.remove(‘pulse’)},220)}
var nodes={};ex.forEach(function(e){load[e]++;nodes[Math.floor(e/4)]=1;var c=$(‘r1e’+e);c.classList.add(‘flash’);setTimeout(function(){c.classList.remove(‘flash’)},260)});
for(var n=0;n<4;n++){$(‘r1n’+n).className=”node”+(s.node?(nodes[n]?’ lit’:’ off’):”)}
render(s,pl);if(t===T-1){timers.push(setTimeout(function(){for(var q=0;q
for(var e=0;e
$(‘r1s1′).textContent=sum?(mx+’ / ‘+avg.toFixed(1)):’0’;
if(s.node)$(‘r1s2’).textContent=”2″;else if(s.ec){var z=0;pl.P.forEach(function(p){if(!p.length)z++});$(‘r1s2’).textContent=z}else $(‘r1s2′).textContent=used+’ of 16′;
$(‘r1s3′).textContent=s.cap?pl.drop.length:(s.bias?’lowered’:sum)}
$(‘r1go’).onclick=go;$(‘r1reset’).onclick=reset;sel(0);
window.addEventListener(‘resize’,post);setTimeout(post,200);
})();
