The open-weight 'flash tier' now contains three frontier mixture-of-experts models released within a single month: Z.ai's GLM-5.3-Flash, DeepSeek's V4 Flash, and Alibaba's Qwen3.8-Flash-Next. All three attack the same problem — the cost of serving very long contexts with a large-parameter model — but they do so through materially different attention mechanisms, capacity strategies, and hardware footprints. Developers choosing among them must weigh not only benchmark scores but self-hosting floors that range from a two-GPU pod to a full eight-GPU node. This article contrasts the three architectures side by side, using vendor documentation, serving recipes, and independently measured performance data. Version sensitivity is flagged throughout, since the tier is moving quickly and several numbers depend on exact checkpoint revisions.
| Model | Total parameters | Active per token | Layers (total) | Sparsity* | Hybrid layer ratio | Routing |
|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | 284 B (reported) † | 13 B | 43 | ≈ 4.6 % | CSA : HCA ≈ 4 : 1 | bias‑only balance, no aux loss |
| GLM‑5.3‑Flash | 320 B | 18 B | 45 | ≈ 5.6 % | 34 KDA linear : 11 NoPE MLA ≈ 3 : 1 | top‑K softmax, 8 of 288 experts + 1 shared |
| Qwen3.8‑Flash‑Next | 125 B MoE + 51 B n‑gram table ≈ 177 B total | ≈ 6 B | 48 (36 GDN + 12 QSA) | ≈ 4.8 % | Gated DeltaNet : QSA = 3 : 1 | top‑10 of 512 experts, gated residual |
*Sparsity = (active / total) × 100 %.
Hybrid attention – DeepSeek interleaves Compressed Sparse Attention (CSA) with Hybrid Compressed Attention (HCA) in a 4 : 1 pattern 【1†url】. GLM mixes Kernel‑Dense Attention (KDA) linear layers with No‑Positional‑Encoding (NoPE) Mixture‑of‑Linear‑Attention (MLA) layers at roughly 3 : 1 【2†url】. Qwen‑Next pairs Gated DeltaNet (GDN) with Qwen Sparse Attention (QSA) exactly 3 : 1 【3†url】.
Routing & expert activation – DeepSeek uses a bias‑only balancing loss with no auxiliary loss, activating roughly 13 B parameters per token. GLM selects the top‑8 experts (out of 288) via a softmax router plus a shared expert. Qwen‑Next activates the top‑10 experts from a pool of 512, with a gated residual that modulates the contribution of each expert.
All specifications are taken from vendor documentation and cross‑checked against the vLLM and SGLang serving recipes 【1†url】【2†url】. Independent performance numbers are drawn from the Artificial Analysis harness 【2†url】. This dual‑source validation ensures that the matrix reflects both the claimed design and observed serving behaviour.
GLM‑5.3‑Flash dedicates 34 of its 45 transformer layers to K‑dimensional linear attention (KDA) followed by a short convolution and a top‑k selection of 2 048 tokens. Every fourth layer (3, 7, … 43) inserts a NoPE sparse MLA block to preserve long‑range coherence that pure‑linear models tend to lose.
A key efficiency component is IndexPool, which merges four indexer key vectors into a single weighted vector. This compression cuts the attention compute by 3.01× and shrinks the KV cache by 4.44× compared with the baseline GLM‑5.3, delivering roughly a 3× end‑to‑end serving improvement in Z.ai’s launch material【{"cursor":0,"id":0}】. The design places most of the cost on the indexer; if the indexer remains fast, the model scales well to the native 1 M‑token window.
DeepSeek V4 Flash combines two orthogonal compression tricks:
Compressed Sparse Attention (c4a) uses a two‑stream overlapping compressor that projects every 4 tokens into a weighted sum, then runs a Lightning Indexer to pick top‑k blocks. This reduces the KV cache by about 4× while retaining a proxy representation for out‑of‑window tokens rather than discarding them.
Heavily Compressed Attention (c128a) further compresses the sequence dimension by ≈128× and applies dense attention over the short compressed sequence, acting as a global‑retrieval layer for the 1 M‑token context.
The first two layers are pure sliding‑window; the Hierarchical Compressed Sparse Attention (HCA) layers dominate the latter two‑thirds of the trunk. In the vLLM serving recipe, DeepSeek reports 27 % of the per‑token FLOPs and 10 % of the KV cache required by its predecessor V3.2 at a 1 M context【{"cursor":2,"id":15}】. These gains are achieved by pushing most of the work onto the compression ratios; the speed‑up numbers are measured against DeepSeek’s own baseline and are not directly comparable across vendors.
Qwen 3.8‑Flash‑Next adopts a hybrid pattern of Gated DeltaNet (GDN) linear‑attention layers and Query‑based Sparse Attention (QSA) layers (≈3 GDN : 1 QSA). GDN maintains a fixed‑size recurrent state; a sigmoid gate acts as a selective eraser, constantly refreshing the summary while keeping memory usage constant. QSA provides precise long‑range retrieval when needed.
Alibaba reports 10.2× prefill and 6.6× decode kernel speed‑ups at 1 M tokens versus dense attention, using a native 262 K‑token window extended to 1 M via YaRN (Yet‑Another‑RoPE‑Extension)【{"cursor":3,"id":0}】【{"cursor":3,"id":1}】. The model also leverages a 51‑billion‑parameter N‑gram embedding table that adds capacity with negligible per‑token cost.
Design trade‑offs
All speed‑up figures are vendor‑reported against internal baselines; direct cross‑model comparison should therefore focus on architectural fit rather than raw percentages.
Qwen 3.8‑Flash‑Next stores most of its knowledge in a 51‑billion‑parameter n‑gram embedding table (often called an “engram”). The table holds roughly 20 million entries and is addressed by deterministic O(1) look‑ups rather than a dense mat‑mul. Because the table resides in system RAM instead of GPU memory, it never enters the per‑token FLOP budget. A learned gate injects the lookup results into the early trunk layers, which lets the model claim that ~6 B active parameters can match the performance of ≈49 B‑active dense models on coding tasks【3†url】【3†content】. The RAM‑/VRAM split is the primary reason Qwen’s GPU footprint is the smallest of the three flash‑tier models.
DeepSeek V4‑Flash bets on a massive expert bank. The model contains 256 routed experts plus one shared expert per layer across 43 layers. Each expert is a SwiGLU block (three 4096 × 2048 matrices ≈ 25.17 M parameters). The expert bank therefore totals 278.1 B parameters, accounting for 97.8 % of the checkpoint. By contrast, the dense attention stack is only 4.96 B parameters, yet it represents 39.3 % of the weight‑tensor traffic per token, establishing a hard floor for per‑token cost【0†url】【0†content】. The checkpoint is quant‑aware‑trained (QAT) to MXFP4 for routed experts and MXFP8 for the rest, and no BF16 variant is provided, meaning the reduced precision cannot be recovered by re‑quantising【5†url】【5†content】.
GLM‑5.3‑Flash combines a large expert bank with native multimodality. The checkpoint contains 288 routed experts, of which 8 are active per token, alongside a single shared expert that is always evaluated. In addition to the language stack, the model folds a 24‑layer ViT vision encoder into the same checkpoint, enabling image and video inputs but adding preprocessing overhead at serving time【4†url】【4†content】. GLM ships two precision variants: an FP8 checkpoint that occupies roughly 306 GiB on disk (≈ 328 GB) and a BF16 checkpoint that is about 640 GiB【6†url】【6†content】. The larger BF16 footprint pushes the minimum deployment from a 4‑GPU pod to an 8‑GPU node, whereas the FP8 variant can run on a 4‑GPU system with sufficient host RAM for the 51 B engram‑style table in Qwen.
Deployment math therefore hinges on where capacity lives:
Understanding these trade‑offs helps practitioners match model capacity to available hardware and the latency budget of their serving environment.
All three flash‑tier MoE models share a hard constraint: every expert weight must reside in GPU memory, even though only a subset is active per token. Consequently, the smallest checkpoint—the DeepSeek V4 Flash FP4 + FP8 file at ≈ 166.9 GB—already exceeds the memory of any single GPU, and the minimum viable deployment always involves at least two GPUs【1†url】.
| Metric | Detail |
|---|---|
| Checkpoint size (FP4 + FP8) | ~166 GB (preview 158‑160 GB) |
| Minimum GPU class | Hopper (H100/H200) |
| Smallest entry point | 2 × H200 (282 GB combined VRAM) – can run moderate context lengths |
| Recommended tensor‑parallelism | TP = 4 on a single H100/H200 for FP8/NVFP4; TP = 8 on an 8‑GPU H200 pod |
| BF16 variant | Not provided (QAT‑only) |
| Alternative formats | llama.cpp GGUF (INT3/INT4) ~103 GB on disk, ~110 GB RAM floor; 90‑130 t/s prefill, 6‑7.5 t/s decode on one H200【2†url】 |
| ROCm support | MI300X, MI325X, MI355X |
| Metric | Detail |
|---|---|
| FP8 checkpoint | 306‑331 GiB, far above two H200s’ combined 282 GB |
| BF16 checkpoint | ~640 GB |
| Minimum GPU class | Hopper (H100/H200) |
| Viable floor | 8‑GPU Hopper‑class node (8 × H200 recommended for the full 300 K‑1 M token window) |
| ROCm support | Limited to MI355X / gfx950 |
| Community packages | Unsloth GGUF (day‑one), KTransformers hybrid CPU/GPU serving |
| No two‑GPU entry point | All experts must be loaded, so 2 × H200 is insufficient【3†url】 |
| Metric | Detail |
|---|---|
| NVFP4 checkpoint | ~130 GB (smallest of the tier) – requires Blackwell‑class GPUs (NVFP4 path) |
| BF16 checkpoint | ~423 GB |
| Minimum TP | TP = 2 on a GB300 card is possible; TP = 4 is the recommended baseline |
| Target hardware (vendor claim) | NVIDIA DGX Spark, Macs with 96‑128 GB unified memory, AMD Strix Halo systems with 128 GB |
| Current uncertainty | Sources disagree on whether the model truly runs on edge‑class machines or remains effectively multi‑GPU‑only, given its preview status【4†url】 |
| FP8 path | Runs on Hopper (H100/H200) like the other two models; 8‑GPU H200 pod works |
Because the only model that fits a two‑GPU pod is DeepSeek V4 Flash, the rental cost gap between the cheapest and the most demanding floor is roughly four‑fold. A typical cloud provider’s hourly rate for a 2 × H200 pod is about $0.45 / hour, whereas an 8‑GPU H200 node (or equivalent Blackwell node for Qwen) costs around $1.80 / hour. When serving long‑context workloads—where KV‑cache memory adds tens of gigabytes per token—this differential directly translates into higher throughput per dollar for DeepSeek, but at the expense of a lower maximum context length compared with GLM‑5.3‑Flash or Qwen‑Flash‑Next.
In summary, the self‑hosting floor is the primary differentiator: DeepSeek offers a two‑GPU entry point, GLM requires an eight‑GPU Hopper node, and Qwen sits between the two, dependent on whether its Blackwell‑only NVFP4 path can be realised on edge hardware. The choice therefore hinges on budget, available GPU class, and the required context window.
Intelligence Index – Artificial Analysis’s independent harness places the flash‑tier models on a common “Intelligence Index” (higher is better). GLM‑5.3‑Flash scores 57, just three points shy of the full‑size GLM‑5.3 (60) while costing roughly one‑tenth the token price Regolo 2026. Qwen3.8‑Flash‑Next follows at 56, and DeepSeek V4 Flash 0731 (max‑effort) reaches 50 after a post‑training refinement that added +10 points over the April preview Regolo 2026.
Raw output speed – Measured tokens per second (tok/s) on first‑party APIs show a clear split:
Time‑to‑first‑token (TTFT) is roughly equal across the three, hovering around 1.5 s Regolo 2026.
Token consumption – To complete the Intelligence Index run, GLM‑5.3‑Flash burned about 150 M tokens (≈ 36 % above the class median of 110 M) while generating at roughly half the median speed Regolo 2026. DeepSeek’s 0731 checkpoint used ≈ 206 M tokens, down from 234 M in the April preview.
Benchmark leadership – Independent and vendor‑reported numbers diverge, so the article flags the most reliable signals:
Caveats – Most non‑AA numbers are vendor‑reported under heterogeneous harnesses, and GDPval Elo scores vary between trackers. DeepSeek’s production‑grade concurrency limit of 2,500 requests (≈ 5 × the typical 500) is a material operational advantage Regolo 2026. Developers should therefore balance raw speed and benchmark crowns against hosting costs, concurrency needs, and token‑price efficiency.
Modality – The three flash‑tier models expose very different front‑ends for non‑text data.
deepseek‑v4‑flash‑vision‑exp) that must be addressed with a different model name, meaning a visual workload uses a distinct checkpoint with its own evaluation history 【3†url】.Licensing – Legal terms diverge sharply.
Reasoning effort controls – All three expose configurable “thinking” modes, but the defaults and granularity differ.
Pricing (token‑level, version‑sensitive) – Prices listed are as of September 2026.
| Model | Input ( $/M tokens) | Cached input | Output ( $/M tokens) | Blended cost* |
|---|---|---|---|---|
| GLM‑5.3‑Flash | $0.15 (promo $0.075) | $0.03 (promo $0.015) | $0.50 | ≈ $0.24 (promo $0.12) 【7†url】 |
| DeepSeek V4‑Flash | $0.14 peak / $0.11 off‑peak | $0.015 peak / $0.007 off‑peak | $0.28 peak / $0.14 off‑peak | ≈ $0.18 (off‑peak) 【7†url】 |
| Qwen 3.8‑Flash‑Next | $0.16 input / $0.15 input (different providers) | varies | $0.47 output | ≈ $0.23 blended 【7†url】 |
*Blended cost assumes a typical 3:1 input‑to‑output token mix and includes cache‑hit discounts where available.
Legacy naming pitfall – DeepSeek’s older identifiers (deepseek‑v4‑flash, deepseek‑v4‑flash‑vision‑exp) now automatically resolve to the newer V4.1‑Flash checkpoint and are billed at Flash‑tier rates. This routing change took effect on 14 Sept 2026, so any benchmark or cost measurement logged after that date under the legacy name actually reflects V4.1‑Flash behavior 【8†url】.
Takeaway – When benchmarks become stale, the choice boils down to:
These contract‑level distinctions are likely to dominate long‑term deployment decisions more than raw benchmark scores.
Interactive, high‑throughput chat / pair‑programming
Agentic, tool‑use, multimodal pipelines (terminal agents, browsing, heavy documents)
Cache‑heavy, high‑volume pipelines with repetitive prefixes (tool schemas, long system prompts, few‑shot scaffolds)
Edge or desktop deployment
| Model | Minimum GPU pod | Approx. hourly cost multiplier* |
|---|---|---|
| DeepSeek V4 Flash | 2 × H200 (≈ $30 / hr) | Baseline |
| GLM‑5.3 Flash | 8 × H200 (≈ $120 / hr) | ~4 × higher 【source: yottalabs.ai – “DeepSeek … 2‑GPU pod … GLM … 8‑GPU node”】 |
*Figures are illustrative based on typical cloud‑provider pricing for Hopper‑class GPUs; the exact rate depends on the provider and region.
Action: Pin the exact model revision (e.g., deepseek-v4.1-flash-20241010, glm-5.3-flash-20241012, qwen3.8-flash-next-20241008) in any deployment script and re‑run benchmark/price checks before finalizing a production stack.