Qwen and Zhipu AI open-sourced their large language models overnight, both are cheaper than DeepSeek, and domestic large language models are caught in a fierce price war.

Reported by AI Technology Review on August 27, last night, the Alibaba Qwen large model team open-sourced the multimodal MoE model Qwen3.8-Flash-Next, which is a pilot preview version built on the next-generation Qwen4 architecture.

The main model of Qwen3.8-Flash-Next has 125B parameters, with an additional 51B N-gram Embedding, activates 6B parameters per token, natively supports 262144 token context, and can be extended to 1 million tokens via YaRN.

Compared with Qwen3.7-Plus, Qwen3.8-Flash-Next reduces training and inference costs, with training overhead only about 1/9 of the former, but delivers stronger performance in programming and office tasks, and the model outperforms the official DeepSeek-V4-Flash release in the vast majority of benchmark tests released by the Qwen team.

In terms of pricing, Qwen3.8-Flash will soon be launched on the Qwen AI platform to provide external API services, with the input price of 1 CNY per million Tokens and output price of 3 CNY, which is 33.33% lower than the input price of DeepSeek-V4-Flash during off-peak hours, and the output price is also 33.33% lower.

Last night, Zhipu also open-sourced the first native multimodal model GLM-5.3-Flash (320B-A18B) of the GLM-5 series, which is also the previously viral anonymous model Ox-Alpha ("Niulai" model) that has driven down prices significantly. Its new model has a total of 320B parameters, with programming performance comparable to Claude Opus 4.8 in the Zhipu self-developed Z.ai Code Bench sensory evaluation, and the price is reduced to 1/10 of GLM-5.3, which is 1/20 of GLM-5.3 during the limited-time discount and 1/40 of Opus 4.8.

▲Price comparison between Qwen3.8-Flash-Next and DeepSeek-V4 series models (chart by AI Technology Review)

Qwen3.8-Flash will be launched first on "Qwen Office", and its API will be available soon.

On the Qwen AI platform, the model is provided under the name qwen3.8-flash. The Qwen AI platform supports industry-standard protocols, including OpenAI-compatible Chat Completions and Responses API, as well as Anthropic-compatible interfaces. Meanwhile, the Qwen API supports the Anthropic API and OpenAI Responses protocols, which can be directly used with Claude Code and Codex.

Its blog mentions that Qwen released the structural changes in advance to let the community test them before launching the full Qwen4 model family.

Under Qwen's Twitter comment section, many overseas developers can't wait to experience Qwen's new model, and some netizens even praised it bluntly that "this is so powerful", "we don't need OpenAI", "Qwen has the potential to surpass Opus in the future".

At present, the model weights of Qwen3.8-Flash-Next have been released on Hugging Face and ModelScope, which is the production version with default 1 million context and built-in official tools.

Technical Report:

https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf

Hugging Face Open Source Address:

https://huggingface.co/Qwen/Qwen3.8-Flash-Next

ModelScope Open Source Address:

https://www.modelscope.cn/models/Qwen/Qwen3.8-Flash-Next

01.

More Than Ten Tests Outperform

DeepSeek‑V4‑Flash, Claude‑Opus‑4.6

Qwen3.8-Flash-Next has systematically upgraded the model around four aspects: Attention, Residual, Embedding and Optimization, which further optimizes computing efficiency, model capacity and training stability while improving model capabilities.

▲Qwen3.8-Flash-Next Model Architecture

The benchmark test results released by the Qwen large model team show that Qwen3.8-Flash-Next surpasses Qwen3.8‑27B, Qwen3.7‑Plus, DeepSeek‑V4‑Flash‑0731, Claude‑Opus‑4.6 (Max) in the vast majority of agent, programming, general capability and multimodal scenarios, taking the first place.

Specifically, in terms of pure text capability, Qwen3.8-Flash-Next takes the first place in 10 tasks, outperforming Qwen3.8‑27B, Qwen3.7‑Plus, DeepSeek‑V4‑Flash‑0731, Claude‑Opus‑4.6 (Max).

In terms of programming capability, Qwen3.8-Flash-Next ranks first in agent programming DeepSWE1.1 and SWE-bench Pro multilingual software engineering, only slightly inferior to DeepSeek in the warehouse-level code generation test.

In terms of agent capability, Qwen's new model takes the first place in the weighted scores of long-cycle office tasks, professional work tasks, cutting-edge agent tasks, and real tool call test sets, while its one-time full pass rate in cutting-edge agent tasks is lower than that of DeepSeek-V4-Flash.

In terms of general capability, Qwen3.8-Flash-Next ranks first in instruction following, scientific reasoning and competitive code, and is only slightly weaker than Claude-Opus-4.6 in multidisciplinary comprehensive reasoning capability.

In terms of multimodal capability, this model outperforms the other three models in 13 tasks and takes the first place.

In the multimodal agent capability section, the test mainly examines whether the model can understand screenshots, operate mobile phones and computers, reproduce APP and web development, and call multimodal tools. Qwen3.8-Flash-Next ranks first in all parts of this section.

The general multimodal capability mainly investigates the model's ability to understand images, long videos, charts and mathematics. Qwen3.8-Flash-Next outperforms the other three models in the vast majority of tests, and its score is only lower than DeepSeek in the scientific research chart analysis test without special reasoning prompts.

Qwen also released the comparison results between Qwen3.8-Flash-Next-Base (with 6B activated parameters) and the base models of Qwen3.8-27B and Qwen3.7-Plus. It achieves the optimal results in 8 out of 14 benchmarks, among which the 51B N-gram embedding parameters are deterministically addressed and do not enter the matrix multiplication budget for each token.

At present, the number of parameters required by the base model and the length of the context window are constantly increasing. The core problem is no longer how large the scale can be achieved, but how high the efficiency of scaling can be. Based on this, the Alibaba Qwen large model team has made architectural innovations, and the open-sourced Qwen3.8-Flash-Next is a phased key achievement of Alibaba moving towards this goal.

This model is an experimental preview model built on the Qwen4 architecture, whose core is to rethink deeply how the core components of modern large language models can work together in large-scale operation scenarios.

02.

Four Major Architectural Innovations

Make the New Model Stronger, More Stable and More Cost-Effective

Qwen3.8-Flash-Next mainly includes four major innovation points:

1. Combination of GDN and QSA (Tongyi Sparse Attention) to achieve efficient memory and accurate retrieval

Traditional global attention can directly access all historical tokens, but the longer the context, the higher the cost of calculation and KV Cache memory access.

It inherits the architectural design of Qwen3.5 and adopts the GDN+Attention Hybrid architecture: three out of every four layers use Gated DeltaNet (GDN) to continuously compress historical information into a fixed-size state; the other layer retains global Attention, which is responsible for accurately retrieving global information.

Researchers further introduce Qwen Sparse Attention (QSA) to global Attention to reduce the calculation under long sequences by only focusing on important contexts. QSA filters important contexts at the micro-block granularity through a compressed lightweight Indexer, reducing the Attention overhead of long sequences. This not only reduces the actual Attention calculation amount, but also reduces the context indexing cost of "looking for important contexts".

It can be understood that GDN is responsible for efficiently "remembering", and QSA is responsible for accurately "searching".

On 1M tokens, the Attention Kernel of QSA achieves up to 7.6x and 4.9x acceleration in the Prefill and Decode stages respectively. Under the experimental setting close to the online high cache reuse scenario (Prefix Cache hit rate is 90%), the Prefill throughput of Qwen3.8-Flash-Next under 1M context reaches 8.6 times that of Qwen3.7-Plus.

2. Introduce Gated Residual (GR) to provide more transmission paths for information

In the traditional Transformer architecture, each layer continuously reads and writes information on the same Residual Stream. As the network deepens, early features are continuously mixed with subsequent information, and important information is more likely to be gradually diluted.

GR can be regarded as a combination of two ideas: on the one hand, it extends the residual stream to a multi-branch design following Hyper-Connection; on the other hand, it integrates the element-wise dynamic gating of GatedNorm into residual read. Finally, the original single residual stream is extended into 4 parallel branches, and the model can dynamically determine how much information to read from different branches and how much information to write to different branches according to the current content.

This can be understood as expanding one information channel into multiple lanes: some branches are responsible for local information transmission, while others can directly retain early information to very deep network layers. In the actual analysis, researchers also observed that one of the branches naturally forms a long-distance channel connecting the first layer of Attention and most of the middle and later layers.

GR further simplifies Hyper-Connection: when read/write is flexible enough, additional branch mixing has no obvious benefit, so it can be removed directly to reduce memory access overhead and unstable factors. The normalized Gate can effectively suppress activation outliers and improve training stability; Residual State supports storage with FP8, further reducing memory access overhead.

3. Introduce N-gram Embedding to expand model capacity at low cost

Inspired by the work of Per-Layer Embedding in Gemma 3n and DeepSeek Engram, researchers further introduced N-gram Embedding to expand model capacity from the dimension outside Transformer parameters.

Ordinary Embedding looks up tables according to a single token; N-gram Embedding looks up tables by combining the current token with the local context composed of several previous tokens, providing additional representations for common phrases and local patterns. Its core advantage is that it can add a large number of parameters while hardly increasing the calculation amount of each token.

Qwen3.8-Flash-Next additionally introduces 51B N-gram Embedding parameters. Since the query location can be determined in advance, these parameters can be stored in Host Memory, overlapping with model calculation through asynchronous Prefetch, without occupying GPU video memory for a long time. Finally, the model only uses one layer of N-gram Embedding at the front part, adding a large-scale "local pattern memory bank" at a low extra cost.

4. Optimization: Optimize the usage of Muon in large model training

Qwen3.8-Flash-Next is trained with Muon Optimizer, and further optimizes the usage of Muon in large model training around three key issues: orthogonalization accuracy, parameter division of labor between Muon and AdamW, and splitting of fused parameter matrices.

For parameters that are truly used as two-dimensional linear mappings, such as the main weights in Attention, GDN and MoE Expert, researchers use Muon; for Embedding, MoE Router, GR low-rank parameters, etc., AdamW is still used.

For the fused and stored QKV, SwiGLU and GDN Projection in engineering implementation, split them according to the corresponding independent linear transformations first, and then perform orthogonalization respectively.