What "black technologies" does DeepSeek V4.1 Flash adopt? It is so surprisingly competitive that it (almost) forced the Pro version to withdraw from the market.

The other day, DeepSeek V4.1 Flash was officially launched. Boasting both high speed and strong performance, it has drawn widespread attention and sparked a wave of public testing.

For example, after conducting a very comprehensive test on V4.1 Flash, the well-known testing organization Artificial Analysis gave it an Intelligence Index score of 40, which surpassed DeepSeek's V4 Pro that has a far larger parameter count.

Of course, this is not unexpected. After all, DeepSeek itself also reached this conclusion, stating that it would take V4 Pro offline and directly route related requests to V4.1 Flash. By the way, DeepSeek has revised its decision twice in a row: first, it delayed the offline of V4 Pro:

Then it abandoned the plan to take V4 Pro offline and continued to provide API call services:

In addition, there are other highlights in Artificial Analysis's systematic evaluation:

• DeepSeek V4.1 Flash has made remarkable progress in agent capability and long-context reasoning.
• DeepSeek V4.1 Flash ranks first on AutomationBench-AA with a score of 69%, tied with GPT-6 Astra (69%) and slightly higher than Grok 4.6 (67%).
• DeepSeek V4.1 Flash is one of the most verbose models it has measured, with 89k Tokens per Intelligence Index task.
• Despite such high verbosity, DeepSeek V4.1 Flash only costs $0.27 per Intelligence Index task, which is mainly attributed to its low pricing.

Sebastian Raschka even believes that the innovation of DeepSeek V4.1 Flash is so significant that it should be named DeepSeek V5.

So how on earth did DeepSeek V4.1 Flash achieve this? How did the "small fat fish" with 552B parameters surpass the "big fat fish" V4 Pro with 1.6T parameters? Below we will conduct an in-depth interpretation based on its official technical report.

Report link: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

This is a report centered around KV cache

Let's first look at the title of the technical report: "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression". Pushing the limits of KV cache compression. After going through the architecture chapter, we found that this is not a rhetorical flourish - almost every design can be traced back to the same goal: compressing the KV cache as much as possible.

Let's first look at several sets of numbers. The global KV cache of V4.1-Flash (the part that always resides in HBM) is compressed to 890 bytes per token, which is about 1/4 of that of V4-Flash; if compared with DeepSeek-V1's 389,120 bytes, there is a 437x gap. The persistent KV cache (the part stored in SSD or host memory for prefix reuse) is compressed to about 1/8 of that of V4-Flash.

The model itself is a multimodal MoE with 552B backbone parameters plus 196B Engram parameters. It natively supports 1 million token context, is pre-trained on 45T tokens of multimodal corpus, only activates 8B parameters per token in the prefill phase, and activates 16B parameters in the decode phase.

Comparison of global KV cache size per token across generations of DeepSeek models

The combination of these numbers is the true meaning of "552B beats 1.6T": the core of comparison is how much VRAM the model occupies, how much data it needs to move from SSD, and how many prefill recalculations it needs to perform in the same agent call.

After sparse attention, storage and data movement become the costly parts

To understand why DeepSeek focused all its efforts on KV cache, we must first see the change of bottlenecks.

From DSA in V3.2 to CSA in V4, sparse attention has sufficiently reduced the computational cost of long sequences. However, the workload of long-running agents has a feature: input-heavy.

For a coding agent that runs for several hours, each tool call will generate a new prefill request, the context only increases and never decreases, while the number of tokens that actually need to be generated is very small.

Therefore, after "computation" is no longer the bottleneck, "storage" and "data movement" have become the new bottlenecks.

The report splits this new bottleneck into three parts:

• HBM capacity limits how much global KV of concurrent requests can be loaded at runtime at the same time;
• SSD and host memory capacity limit how long the prefix cache can be stored and how high the hit rate can be;
• IO and interconnection bandwidth limit the speed of cache migration and loading.

The combination of these three issues determines the throughput upper limit and unit cost of an agent service. In the V4 era, DeepSeek has mixed CSA and HCA to achieve sequence dimension compression, while V4.1-Flash chooses to make improvements at three levels: architecture, precision, and deployment.

CED: Cut half of the prefill computation directly

The language backbone of V4.1-Flash has 40 layers, which is split into two halves: the first 20 layers are Causal Encoder, and the last 20 layers are Decoder. This structure is called Causal Encoder-Decoder, abbreviated as CED. This is also the source of the asymmetric design "8B activated for input, 16B activated for output" on DeepSeek's official release page.

Interestingly, Peter Gostev from Arena even asked Astra to read the DeepSeek v4.1 Flash paper, and made a very intuitive 3D comparison between its architecture and the original Transformer architecture. He also shared the result, you can zoom in and view each element side by side:

https://transformer-architecture.petergostev.chatgpt.site/

The key point is: how does the global KV of the 20 decoder layers come from?

In conventional Transformer, K and V of each layer are calculated from the hidden state of that layer, so processing a prompt must run all 40 layers.

CED does not do this: The KV entries of each decoder layer are directly mapped from the hidden state of the 20th layer (the last layer of the encoder) using their respective projection matrices. In other words, in the prefill phase, you only need to run the first 20 layers, and the global KV cache of the upper half can be obtained at a very low cost. When the sequence length is much larger than the window, the complexity of prefill is reduced from O(NL) to approximately O(NL/2), almost halved.

This idea is inherited from Microsoft's YoCo (You Only Cache Once), but CED makes structural enhancements: YoCo makes the upper half directly share the same KV cache generated by the lower half, while CED configures independent projection weights for each layer, which expands the effective capacity of KV and generation depth while keeping the "only calculate half" advantage.

There is also a cost, which appears in sliding window attention. SWA is still computed layer by layer, and the local K and V of each layer come from the hidden state of this layer, so as to maintain the computation depth of local information. But this means that during prefill, the decoder still needs to additionally process n_win × L/2 tokens to complete the SWA state, which becomes a new overhead in the scenario of multiple rounds of short prompts.

DeepSeek's solution is "approximation": Since existing studies have shown that the actual effective receptive field of SWA is much smaller than the theoretical n_win × L/2, only the last n_win tokens of the prompt are replayed. This is the Decoder SWA Bounded Replay that will be discussed later.

CSA2: Three compression dimensions are fully utilized for the first time

CED handles computation, while CSA2 (Compressed Sparse Attention 2) handles storage.

The report summarizes the compression space of KV cache into three mutually multiplied dimensions: entry size (GQA reduces the number of KV heads, MLA makes all heads share a small latent), sequence dimension (m tokens are compressed into one entry, V4's CSA and HCA belong to this category), and layer dimension (let some layers reuse the cache and selection results of other layers).

The report also introduces three previous achievements:

• IndexCache reuses Top-K indexes across layers, which saves the computing power of the indexer rather than the cache;
• YOIO calculates sparse routing once and shares it across the network, but full-network sharing will hurt performance;
• HySparse lets sparse layers reuse the KV cache of dense layers, but it still retains the full attention layer.

None of the three covers all three dimensions, while CSA2 aims to utilize all of them at the same time.

Its approach is to statically assign one of three modes to each CSA2 layer.

• Full mode: Calculate the main KV and indexer Q by itself, project indexer K from main KV, run the complete indexing process to generate fresh Top-K indexes.
• Reindex mode: Reuse the main KV and indexer K of a previous layer, but use its own indexer Q to re-score and select its own Top-K - the cache is shared, but the selection is independent.
• Reuse mode: The most memory-saving mode, both main KV and Top-K index are fully reused, directly perform sparse attention, without even calculating the indexer Q.

The common point of the three modes is that each layer still retains its own global Q and SWA KV, so the expression capabilities between layers are not leveled. For specific configurations, please refer to the original report.

FP4 and Bounded Replay

Apart from architecture, there are two other optimizations.

The first optimization is on precision. V4 has already performed FP4 quantization-aware training on the Q and K of the indexer, and V4.1-Flash extends FP4 to the main KV cache. It selects the E2M1 format with one E4M3 scaling factor per 16 channels, which is close to NVFP4 but omits its secondary global scaling. The report also rigorously demonstrates this approach, interested readers can check it by themselves.

The second optimization is on deployment, which is the SWA Bounded Replay mentioned earlier. Since the dependencies of SWA accumulate layer by layer, to accurately reconstruct the SWA KV of L layers, you need to replay L × n_win tokens. The V4 technical report once proposed the idea of "Zero SWA Caching", but this cost was proven to be too high in production deployment. V4.1-Flash simply accepts approximation: only replay the latest n_win tokens (n_win = 128 in the configuration), and truncate SWA to within the replay segment.

The benefit brought by this "accepting approximation" is very obvious. In V4's deployment, SWA KV occupies nearly half of the persistent cache capacity, and its access pattern does not match the long-term retention strategy of persistent cache at all - global KV has long-tail reuse value, but SWA KV is only useful in the minute-level window within an active session, and becomes dead data once the session ends.

Therefore, V4.1-Flash moves the entire SWA KV out of the persistent cache, and puts it into a distributed memory pool composed of 10% of the host DRAM on each machine, with a TTL of only a few minutes, serving the vast majority of concurrent sessions with high turnover; the global KV continues to reside on SSD, ensuring a lifecycle of at least 72 hours.