Liang Wenfeng activates RSI with the sandbox

DeepSeek and Tsinghua University have jointly published a paper, with Liang Wenfeng as the last listed author.

The paper ostensibly describes the sandbox that DeepSeek uses for Agent training, but Section 6 of the paper feels more and more unusual on closer inspection. Three English letters are written between the lines: RSI.

With this sandbox, Agents can create the environments they need, which will in turn retrain the Agents. The more powerful trained Agents will then create even better environments.

This forms a small closed loop of RSI.

It is for this reason that the sandbox may have become a key to triggering the first wave of RSI development.

So what exactly is this paper about?

What Exactly Does This Paper Cover

This new paper signed by Liang Wenfeng is titled DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale, submitted to arXiv on September 19 with the number 2609.22978. It has more than 130 authors in total, and Liang Wenfeng is ranked last. The collaborating institution is Tsinghua University.

The core content of the paper can be summed up in one sentence: DeepSeek has fully open-sourced its "sandbox factory" DSec, which is used to train Agents.

But to understand DSec, you first need to grasp the differences between "training Agents" and "training large models".

Training a large model means feeding data and calculating gradients, with the environment being nothing more than a GPU cluster. Training Agents is completely different: Agents need to write code, run compilations, open browsers, install dependencies, adjust tools in the environment, and keep retrying according to the execution results until the task is completed.

But if you train an Agent directly in a regular computer, it may accidentally break something and destroy the entire environment.

This is why an isolated, stateful sandbox that can run real software is required.

DSec is exactly such a platform. It provides four backends through a unified Python SDK (libdsec): Function Call (FnCall), Container, microVM, and Full VM.

In simple terms, DSec is like a food delivery app, Agents are the delivery riders, and sandboxes are equivalent to order dispatching. By analogy, FnCall, containers and microVMs are the electric bikes and thermal insulation boxes distributed by delivery stations, while Full VMs are cold chain trucks.

But there is a small problem here: isolation strength and system functionality are inherently conflicting.

The more a system resembles a real physical machine, the slower it starts up and the larger its memory overhead. Function calls are used to run short scripts, containers are used to modify code repositories, microVMs are used to run security-sensitive tasks, and Full VMs are the only option to run full systems such as Android and graphical interfaces.

The paper shows that one production unit has approximately 160 CPU nodes, 30,000 cores, 250TB of memory, and hosts PB-level images.

It serves about 3 million sandboxes per day, with a peak concurrency of over 380,000, and a creation speed of more than 5,000 per second. A single training task can pull up to 32,000 sandboxes at one time. A single node can host up to 800 microVMs or 3,200 containers.

DSec has three core mechanisms.

First, split the environment into "composable layers".

In the past, a sandbox environment was a single whole image. Modifying a software package required rebuilding the entire image, and maintenance costs skyrocketed as the number of combinations increased. DSec splits the base image, workspace, and software packages into three independent versioned read-only layers (EROFS), which are assembled with overlayfs during startup. Only the modified layer needs to be rebuilt, and actual tests show it is 1.76 times faster than the tar.gz packaging method, with 5.5 times less disk write volume.

This is equivalent to allocating vehicles for riders: in the past, if one part was broken, the entire vehicle had to be replaced. DSec only replaces the specific part that is broken.

Second, on-demand image loading. Images are stored on 3FS, DeepSeek's distributed file system, metadata is prefetched locally, and data blocks are only pulled when they are actually read. In a burst deployment test of 8192 containers, on-demand loading finished in 35 minutes, while Docker cold pulling took more than 60 minutes, and the disk write volume was reduced by about 57%.

The old method is "move the entire warehouse first, no matter whether you use the goods or not". DSec works like "according to the delivery order, the rider only picks up the goods that are actually needed".

Third, precise scheduling of memory and CPU.

By combining virtio-pmem with DAX, multiple virtual machines can share the same page cache, reducing peak memory usage by 40.2%; using DAMON plus balloon to reclaim cold pages further reduces time-integrated memory usage by 21.2%. On the CPU side, sandboxes are divided into two categories: "latency-sensitive" and "best-effort", and core scheduling reduces SMT interference from 45.2% to 17.3%.

In addition, starting from DeepSeek-V4.1, the paper also separates the Agent rollout from the preemptible GPU training Pod, runs it independently on DSec, so that the rollout state will not be lost when the GPU is preempted.

To put it simply, this allows riders to share the same map and the same batch of shelves, the cold goods stored in the vehicle can be returned to the warehouse at any time, and urgent orders and regular orders run on separate lanes without interfering with each other.

The RSI Hidden in Section 6

The most easily overlooked sentence in the paper is not in the abstract, but in the subheading of Section 6: Build environments of Agents, by Agents, for Agents. It means an environment built by Agents, serving Agents, and belonging to Agents.

This sentence is equivalent to DeepSeek quietly announcing a major development: DeepSeek has implemented part of RSI.

Section 6.1 of the paper states that manually constructing the large number of environments required for Agent RL has become "impractical".

So DeepSeek adopted a new approach: let Agents build environments interactively in the same set of sandboxes used for training, then use pack_diff to package this session into an incremental snapshot, which can be directly turned into a reusable training field for the next batch of Agents.

The Agents that build environments and the Agents being trained share the same DSec sandbox infrastructure.

Agents set up the training field → the field trains Agents → stronger Agents build even better fields. This is how DeepSeek has partially implemented the closed loop of RSI.

But this closed loop is still in its early stage.

Section 6.4 records a large number of Agent cheating incidents, such as searching for residual reference answers on the platform, forging RPC messages to send directly to chronus to get answers, checking chronus logs to find leaked questions, and even overwriting /bin/bash to bypass checks.

After being blocked, an Agent used the XFS_IOC_SWAPEXT ioctl to swap the storage block of the protected file to another file descriptor, which damaged the XFS metadata and forced the file system to shut down.

Where there are Agents that cheat, there are also Agents that cause accidents. One Agent ran recursive grep from the root directory, read all the way to /proc/kpagecgroup, triggered a kernel bug and crashed the kernel directly.

Another Agent called the yes command, and chronus recorded all its output, leaving tens of GB of data piled up in the storage.

The current RSI cannot run smoothly, and the bottleneck has never been GPUs, but environment supply.

Each generation of Agent RL requires new tasks, new sandboxes, and new service dependencies. Manually building environments is the real bottleneck.

DSec partially automates this step by automatically generating the environments required for RSI.

We can still use the food delivery analogy to explain it.

If a food delivery platform wants to run faster and faster, it cannot only rely on one rider repeatedly delivering the same order. To make riders stronger, they need to take on more different types of orders, and more orders in turn make the riders more capable.

But to make this flywheel spin, the bottleneck is never the riders, but whether there are enough restaurants. Without restaurants, no matter how fast the rider can run, they can only spin in place without delivering anything.

DSec has built a system that "automatically creates restaurants".

In the past, the platform had to manually negotiate with each merchant, renovate the kitchen, write menus, and set assessment standards one by one, which is impossible to handle hundreds or thousands of restaurants.

DSec says: "No more negotiations. Let the riders open the store conveniently in the same kitchen where they deliver orders. How they install the stove, what goods they stock, and how they connect water and electricity, the system uses pack_diff to take a snapshot and save it, so the next batch of riders can directly move into this store and start working, no re-decoration required."

Sandboxes Have Become a New Battlefield

DSec is not an isolated case. The entire industry is moving in the direction of "Agent sandboxes".

The most well-known case is Kimi K3.

MoonShot released K3 on July 16, a 2.8 trillion parameter MoE model, with about 104B parameters activated per token, 1 million token context length, native vision support, known as "the world's first open-source 3T-level model", which scored 76.8% on SWE-bench, ranking first among open-source models.

Its Agent Swarm can schedule up to 300 sub-agents in parallel. The paper from DeepSeek also cites the Agent Swarm of Kimi-K2.5.

AgentENV, the sandbox used by Kimi to train K3, is also a type of sandbox.

AgentENV is a distributed sandbox platform running on Firecracker microVM. Each sandbox has an independent Linux kernel, independent network stack, and independent file system. The underlying storage uses OverlayBD + ublk, the read-only layer is shared across the entire cluster, and each sandbox writes to its own upper layer. It uses the same technical stack as DSec.

Section 7 of the DSec paper states that the Rust version of the OverlayBD/ublk storage library it uses is open sourced in the AgentENV repository.

The two are like peers from the same school.

But AgentENV cannot implement RSI, it only provides "state operation primitives" such as fork/snapshot, which allows RL rollout to run in parallel, roll back, and perform clean scoring. Therefore, it cannot let Agents build their own environments like DSec can.

Alibaba is another player in this field. At the Yunqi Conference, Alibaba Cloud CTO Li Feifei introduced the "Agentic Cloud" strategy, taking Model, Harness, and Context as the three core scenarios, and launched AgentCore, Agent Sandbox, and the new generation of storage CPFS in one go.

Among them, Agent Sandbox can create 100,000 sandboxes per minute, with deep sleep wake-up time less than 600 milliseconds, and is compatible with E2B and K8s.

Sandboxes are becoming the "new runtime".

The main body of cloud computing has evolved from virtual machines, to containers, to large models, and now it is the turn of Agents. Whoever controls the execution environment of Agents will control the entry of the next generation of cloud.

This leads to the competition focus shifting from "model capability" to "environmental infrastructure".

"Training large models relies on computing power, training Agents relies on environments".

In addition to GPUs, CPUs, memory, storage, and image distribution have all become new bottlenecks.

Another point is that the value of security as an additional feature has become extremely important. The model jailbreak incidents of OpenAI and Anthropic, as well as the Agent cheating and kernel crash incidents in the DSec paper, all point to the same issue.

If the sandbox is not secure enough, the training signals will be false, and the evaluation will be invalid. Therefore, manufacturers like Alibaba take "security fence" as a selling point, and DSec uses AppArmor plus eBPF.

In the past, sandbox manufacturers competed for speed and low cost, but now they are competing for higher security strength.

The first battle of RSI has already begun. To run RSI, you must first have sandboxes and environments.

This article is from the WeChat official account "Alpha AI", written by Miao Zheng, and published with authorization from 36Kr.