Authored by Liang Wenfeng, DeepSeek has released a new paper targeting large-scale Agent training.
Reported by Zhidx on September 23, the latest paper signed by Liang Wenfeng, founder of DeepSeek, has just been made public. It systematically released for the first time the technical details of DSec (DeepSeek Elastic Compute), the Agent training sandbox platform of DeepSeek. The paper was submitted on September 19, with a list of more than 130 authors including Liang Wenfeng.
Paper address: https://arxiv.org/pdf/2609.22978
The DSec platform first appeared in the DeepSeek V4 technical report. Its core function is to provide sandboxes for Agent training, so as to realize the stable operation of large-scale Agent training. The paper clearly states that for all RL training and evaluation from DeepSeek V3.2 to V4.1, all sandbox workloads run on DSec. The public disclosure of technical details now is equivalent to fully opening up its core technical formula.
The paper shows that the DSec platform has a very large scale, and one of its production units consists of about 160 CPU nodes, 30,000 cores, and 250TB of memory, hosting PB-level images.
In terms of capability, the DSec platform serves about 3 million sandboxes per day, with a peak concurrency of over 380,000, a creation speed of over 5000 per second, and a single training task can pull up to 32,000 sandboxes at one time.
▲ Distribution of the number of sandboxes created by each task
Here comes the question: why are such a huge number of sandboxes needed in Agent training? Facing the complex execution environment, how does DSec solve the problems of scale, scheduling and resource management?
01.
Agent RL naturally requires a huge number of sandboxes
Training environment becomes the development bottleneck
Reinforcement learning for traditional LLM training can mostly be carried out around static inputs, outputs and reward signals, but Agent training is completely different from LLM training. It needs to actually enter the real environment to perform tasks such as checking code, calling tools, executing commands, and modifying files. Every step the model executes may trigger a change in the environmental state, and the next step of execution is built on the previous results.
This means that during the Agent training process, in addition to models and data, researchers also need to maintain a large number of "working scenes". These environments need to be close enough to real machines to install dependencies, run software, etc., and can also restore a clean state after one task is completed, for the next round of rollout to continue to use.
The problem is that these sandboxes are large in number and not lightweight.
The paper shows that during the training process, a single task once pulled up 32,000 sandboxes at the same time, and these sandboxes are not fully loaded. When the Agent executes tasks, the sandboxes are often in a state of waiting for the next operation, and the CPU utilization rate is not high. However, the idle CPU does not mean that the resources have been released, and the memory and writable state still need to be retained continuously.
▲ CPU usage is intermittent, while memory and state are continuously occupied
Therefore, the traditional idea of "starting a container and running a task" can hardly continue to support the demand. The purpose of the birth of the DSec platform is to solve the problems of batch creation of sandboxes, resource scheduling, environment replication, state saving, suspension and recovery, and security isolation at the same time.
02.
Four major environmental requirements: a set of SDK for unified management
Functions, containers and virtual machines
As the tasks that Agent needs to perform become more and more complex, the working environment behind can no longer be "handled" with the same specification.
The lightest task may only require one function call, executing code and returning results; software engineering tasks require a complete Linux user space to install dependencies, modify code and run tests; security attack and defense and Computer-use have higher requirements for isolation; if some commercial software needs to be operated, the required environment even needs to be close to a complete computer.
DSec provides four backends: FnCall, container, Firecracker microVM and full virtual machine, which respectively cover different requirements from short-time function calls, software engineering, to security-sensitive tasks and full OS environment. The training framework does not need to care whether the underlying layer is a container or a virtual machine. Through the Python SDK (libdsec), the training framework can directly complete sandbox creation, command execution and result acquisition without adapting to different environment types.
▲ DSec provides four backends
The four backends of DSec are scheduled uniformly by the same platform. After the training framework initiates a request, the platform first completes identity and permission verification, then selects an appropriate node according to the cluster load, and the Edge on the node is responsible for creating the sandbox. After the sandbox is started, Aether and Chronus are responsible for connecting the platform with the execution process inside the sandbox, and the image data is provided by 3FS on demand.
▲ DSec architecture
However, when these environments expand from several types to tens of thousands of instances, new problems arise: how to quickly replicate a huge number of environments of different types.
03.
The more environments there are, the harder replication becomes:
How DeepSeek quickly deploys tens of thousands of sandboxes
As mentioned earlier, the Agent training environment is not only large in number, but also complex in combination.
The paper counts the data of one production week: the container backend involves 11266 basic images, 102171 workspaces and 103 toolkits. In actual operation, 67.8% of the sandboxes will also overlay workspaces or toolkits on the basic image. DeepSeek Harness is one type of component that needs to be updated frequently.
If all these components are placed in one complete image, any change in any layer may require rebuilding and distributing the entire image. When the number of environments increases, the cost of image maintenance and deployment will also rise accordingly.
DSec splits the basic image, workspace and toolkit into three independent versioned read-only EROFS layers, which are combined through overlayfs when the sandbox is started. In this way, if any component changes, only the corresponding layer needs to be updated, and the entire image does not need to be reprocessed.
Image distribution adopts on-demand loading. The paper finds that the data actually read during the running of the sandbox only accounts for 4.2% to 13.3% of the complete image. Therefore, DSec places the image data on 3FS, reads it on demand during operation, pre-fetches metadata to the local at the same time, and retains writes on the local disk of the node. Considering that 3FS is more suitable for large-block, continuous reading, this method can also avoid the efficiency problem caused by small-block random I/O.
▲ DSec splits the environment into composable layers
The actual effect shows that when 8192 containers are deployed in bursts at the same time, the on-demand loading takes 35 minutes, while Docker cold pull takes more than 60 minutes, and the cumulative disk write volume of a single node also drops from about 1600GB to about 700GB.
The paper shows that the construction of the environment in the DSec platform can also be completed by the Agent. Through pack_diff, the Agent generates an incremental snapshot after configuring the environment, which can then be restored to a new sandbox.
04.
Moving rollout out of GPU:
Separate training from execution
In the early scheme, Agent inference and rollout shared the GPU Pod with model training. Once the GPU task is preempted, the ongoing rollout will also be forced to interrupt.
Starting from V4.1, DeepSeek splits rollout from the GPU training environment and hands it over to the DSec platform for independent operation. The Agent sandbox is responsible for running execution environments such as DeepSeek Harness, and the worker container is responsible for specific tasks, neither of which depends on GPU resources anymore. In this way, when GPU training is preempted, the rollout state can be retained independently.
▲ DSec's CPU scheduling, memory reclamation, layered image and on-demand loading mechanism.
If the cluster capacity is insufficient, DSec also supports scaling up to the cloud. For example, after the cluster utilization rate exceeds 80%, eligible sandboxes can be migrated to cloud virtual machines; in order to reduce the overhead of re-pulling images in the cloud, DeepSeek pre-prepares a deduplicated image set of about 30TB, of which about 70% of the files will be actually accessed by container tasks. In the production environment, 200 cloud VMs can carry about 30% of the peak load.
05.
The more realistic the environment is
The greater the risk brought by Agent operations
DSec solves the problems of scale and efficiency, but there are other risks in the real environment. For example, the Agent may not complete the task according to the expected path.
The paper records a variety of abnormal behaviors: some Agents will sift through logs, forge RPC requests, even modify /bin/bash, trying to bypass the normal task process. Some Agents will scan reachable services, pull external code, and find answers through paths outside the evaluation.
What's more troublesome is that sometimes the Agent will break the environment itself. The paper records cases where recursive scanning of system files leads to kernel crashes, and even a simple yes command can cause the log to expand rapidly to tens of GB.
To address these risks, DSec mainly limits the operation scope of the Agent through AppArmor and eBPF. The former controls file and socket access, while the latter restricts network access, and can dynamically adjust rules according to the task phase.
However, these measures can only cover part of the risks at present, and kernel layer vulnerabilities are still difficult to completely prevent.
06.
Conclusion: DSec platform capability
Become a key part of large-scale Agent training
DSec presents an infrastructure solution for large-scale Agent training. From sandbox creation, environment reuse, to rollout scheduling, state preservation and security isolation, Agent training is forming a set of independent infrastructure requirements.
As Agent tasks continue to get longer and the interaction process increases, the scale of the execution environment will be further expanded. How to make tens of thousands or even more sandboxes run stably while controlling resource costs and security risks will become the problem to be solved for the further expansion of Agent training.
For DeepSeek's Agent training, while the model capability continues to improve, the execution platform that carries these tasks also needs to keep up. The engineering solution provided by DSec is perhaps a section of the evolution of Agent infrastructure at this stage.