Is soft-body manipulation most in need of data and most afraid of simulation distortion? A new study enables the realistic reproduction of fabric physics.

In recent years, researchers have been trying to mass - produce embodied training data through simulation environments.

However, there are still differences between the simulation environment and the real environment. In some complex scenarios, the simulation environment still cannot produce high - quality data.

Shanghai AI Lab's latest research has achieved the real - world reproduction of the physical properties of tricky items such as fabrics. The quality of the simulation data under relevant complex conditions has reached a usable level.

It makes simulation no longer just a tool for "approximating reality", but a new entry point for real - world data generation and strategy learning.

The progress of embodied intelligence is rapid, but the bottlenecks are becoming clearer: it's not that the models are not large enough, but that there is not enough data, especially high - quality, executable, and generalizable data.

Manipulation of soft objects is the category that lacks the most data. It not only has a huge state space, including deformation, contact, and topological changes, but also naturally depends on complex physical processes. Manual tele - operation is inefficient and costly, keeping the real - world data in a state of "just enough but not sufficient".

In this context, what SIM1 wants to answer is not "Can we make the simulation data even larger?", but another more crucial question: Only when the data is correct can the benefits of data augmentation truly emerge.

Simulation has always been considered a promising solution to the scarcity of robot data, aiming to fill the data gap in the real world with "massive simulation data". However, problems soon emerged: Although there seems to be a large amount of simulation data, it is not truly aligned with real - world scenarios. It can be used for pre - training, but is difficult to deploy directly. Once in real - world robot scenarios, post - training and real - world data patching are still required.

This also makes people start to wonder: Is this the limit of simulation data?

What SIM1 wants to convey is that perhaps everyone has been pursuing "more" but overlooked a more important thing - the data must be correct first, and then the benefits of scaling will become apparent.

In this sense, SIM1 proposes a real real - to - sim - to - real new paradigm: Starting from a small number of real - world demonstrations, it generates simulation data that can be directly executed in the real physical world, and finally transforms this data into deployable, scalable, and zero - shot transferable strategic capabilities. This means that for the first time in the field of robotics, there is an opportunity to truly discuss its own scaling law - intelligence no longer needs to grow in sync with real - world data collection.

First, look at the results: What can SIM1 deliver?

In the presented results, SIM1 can expand a small number of demonstrations into 100× scale trajectory data; pure simulation training can achieve a 90% zero - shot success rate; compared with the real - world data baseline, the generalization ability is improved by 50%; even when training from scratch, it can still achieve a 76% success rate. At the same time, it also brings significant efficiency advantages, including 27× lower cost and 6.8× faster training.

These results indicate that SIM1 is not just "creating a larger simulation set", but redefining the data production method: Data is no longer collected manually one by one, but automatically expanded from a small number of seeds by aligning with reality.

Why has sim - to - real always failed?

A key point that has been long overlooked is that the sim - to - real gap is never a single - point problem, but a superposition of multiple mismatches. SIM1 summarizes it into three co - existing gaps - geometry, physics, and motion. Geometry determines whether the spatial structure is consistent, physics determines whether the interaction response is reliable, and motion determines whether the trajectory conforms to the timing and rhythm of real - world operations. All three are indispensable. As long as any one of the links is not connected, the trained strategy will be difficult to execute effectively in the real world and will be limited to ideal performance in the simulation environment.

The starting point of SIM1 is to bridge these three mismatches simultaneously. SIM1 reconstructs the entire data generation link into a closed - loop: Starting from the real - world scenario, it replicates a high - fidelity simulation environment, expands large - scale operation data under the premise of physical consistency, and finally feeds it back to the real world for verification and deployment. Simulation is no longer an approximation of reality, but an extensible expression of reality.

The closed - loop of SIM1: Scan it, Simulate it, Scale it

SIM1 is not a single - point technique, but a complete data engine.

Its first step is Scan it: Through sub - millimeter - level scanning, real soft objects and scenarios are reconstructed into high - precision digital twins. Different from traditional coarse - grained modeling, the goal of this step is not to "draw a similar world", but to retain as much as possible the geometric structure, wrinkle texture, and precise spatial relationships in the real environment, so that the starting point of the simulation is directly based on reality.

The second step is Simulate it: Build a physical system aligned with real - world interactions, so that the way the robot acts in the simulation, the way the fabric responds, and the overall deformation dynamics are as close as possible to the real world. For SIM1, this is not simply calling a physical engine, but calibrating the simulation system into a reliable data generator.

The third step is Scale it: Instead of relying on scripted trajectory orchestration, it introduces generative methods to expand operation data. Basic operations such as grasping, lifting, folding, and releasing are regarded as combinable action vocabularies, and then the model learns how to splice, recombine, and extend them to finally generate smooth, diverse, and previously un - explicitly demonstrated trajectories. With random changes in material, lighting, and perspective, SIM1 can expand a small number of demonstrations into tens of thousands of executable data.

Core breakthrough: Let the simulation truly "understand" fabrics

If the closed - loop of SIM1 solves the problem of "where the data comes from", then the Deformation - Stable Solver solves the problem of "why the simulation always distorts fabrics".

The reason why fabric manipulation is difficult is that the deformation response in the real world is not local, slow, or isolated, but global, fast, and strongly coupled; a local stretch often affects the entire surface in a very short time.

Traditional simulations in such scenarios are prone to problems such as delayed propagation, particle drift, local jitter, and even over - stretch artifacts. The key design proposed by SIM1 is to upgrade the fabric from a "local particle system" to a "global response system": When the local stretch exceeds the threshold, the correction force will be propagated to the entire grid in a single step, thus maintaining the consistency and stability of the deformation.

The significance of this step is not only to make the picture more stable, but more importantly, to make the simulation truly respect the physics of the fabric for the first time. For operation tasks that rely on deformation and contact, this global consistency directly determines whether the simulation data can become a real training signal.

From manual collection to a data factory

Today's robot data collection essentially remains in the "handicraft stage": one operator, one demonstration, one trajectory. This model is not only costly but also difficult to cover a sufficient variety of changes. Once the task complexity increases, the data quickly becomes scarce, expensive, and non - extensible.

SIM1's idea is to reconstruct this link into an automated data factory. Starting with about 200 tele - operation demonstrations, the system first extracts a set of basic operation segments as "action templates", and then combines, rearranges, and expands these basic units through a generative model to generate new operation trajectories. At the same time, systematic changes are introduced in terms of material, lighting, and perspective to further expand the data distribution, and finally expand dozens of demonstrations into tens of thousands of trajectory data with execution significance.

More importantly, this is not simply data amplification, but a paradigm shift: from "manual collection" to "combinatorial generation", from "limited coverage" to "controllable expansion". The value of SIM1 lies here - it is no longer a tool for one - time sample production, but a data engine that can continuously generate, learn, reuse, and expand training signals.

Can simulation data replace real - world data?

SIM1's answer is: Not only can it, but it is even stronger in some scenarios. With the same data scale, the strategy trained by SIM1 can achieve a high zero - shot success rate and demonstrate stable execution ability in real - world deployment. More notably, when the test enters out - of - distribution scenarios - such as spatial changes, material changes, or lighting changes - SIM1's advantages become even more obvious.

The reason behind this is not complicated: Real - world data is naturally scarce and can only cover a limited number of sampling points; while simulation data, as long as it is sufficiently aligned with reality, can cover a wider and more systematic range around the task distribution. The value of SIM1 is not to replace the "small amount of fine - grained" real - world data, but to complement the "wide coverage" that real - world data is difficult to achieve.

More extreme verification comes from training from scratch. When only using real - world data, the strategy can hardly start; while when only using SIM1 data, the system can still learn an effective strategy and achieve an impressive success rate. This result shows that the key to performance improvement lies not only in the model itself, but in the data distribution itself.

Real - world verification and project significance

Ultimately, SIM1's goal is not to stay within the simulation, but to return to real robots for verification.

Multi - task and multi - scenario real - machine experiments show that the strategy trained based on SIM1 can be stably executed and maintain strong generalization ability under different conditions. This indicates that SIM1 is not constructing a "prettier simulation world", but establishing a real training path that can lead to reality.

From a more macroscopic perspective, SIM1 represents a new data paradigm: Simulation is no longer just a proxy for reality, but a part of real - world data; robots do not have to wait until enough real - world data has been accumulated to start gaining large - scale capabilities. Humans still provide the starting point, but from then on, expansion, generation, and learning can all be automatically completed by the system.

SIM1 is not just a method, it is more like a declaration: When simulation truly becomes reality itself, the ceiling of robot data is redefined.

The project quickly gained attention after its launch: It exceeded 20K views on the X platform within 17 hours, received 245 likes, and sparked discussions and interactions among researchers from multiple directions. This includes Anka, the author of VBD, Eric Heiden, the author of the Newton project, Masoud Moghani, the author of SoftMimicGen, and the attention of institutions such as NVIDIA GEAR, DeepMind, Stanford, CMU, and Princeton.

Project homepage: https://internrobotics.github.io/sim1.github.io/

Paper address: https://huggingface.co/papers/2604.08544

This article is from the WeChat official account "Quantum Bit", author: Fei Yang. Republished by 36Kr with authorization.