Major Discovery: GPT-6 has achieved a breakthrough in embodied intelligence, how can China fight the embodied defense war?

Recently, an anonymous simulation evaluation report published on GitHub has sent a huge shockwave across the embodied intelligence community. The architecture integrated with GPT-6 Astra scored 62.6 points, while the second-place entry only got 38.26 points.

A 64% performance gap. This is not fine-tuning optimization, but a dimensionality reduction strike at the architectural level.

The study did not use GPT-6 Astra to directly control the robotic arm. It adopted a smarter approach: leveraging GPT-6 Astra to provide judgment and correction at the semantic level, forming a complementary architecture with π₀.₅'s physical space interaction and action prior.

How was this evaluation conducted?

The writers of this report are researchers from the Galaxy General team, and the first author is a PhD student supervised by Professor Wang He, founder and CTO of Galaxy General. This is not a casual demonstration, but a set of strictly aligned closed-loop control comparative experiments.

The study analyzed two closed-loop control architectures for simulated dual-arm operation:

Direct mode: Without running π0.5, GPT-6 Astra directly reads three-channel camera frames and robot status, outputs the end pose of the dual arms and the opening/closing state of the gripper, and executes 1 to 5 control steps each time.

Hybrid mode: At each decision moment, π0.5 first generates 50 candidate actions. GPT-6 Astra reads the same frame, execution history, and the corresponding dual-arm trajectories of the candidate actions, then makes a binary choice: either follow the first 1 to 15 steps from π0.5, or generate 1 to 5 end pose corrections on its own. After completing this segment, it reads the new frame and makes the next decision.

This architecture is actually very simple. The first author also mentioned in the report that this TopK idea of "candidate action selection" has been discussed many times in industry exchanges. But this is the first time it has delivered such a huge performance gap.

The results are very clear. The hybrid mode achieved a 48% success rate and an average score of 62.60, with the average score even 64% higher than the second-place entry by a large margin.

RoboDojo: Greatly Improved Success Rate After Integrating GPT-6 Astra

The first part is the RoboDojo experiment.

The team selected 10 types of complex robot operation tasks, including desktop sorting, object classification based on language, number sorting, packing, tower building, mahjong tile drawing, clothes folding, and putting bottles into buckets. Each task was tested 5 times, for a total of 50 tests.

Two schemes were set in the experiment.

One is GPT-6 Astra Direct, where Astra directly generates robot actions based on vision, proprioceptive state and task objectives; the other is π0.5+Astra, where the robot VLA model π0.5 provides actions, and Astra observes the environment and candidate actions to make corrections when necessary.

The difference in results is obvious.

Astra Direct succeeded in 13 out of 50 tasks, with a success rate of 26%, and the average score of 48 tasks with complete native scores was 37.81.

After accessing π0.5, the number of successful tasks increased to 24, the success rate reached 48%, and the average score rose to 62.60. It is more noteworthy that Astra only modified 14.4% of all execution control steps.

In other words, most of the actions are still completed by π0.5, and Astra is mainly responsible for understanding tasks, judging the current state, and deciding whether correction is needed at key nodes.

This set of experiments first shows that the two types of models have obvious complementarity.

Astra is best at semantic understanding, visual recognition, spatial relationship judgment and task planning. In tasks such as "object classification", Astra Direct even scored 100 points; in the face of grasping failures, missing objects or environmental changes, it can re-observe the scene and then decide the next action.

However, for tasks that require precise contact, stable grasping and continuous control such as tower building and mahjong tile drawing, the shortcomings of large models are quickly exposed.

Astra Direct averaged only 12 points in the tower building task, and 0 points in the mahjong tile drawing task. The model can know where the wood block should be placed, but it may not be able to stably control the landing point, friction and collision.

After adding π0.5, the score of tower building increased from 12 to 64, the score of mahjong tile drawing increased from 0 to 40, and tasks such as clothes folding and putting bottles into buckets were also significantly improved.

This is the most important message of the report: large models have been able to enter the robot policy layer, but the understanding of the physical world and the operation of the physical world are still two different capabilities.

RoboLab: Further Verification of Capabilities on Tasks with Semantic Understanding

The RoboLab experiment further demonstrates Astra's advantages in another type of tasks, examining the understanding of general robot policies on vision, object relationships, task instructions and spatial structures. The team selected 10 types of tasks, each of which was evaluated 5 times, for a total of 50 tasks, including putting blocks into containers, finding and moving pumpkins among sundries, stacking blocks in a specified order, adjusting the orientation of cups, and placing different objects in specified positions.

The results are very outstanding.

GPT-6 Astra Direct succeeded in 49 out of 50 experiments, with a success rate of 98%; Astra+π0.5 completed 46 times, with a success rate of 92%. For reference, π0.5 completed 18 times in the same set of statistics, Cosmos3-Nano-Policy completed 18 times, and DreamZero completed 17 times.

It is particularly worth noting that Astra did not receive additional training for these specific tasks. Based on the current frame, robot status and natural language task description, the model continuously judges how to operate in the next step.

From the results of each task, Astra achieved 5 full successes in multiple tasks, including putting blocks into boxes, handling pumpkins among sundries, placing objects according to relationships, stacking in a specified order, and re-adjusting the orientation of cups. The only task that did not achieve full success was "putting the large raisin box into the container", which was completed 4 out of 5 times.

The core capability demonstrated by this set of experiments is zero-shot generalization. The model can identify targets, understand spatial relationships, convert language requirements into actions, and continue to adjust policies after environmental changes.

However, this number needs to be understood in the context of the experimental caliber. The report clearly states that the RoboLab experiment only selected 10 tasks, with 5 final evaluation slots for each method per task, which is not a complete reproduction of the RoboLab leaderboard; the historical baselines and Astra do not strictly guarantee exactly the same initial state and control configuration, and some Astra experiments include authorized retries.

Therefore, 98% is more suitable for observing the upper limit of Astra's capabilities on these tasks, and cannot be simply used as a leaderboard score under strictly identical conditions.

Astra's Strongest Point Is Understanding Tasks and Finding "What Went Wrong"

Returning to the per-task results of RoboDojo, we can further see exactly where its strengths lie.

In the "object classification" task, Astra Direct scored 100 points and succeeded 5 times in all; the average score of "classification by language" reached 60, with a success rate of 40%; the score of "arranging the largest number" was 57.

Score heatmap, success rate heatmap

These tasks are highly dependent on language understanding, visual recognition, relational reasoning and target planning, which are exactly the capabilities that large models are good at.

A large number of behaviors rarely demonstrated by traditional robot policies also appeared in the experiment.

For example, after the model finds that a grasping posture is not ideal, it will actively change the approaching method; after the objects are roughly placed, it will re-observe the environment, find that the task is not actually completed, and continue to adjust; during the packing process, it can find missing objects behind the box; when the execution of the original plan is blocked, it will re-plan according to the new visual feedback.

Figure: During the experiment, Astra will continue to judge the task status according to the new frame after execution, and adjust actions when necessary. For example, in the object classification task, the model finds that the current grasping requires local correction; in long-term tasks such as packing, the model will continuously check the remaining targets in the scene before deciding subsequent actions.

Excellent recovery performance also appeared in the number placement task: after the movement was blocked, the model re-understood the control feedback of the robotic arm, adjusted the action and continued to execute. It demonstrates the local adaptation ability of GPT-6 Astra to control constraints, rather than just passively repeating failed commands.

Figure: Number placement: excellent recovery after understanding control feedback

These phenomena indicate that Astra does not mechanically execute a pre-generated trajectory, but forms a closed loop of "observe — decide — execute — re-observe".

The model will even actively choose some policies that may not be common in training data. For example, in the "putting bottles into buckets" task, Astra Direct once tried to sweep the bottles towards the trash can instead of grasping them one by one. This solution eventually failed, but it shows that the model can find new solutions on its own according to the target.

The Real Bottleneck Still Lies in Physical Interaction

However, the research report also reveals Astra's weaknesses.

In the "tower building" task, Astra Direct averaged only 12 points, with a success rate of 0; the score of "mahjong tile drawing" was 0; these tasks require precise contact, stable grasping, collision control and continuous judgment of the force state of objects. Compared with semantic and visual tasks, large models are obviously struggling in this field.

Figure: Astra Direct failed in the tower building task

A typical problem is that understanding the target does not mean being able to complete the action stably.

The model may know where the wood block should be placed, but cannot guarantee that the tower will not fall when it is put down; it knows that the object should enter the container, but it may fail due to collision between the gripper and the edge of the container. The current control adopts segmented execution, and the model usually needs to wait for an action segment to end before re-observing, so problems such as slipping and collision that occur during action execution are sometimes not discovered until the next round of feedback.

This is also where π0.5 generates value.

After adding π0.5, the score of "tower building" increased from 12 to 64, with a success rate of 60%; the score of "mahjong tile drawing" increased from 0 to 40; the score of "clothes folding" increased from 40 to 100; the score of "putting bottles into buckets" also increased from 36 to 100.

π0.5 provides motion prior formed by robot action data training, while Astra is responsible for identifying tasks, judging progress and correcting errors at a higher level. After the combination of the two, some tasks that are difficult to complete by relying only on Astra are significantly improved.

The study also analyzed some interesting but not necessarily robust or physically unrealizable situations that appeared in the GPT-6 Astra mode. Due to the lack of mature action and object interaction prior provided by embodied Policy, the model needs to solve the details in grasping, supporting, contacting, releasing and dual-arm coordination by itself, and some seemingly reasonable and interesting solutions are not stable in actual execution.

Zero-shot Behavior: Interesting Solutions, Not Necessarily Robust Actions

GPT-6 Astra (direct) will try to sweep the bottles into the bucket with the robotic arm, grasp the tower building board with one hand, or complete operations that are originally suitable for dual-arm cooperation with a single arm. These are different implementations of the same goal, not just task understanding errors.

The problem is: proposing a potentially effective operation and stably executing that operation are two different levels of capabilities. Without the support of mature action prior, the solution may ignore reliable grasping posture, sufficient support or appropriate release timing, making locally feasible actions difficult to translate into stable task completion. The value of embodied Policy is not only to provide an action answer, but also to provide an execution mode formed through interactive experience, so that the semantic-level solution is built on a more reliable operation foundation.

Figure: Zero-shot solution: trying to sweep the bottles into the bucket with the robotic arm

The Cost of Capability Improvement Remains Very High

The report also emphasizes that the current solution is still far from real deployment.

The first is the reasoning cost. In the RoboDojo experiment, π0.5+Astra consumed a total of about 625 million Tokens, and Astra Direct consumed more than 1