Apple has equipped Claude and Gemini with robotic arms, which can pick up spoons to perform tasks without any prior teaching, and can also correct their own errors autonomously.
There is a mustard bottle on the table. The robotic arm stretches out to reach it but fails to get it. According to conventional logic, the task would fail, or a human intervention would be requested.
But this time, it did not stop.
It glanced around, found a wooden spoon, reached out to knock the mustard bottle over, then adjusted its angle, grabbed the bottle cap, and put the mustard into the bowl.
The robotic manipulation paper AMP1 (Agentic Manipulation Policies 1) released by Apple on November 6 records the above scenario. No one told it in the prompt that it could use tools, nor did anyone write corresponding subroutines for it.
From taking photos to writing Python code, then calculating the end-effector trajectory, executing the action, and reviewing the result, the entire process is controlled by only one general large model.
Paper link 🚪: https://arxiv.org/abs/2611.09892
There are similar scenarios: when the target object is blocked, it will actively move away the obstacles and then scan the scene again; when a pomegranate is too wide to be grasped by the standard parallel gripper, it presses the pomegranate against a heavy object to fix it, then uses the gripper to scoop it up from below like a forklift; when the action on the real machine deviates, it compares the image with the position specified by the instruction, diagnoses that the error comes from camera calibration, corrects the parameters on its own and continues the task; after finishing a game of chess, it puts the chess pieces back into the corresponding drawers and closes them properly.
The idea of AMP1 is not the mainstream idea in the robotics field in the past two years.
The core logic of the Vision-Language-Action model (VLA) and the World Action Model (WAM) is data-driven: train a dedicated policy with a large number of robot teaching videos to directly map observations to actions. In contrast, AMP1 allows the general large model to view images in real time, write code, and send trajectories, starting from scratch every time without relying on any preset skills or special training data.
The paper collectively refers to this type of solution as "Agentic Manipulation Policies".
Among the 120 tasks in the RoboLab-120 simulation benchmark, the success rate of Gemini 4 Argon paired with the AMP1 framework is 85.6%, and that of Claude Opus 5.5 is 82.8%; the strongest VLA and WAM on the leaderboard at that time achieved only 49.2% and 42.9% success rate respectively.
In the real machine test (Franka robotic arm, 12 tasks, 108 trials in total), AMP1 succeeded 104 times after enabling the error recovery function, with a success rate of 96.3%; when the function was disabled, it succeeded 94 times, corresponding to a success rate of 87.0%. In the control group, the success rate of FLUX 3 Action was 45.4%, and that of π0.5 was 27.8%.
The Large Model Takes the "Driver's Seat"
The basic architecture of AMP1 can be summarized in four words: Observe, Reason, Execute, Review.
In each round, the model reads images from three cameras, obtains depth information and camera calibration parameters, then writes Python code to calculate the motion trajectory of the end effector, previews it first to confirm that the trajectory is reachable, then sends it to the robotic arm, and takes another photo after the action ends to decide the next step.
The paper counts the basic action patterns repeatedly developed by AMP1 in simulation tasks: in about 61% of the tasks, it first uses point cloud to align the grasping axis, in 66% of the tasks it first measures the size of the container, in 40% of the tasks it repairs the plan on its own after the action is rejected by the system, and in 39% of the tasks it diagnoses the cause first and then re-plans after the action fails.
Under the same framework, the performance gap between different models on the simulation benchmark is quite obvious. The success rate of Opus 5.5 in the simulation benchmark is 82.8%, that of Haiku 5.5 is 51.6%, and the older Opus 4.5 has only 35.2%.
From the perspective of perception, after removing the depth information, the success rate of Opus 5.5 barely changes, only dropping from 82.8% to 81.1%. The cost is that it needs to "think" harder, which is reflected in the fact that the average number of model calls per task increases from 34.8 to 46.3.
But weaker models cannot do this. After removing depth and calibration information, the success rate of Haiku 5.5 drops directly from 51.6% to 10.7%. Even if more reasoning computing power is allocated to it, it will not help.
The paper also tested the scheme of replacing images with text descriptions, and the result is contrary to the suggestions of some previous works: when Opus 5.5 only receives text descriptions, its success rate is roughly halved. The geometric alignment details between the gripper and the object contained in vision are difficult to be accurately conveyed through language.
After adding cross-episode persistent memory to AMP1, its performance on the bimanual manipulation benchmark RoboDojo changes more significantly. The zero-shot success rate rises from 26.4% to 84.1%, exceeding the top score of 68.9% on the leaderboard at that time.
In the continuous learning experiment, 42 tasks are executed once each in random order, and the overall success rate rises from 26.4% to 57.6%. The experience accumulated from earlier tasks also has a transfer effect on subsequent unseen tasks.
AMP1 also reminds me of VIA (Visual Interface Agent) and CaP-X (Code-as-Policy).
VIA tries to bypass the traditional route where robots must rely on a large amount of motion data for training, allowing AI Agents like Claude Code and Codex to directly manipulate robots in the real world. CaP-X explores using Gemini-3-Pro and Claude Opus 4.5 to complete long-horizon manipulation tasks, which also does not require post-training.
Source: VIA: Visual Interface Agent for Robot Control https://via.hengyuanhu.com/?utm_source=chatgpt.com
Some reviews from domestic practitioners point out that the problems of high collection cost and poor generalization of robot teaching data have always existed.
AMP1, in a sense, is a workaround for this dilemma.
Is Slowness a Big Problem?
In the real machine test, AMP1 takes an average of 280 seconds to complete a successful task, of which the reasoning time is about 211 seconds. The FLUX 3 Action in the control group takes an average of 44 seconds, and π0.5 takes 62 seconds.
Most of the time, the robotic arm is waiting for the model to "think", and currently AMP1 cannot re-make decisions in the middle of trajectory execution. We can also see that the test scenarios in the paper are basically quasi-static: objects do not move, and millisecond-level response is not required.
Therefore, AMP1 cannot be used for time-sensitive tasks at present.
The paper mentions several acceleration paths: save the code written for each task for direct reuse next time; run a fast feedback controller locally to reduce calls to cloud models; plan the next segment while executing the current trajectory; use the motion data generated by AMP to distill a smaller local model. The results of the memory experiment confirm the feasibility of this direction.
In the past few years, the reasoning cost for a given level of capability has decreased by about an order of magnitude every year. If this trend continues, today's speed disadvantage will be gradually eliminated over time.
For the industry, the significance of AMP1 may lie in that it changes the position of the competition layer. Robot manufacturers do not necessarily need to develop their own general cognitive models, and can directly purchase or deploy model capabilities from other companies.
On November 4, Apple's research team submitted the humanoid robot paper HALO at the same time. Deployed on the Unitree G1 robot, it completed 360 single-task tests in 6 real rooms that were not pre-scanned, with an average success rate of over 90%. The planning model runs on the Mac mini M4 Pro carried on the back of the robot, and calls the Qwen series models, which is also not Apple's self-developed model.
Source: Perceptive Humanoid Motion from Simulation https://arxiv.org/html/2611.06908
Two days later, AMP1 was released. Judging from these two papers, Apple is pushing robots from hardware design to software capability, and it does not plan to rely on its own models. Mark Gurman previously reported that Apple is developing a desktop device with a robotic arm, and the prototype video of the desk lamp robot has long been made public.
Desk Lamp Robot
For hardware manufacturers, the threshold for robot software R&D may decrease, application scenarios may increase, and the potential customer base of hardware manufacturers will also expand.
But conversely, if everyone uses similar general models, the differentiation of robots themselves may return to mechanical design, actuators, sensors, reliability and cost.
That may not be good news for companies that only know how to assemble standardized robotic arms but have no deployment channels and engineering advantages.
Future robotics hardware enterprises can claim that they have the strongest AI model, but they should also try to make any mainstream model able to control their own robots safely, stably and at low cost.
This article is from the WeChat official account "APPSO", written by the product discoverer of tomorrow, and published with authorization from 36Kr.