Just now, Anthropic has released the physical MCP: Claude begins to take over the real world.

Large language models have truly gained their "hands" to interact with the physical world now.

This Thursday, Anthropic announced the launch of MCP for Hardware: Model Hardware Standard (MHS), a new standard designed to help all large model-driven AI agents safely and quickly manipulate physical devices.

Now large language models like Claude can operate hardware such as microscopes, robotic arms, liquid handlers and lasers in the physical world just like how they use MCP (Model Context Protocol). This progress is regarded by the industry as a key step for AI to move from the digital world to the physical world.

People who use AI agents may have heard of MCP, which is an open standard protocol launched by Anthropic in November 2024, aiming to provide a standardized, secure two-way communication interface between large language models (LLMs) and external data sources, local files, development tools and various application services.

The core positioning of MCP is "the USB-C interface in the AI field", which can connect large language models with software environments such as GitHub, Slack, local file systems and databases. With the development of AI technology in recent years, MCP has evolved into an indispensable de facto standard in the entire agent ecosystem.

Now Anthropic hopes to extend this set of standard communication specifications to physical hardware, sensors, embedded systems and test devices through MCP for Hardware.

The development of MHS began with the cooperation between Anthropic and the Howard Hughes Medical Institute (HHMI) Janelia Research Campus, and the research preview version is now open to the first batch of scientific research laboratories and advanced manufacturers.

Normally, laboratories or manufacturing plants need weeks or even months to set up and integrate hardware. Most devices cannot communicate directly with each other, so experts are required to build customized integration solutions. MHS can shorten this integration work to hours or even minutes. In addition, by integrating AI into these tools, MHS also helps researchers and engineers more easily coordinate autonomous, around-the-clock experiments and workflows. Agents can reason about every step in the experiment, update parameters in real time, and in some cases recover from hardware failures without human intervention.

MHS works with any device with a programmable interface, and it is also independent of the type of base model. Any agent framework can use standard protocols such as MCP to access it.

Even without considering the additional difficulty of integrating AI into the system, it is not easy to make multiple devices in a laboratory or factory workshop communicate with each other. Each device usually has its own programming interface, and there is no standardized integration method at present. Moreover, after the devices are connected, there is no universal way for them to share data with agents, let alone for agents to operate these devices safely.

Now MHS addresses these challenges by introducing standardized drivers: a type of software that translates between the computer operating system and hardware devices. MHS drivers use a set of simple basic commands, such as reading (e.g. getting temperature) and writing (e.g. setting temperature), which any hardware device can understand and execute. It enables each device to be discovered in a standard format, so that devices and agents can discover each other and communicate across the network without using a customized "translation" program between them.

MHS drivers also help agents understand and use devices they have never seen before, and provide them with machine characteristic information that may not be obtained through code alone (such as the weight of the robotic arm, which is critical for safe operation). To date, most of this information is stored in paper manuals, users' computers or as tacit knowledge. But MHS drivers include tags that allow users to enter this information directly in natural language (users can enter it themselves, or chat with the agent so that the AI can ask about their hardware setup).

Using the information in these tags, the MHS driver automatically generates a reference file containing general characteristic information of the device, such as what it can measure, what it can adjust, and what safety restrictions will be enforced. This file provides agents with all the information they need to operate the device.

After the devices are connected and the agent program understands how to use each device, a method is needed to control these hardware. For MHS, there are three such mechanisms: MCP, command line interface and code files (API). They work together to enable users to coordinate multiple devices with one line of code.

Once the agent can control the devices, it can receive operational data from each device and supervise and guide the work at a high level. The agent can sort the operation steps of different instruments, monitor the results, and adjust parameters according to real-time changing conditions. When the agent needs to perform long-running tasks or operate devices at a speed beyond its online reasoning capability, it can string driver commands from one or more devices into a code file. In this way, the device can perform operations on its own without the agent reasoning at every step.

During the testing of MHS, Anthropic found that Claude interacts with experiments and hardware in as exploratory a way as scientists do.

Anthropic observed Claude adjusting the laser, observing the results through the camera, evaluating how the adjustment moves the laser beam, and then repeating this process to try to understand the sequence of events. After that, Claude packaged the learned information into a code file and wrote a deterministic script that allowed it to calibrate the laser without reasoning at every step, so that the whole process could be run as a single command.

In the process of developing MHS, Anthropic shared this technology with several laboratories and hardware manufacturers in the fields of biotechnology, robotics, quantum computing and other fields. In these early projects, we can see that MHS shortens the device integration time, improves the iteration speed in various experimental environments, and facilitates real-time operation and fault detection of machines.

Hardware vendors and their supporting software companies have also built MHS support into their devices so that agents can discover and operate the devices. For example:

Amazon Web Services (AWS) supports MHS through Strands Robots, a library for connecting AI agents to physical devices. During the MHS research preview period, AWS will provide participants with a private pre-release version of the Strands Robots software package.

Automata is adding MHS support to its laboratory automation platform LINQ to enable intelligent error handling for instruments in autonomous laboratories.

Danaher is actively exploring how MHS-enabled capabilities can allow its smart instruments and autonomous laboratories to scale up biomedical research and development.

Doosan Robotics is testing MHS with its robotic arms, including performing automated quality assurance and coordinating tasks between multiple robots.

MBF Bioscience is developing MHS drivers for ScanImage, the software that runs laser scanning microscopes in hundreds of neuroscience laboratories around the world, with the aim of integrating AI agents into real-time data analysis and experiments.

QIAGEN is trialing MHS through a proof of concept on its nucleic acid purification platform QIAsymphony Connect to demonstrate how AI agents can help laboratories troubleshoot instruments faster, guide operators to perform recovery, improve instrument uptime, and reduce the risk to biological samples at the same time.

Tecan is adding MHS support to its Fluent liquid handling platform so that AI agents can directly discover and operate these platforms.

Universal Robots has received early access to MHS and plans to add support for MHS on its robot platform.

Before open sourcing, Anthropic also hopes to make more improvements to the standard.

As a large language model, Claude learns about the physical world through text and images, which means that its spatial and physical reasoning capabilities have limitations and still require expert supervision. For example, when processing protein samples, researchers at Genentech had to guide Claude to recognize that the error caused by sample foaming was a physical fault rather than a software error, which could only be mitigated by corresponding physical corrections.

MHS is currently not compatible with hardware that lacks a programmable interface, so Anthropic is working with manufacturers of such devices to integrate MHS drivers into the hardware. Many developers are already using Claude Code to operate individual physical devices. In the next phase of MHS, Anthropic hopes to extend the standard to cover more devices used by developers.

Early adopters include Hugging Face (who are adding MHS support to their robotics library LeRobot) and Raspberry Pi (who are enabling MHS integration across multiple of their products after successful testing with their Camera MHS driver).

Reference content:

https://www.anthropic.com/news/model-hardware-standard-research-preview

This article is from the WeChat official account "Synced" (ID: almosthuman2014), written by Synced that focuses on Agents, authorized for release by 36Kr.