Published in Science, Chinese Scientists Unveil Universal Biomedical AI Agent With Real-World Research Performance Approaching Human Experts

In the early hours of today, a general biomedical AI Agent — Biomni, jointly developed by the team of Chinese young scientist Huang Kexin and their collaborators, was published in the authoritative scientific journal Science.

According to the paper, Biomni does not rely on a fixed workflow template. It can independently break down tasks and call tools based on questions raised by researchers, assisting people in completing diverse biomedical research. It demonstrates strong generalization capabilities in biomedical tasks such as genetics, genomics, and pharmacology; in some real scientific research tasks, its performance is close to that of human experts, and it takes less time.

Real-world case studies further prove that Biomni can interpret multimodal datasets, optimize protein stability, coordinate wet lab instrument operations, and generate experimental protocols that can be verified by experiments.

Paper link: www.science.org/doi/10.1126/science.adz4351

The research team stated that this work opens up a promising direction for biomedical research: AI Agents are expected to collaborate with human scientists to assist them in completing complex research and experiments, thereby accelerating the translation of basic research into applications.

Biomni: A Scalable General-Purpose Biomedical Agent

In the past, Agents in the biomedical field were often only "specialists" in a specific domain. Although they can play a role in specific scenarios, they are difficult to cover multiple types of biomedical research tasks.

Different from traditional task-specific biomedical Agents, Biomni constructs a unified biomedical action space. Biomni-E1 is its execution environment; Biomni-A1 is the Agent that works in this environment. The specific process is as follows:

Biomni-E1: Responsible for integrating commonly used tools, databases, and software in scientific research, making them callable and combinable resources. The research team selected 100 recent papers from each of the 25 biomedical topics defined by bioRxiv, analyzing a total of 2500 literatures. The Action Discovery Agent reads the papers one by one, extracting the tasks, tools, databases, software, and experimental protocols required to reproduce or carry out related research. After manual verification, these resources are organized into an executable environment, enabling the Agent to call them through Python, R, and command lines.

Biomni-A1: Responsible for selecting resources, planning processes, and executing tasks based on specific research questions. Specifically, it includes the following links:

1. Resource Selection: Biomni-A1 dynamically filters the most relevant resources from biomedical tools, databases, and software according to user goals to complete the current task.

2. Code Execution: It uses code as a unified interface to connect tool calls, database queries, data processing, and analysis steps to form an executable workflow.

3. Adaptive Planning: Biomni-A1 first generates an initial plan based on biomedical knowledge, and continuously revises and refines it according to intermediate results during the execution process, making subsequent steps more suitable for the current task.

Performing Real Scientific Research Close to Human Experts

The research team found that Biomni outperforms multiple types of baseline systems on general biomedical benchmarks, and its performance in real scientific research tasks is close to that of human experts. It can participate in experimental design and automated execution, providing experimental protocols that can be tested in experiments; and can further improve specialized task capabilities through reinforcement learning.

1. General Biomedical Research Benchmark

In general biomedical research tasks, the performance of Biomni is significantly better than multiple baseline models. The Biomni-Eval1 results show that Biomni achieves the highest performance in average accuracy; even using the same Biomni-E1 environment, the performance of the ReAct bioinformatics Agent is still inferior to the system driven by Biomni-A1. This indicates that the capabilities of Biomni come not only from the integration of tools, databases, and software resources, but also from the Agent framework itself.

To test the generalization ability of Biomni on unseen biomedical problems, the researchers used the biomedical subset HLE-Bio of Humanity’s Last Exam for evaluation. The results show that when multiple cutting-edge LLMs are connected to the Biomni-A1 agent architecture and Biomni-E1 environment, the accuracy increases by about 6%-12%. This shows that Agent capabilities mainly come from the Agent architecture and environmental support, rather than a specific underlying model.

2. Comparison with Human Experts: Equal Accuracy, Significantly Reduced Analysis Time

In real scientific research tasks, the overall accuracy of Biomni is close to that of experts, and the analysis speed is significantly faster. Especially in rare disease diagnosis and GWAS tasks, the analysis that originally takes experts one or two hours to complete can be finished by Biomni in a few minutes.

3. Real Case Verification

The research team pointed out that Biomni can not only complete computational analysis, but also propose hypotheses, assist in experimental protocol design, integrate complete workflows, and convert part of the experimental process into runnable code.

Proposing Hypotheses: Biomni has the ability to automatically generate verifiable biological hypotheses from complex data. It can not only discover the laws related to sleep structure, sleep efficiency, and sleep quality in wearable device sleep data, but also reproduce known osteogenic regulatory relationships in multi-omics data of human embryonic skeletal development, and propose candidate transcriptional regulators to analyze the transcriptional regulatory mechanism of skeletal lineages.

Figure | Exploration of sleep data based on wearable devices: research design, analysis process, and main findings.

Assisting in Experimental Protocol Design: In the molecular cloning task, Biomni generated an end-to-end cloning protocol and plasmid map. The results of blind review were close to human experts and better than human trainees. Subsequently, the researchers carried out wet experiments according to the B2M sgRNA cloning protocol designed by Biomni. Colonies were obtained on the second day of the experiment, and the sequencing results of the two colonies showed that the sgRNA was inserted correctly, proving that the protocol is experimentally feasible.

Integrating Complete Workflow: In the protein thermostability optimization task, the research team only needs to provide the protein sequence and put forward the goal of "improving thermostability". Biomni will select and combine resources such as AlphaFold-2, ThermoMPNN, and literature retrieval to predict the protein structure, evaluate the sequence thermostability, and propose candidate mutations by combining structural information and existing literature.

Task Execution: In the experimental automation task, Biomni demonstrates the ability to convert natural language experimental requirements into robot executable code. Researchers only need to describe what experiment to do and provide information about the liquid handling platform used. Biomni can select PyLabRobot as the automation interface and generate executable code according to the hardware configuration of Hamilton STAR. Subsequently, the robot can perform experimental operations according to this set of automation schemes, thereby directly connecting the researcher's experimental intention to the actual automated execution process.

4. Reinforcement Learning Improves Specialized Task Capabilities

Reinforcement learning can help Agents improve their tool usage and task planning capabilities. Although Biomni has strong generalization capabilities, it still does not reach the expert level in some specialized tasks. To solve this problem, the research team trained the open-source model Biomni-R0, allowing the model to interact with tools, databases, and task processes in the Biomni-E1 environment, and optimize the task completion effect with reward signals labeled by experts.

The training results show that after reinforcement learning training, the performance of the open-source Biomni-R0 model on specialized tasks is significantly improved. The average task score of the 8B version increased from 0.32 to 0.59, exceeding the 0.56 of Claude 4 Sonnet; the 32B version further increased to 0.67. The results at the task level also show that the improvement brought by reinforcement learning covers multiple types of specialized biomedical tasks, including experimental design, gene and variant analysis, database query, and disease diagnosis.

Shortcomings and Future Directions

The research team pointed out that although Biomni demonstrates the potential of a general-purpose biomedical Agent, there are still some shortcomings.

First of all, the coverage of Biomni is still limited. The existing evaluation tasks only cover part of biomedical research, and many key areas have not been fully tested; the action discovery stage mainly relies on recent literature, which may miss some basic concepts and classic technologies that have faded out of current discussions but still have long-term value. In the future, research still needs to incorporate more biomedical subfields, real task scenarios, and a wider range of literature sources.

Secondly, in complex multi-step tasks, Biomni still relies on relatively clear structured prompts. Taking scRNA-scATAC multi-omics analysis as an example, writing out key analysis steps in advance helps to improve the stability and reproducibility of the results, but it is still difficult for it to automatically supplement the domain knowledge and analysis conventions in complex analysis. In the future, research still needs to further improve planning and reasoning capabilities.

The research team also pointed out that the performance of Biomni on different tasks is not balanced. It has approached the human level in tasks such as database query, sequence analysis, and molecular cloning, but it still has shortcomings in tasks that require careful clinical judgment, experimental reasoning, or deep biological synthesis. To this end, the research team proposed that in the future, its planning and execution capabilities can be improved through reinforcement learning, and multimodal information such as text, images, and structured data can be further integrated.

Finally, AI Agents in the life sciences also bring biosafety issues. Since such systems can synthesize literature, generate experimental protocols, and perform automated analysis, they may make biological knowledge more prone to misuse. The research team emphasized that in the future, it is necessary to adhere to openness and transparency, strict evaluation, and maintain communication with the biosafety and policy communities, so as to reduce potential risks while expanding the benefits of scientific research.

For more technical details, please refer to the original paper.

This article is from the WeChat public account "Academic Headline" (ID: SciTouTiao), written by Xia Qiansi, and published with authorization from 36Kr.