Andrew Ng released a paper auto-reviewer, achieving near-human performance at ICLR.
Can AI be used by reviewers of academic conferences and journals for paper review?
Currently, even in the field of AI, there is no unified standard yet. Among the world's major top - tier conferences, ICLR's rule is that the use of large models must be disclosed, while CVPR stipulates that large models cannot be used to write review comments at any stage.
However, faced with the ever - expanding number of paper submissions, in many cases, humans have no say in this matter. At ICLR 2026, which has introduced the "strictest control rules", someone has counted that as many as one - fifth of the review comments were generated by large models with a single click.
Nevertheless, the review cycles of major conferences are still extremely long.
Andrew Ng, a professor at Stanford University and a well - known scholar in artificial intelligence, has become exhausted by the increasingly long review feedback cycles.
The paper of one of his students has had a "tortuous fate" — it was rejected six times in three years, and each time the student had to wait about six months for the review results.
Such a slow feedback loop not only affects the process of publishing research results but also goes against the requirements for research efficiency in the current rapidly iterating technology development cycle.
Since we cannot change the paper review cycle, can we use the powerful capabilities of AI to build an efficient "paper feedback workflow" so that researchers can obtain high - quality review comments before formal submission, iterate on the paper content faster and with a clearer direction, and thus reduce the cost and time of repeated rejections at major conferences and journals?
For this purpose, Professor Andrew Ng has released a brand - new "Agentic Reviewer" for research papers.
This project was initially just a small tool he wrote for fun on weekends, and later, with the help of doctoral student Yixing Jiang, it was significantly enhanced.
We trained the system on the review data of ICLR 2025 and measured the Spearman correlation coefficient (the higher, the better) on the test set:
- Correlation between two human reviewers: 0.41
- Correlation between AI and human reviewers: 0.42
This indicates that the agent reviewer is gradually approaching the human level.
At this stage, this agent generates well - founded feedback by retrieving data from arXiv, so it performs best in research fields such as artificial intelligence, which are mainly published on arXiv.
Link to the agent review system: https://paperreview.ai/
However, this agent is still in the experimental stage. Most netizens have a positive attitude towards such tools and hope they can become truly practical research tools.
Some netizens hope that the agent reviewer can conduct reviews for specific conference or journal scenarios and even give estimated scores:
AI agents can accelerate scientific research and the talent cultivation cycle, and they are an engine for promoting academic progress:
However, some netizens also expressed concerns. Will researchers' pre - review by AI before publishing their results lead to a decline in academic diversity?
With researchers having handy AI review reference tools and reviewers using AI to generate review comments, is it time for a transformation in the academic achievement review system?
In the future, in what way will AI, as an auxiliary tool, promote the development of academic research? We still don't know.
This article is from the WeChat official account "Machine Intelligence" (ID: almosthuman2014), author: Someone concerned about AI. It is published by 36Kr with authorization.