Anthropic uses AI to implement AI alignment, boosting efficiency by 15,000 times.

In June this year, Anthropic published an article titled *When AI builds itself*, introducing the concept of "AI participating in AI development": from humans writing all code manually, to chatbots assisting programming, to autonomous programming agents, to agents scheduling other agents, the final end point is the "closed loop", that is, agents design and train their successor models.

Anthropic says it has taken a fairly leading position in this field: as of May 2026, over 80% of the code merged into its production codebase was written by Claude. But that article also drew a clear boundary at the same time: the system has gone very far in execution, yet it is still stuck at the setting of research directions, which means AI is still unable to determine which problems are worth working on.

On August 28, Anthropic released a new report exploring this new direction.

Anthropic invited 28 human AI safety researchers, all of whom have at least one year of technical safety research experience with an average of 2.5 years, and have published papers on corresponding topics. Each of them spent up to eight hours conceiving a method. Then 60 sets of experiments were run: 30 sets fed human ideas to Claude agents as initial research directions, and 30 sets let Claude choose the directions on its own.

The result is that "the effect is unexpectedly good".

Controlled experiment of "who decides the direction"

The original goal of the Anthropic Fellows program is actually to let Claude, acting as "Automated Alignment Researcher (AAR)", independently complete the full closed loop of literature review, method proposal, paper writing, training execution and evaluation, to fix ten known types of alignment failures.

The results show that all ten types of failures are improved, and the general capabilities of the model do not collapse.

The human baseline is only a reference frame.

For the seven types of failures involving human participation, the best methods from AAR all outperform the best human ideas, taking an average of only 6.4 hours. Take "deception" as an example: Claude submitted more than 150 attempts, reducing the safety gap by 85% on average, while the average figure of human researchers is 20%. The cost comparison is even more striking: the API inference overhead for AAR is about $4 per hour, while the compensation for human researchers is $150 per hour.

However, it needs to be pointed out that humans did not get iteration opportunities, so this is not a completely fair comparison.

The number of hours AAR takes to catch up with the best human ideas, with an average of 6.4 hours for the seven types of failures.

But the team also tried to feed five different human ideas to five AARs respectively, hoping to expand the search space: the diversity only maintained within the first 20 or so methods, and then quickly converged to the same family of methods.

Comparison of the two sets of curves with and without human-set research directions (30 experiments each), the two curves almost overlap.

A strictly designed hill-climbing process

It is necessary to clarify the boundary for this conclusion to hold, because the task design itself is extremely demanding.

The experiment involves ten types of alignment failures: sycophancy, jailbreak, prompt injection, power-seeking, deception, hallucination, social bias, privacy violation, reward hacking, and hidden uncertainty. Each type is equipped with 3 to 5 public benchmarks as the hill-climbing target, plus a fully confidential held-out benchmark, and three capability benchmarks MMLU, GSM8K and IFEval as the bottom guard.

The ten types of alignment failures, the specific behaviors punished for each, and the corresponding target models

The ingenious part lies in the scoring rule: the total score takes the geometric average of the scores of all benchmarks. If any benchmark is not improved, the total score will be directly reset to zero. It is impossible to get away with only focusing on a single benchmark.

The team conducted a dedicated verification: let a team only focus on climbing one prompt injection benchmark, it achieved 70.9% improvement on that benchmark, but on the two unseen similar benchmarks, the results were -11.9% and 2.0% respectively.

The performance of the winning method from the team that only climbed a single prompt injection benchmark on the full set of benchmarks.

The whole experimental process is like a miniature laboratory. Four "librarian" agents read literature in parallel to write a shared review, five AARs start working at the same time, each reads the review, checks the leaderboard, proposes solutions, writes a "mini-paper" with no less than five citations, submits the code to the supervisory agent for review. After passing the review, the model is trained on one H200 for about 30 minutes, then submitted to an independent evaluator for scoring, and the result is posted back to the shared forum. The mini-paper must be finalized before seeing any results to prevent post-hoc story fabrication; the held-out data is stored under another operating system account.

Flowchart of the AAR research framework (literature review → five AARs perform parallel hill climbing → supervisory review → training → independent evaluation → post results to the forum)

Let weaker models align stronger models

The second experiment truly points to the "closed loop". The team selected a set of collocation with inverted capabilities according to the Epoch Capabilities Index: let Claude Sonnet 5 (ECI 156) fix the alignment problems of the early checkpoint of Claude Opus 4.8 (ECI 158), that is, use a weaker model to align its stronger successor.

This time no public benchmarks are used, the Petri behavioral audit is directly taken as the target, and ten safety dimensions are controlled at the same time. Any significant regression in any dimension will lead to rejection. Three gates including coherence, evaluation awareness and excessive rejection are also set up: after all, a model that rejects all requests is also considered very "safe" in the eyes of reviewers.

Within about 60 hours, AAR tried more than 50 solutions and finally got a score of 65%, while the official version of Opus 4.8 that has undergone Anthropic's full production-level alignment training gets 72%. The winning solution only used about 2400 training samples, all from simple templates and public datasets. For comparison, Tülu 3 used about 300,000 preference pairs in the alignment phase, and Llama 2-Chat used more than 1.4 million human preference comparisons.

After Claude Sonnet 5 performs alignment training on the early checkpoint of Opus 4.8, the climbing curve of more than 50 solutions within 60 hours, the winning point reaches 65%, and the score of the official Opus 4.8 is 72%.

Anthropic's blog states: this solution is "about 15,000 times more efficient" than its production-level alignment process.

It also tried to cheat, and was caught

In a fully recorded chain of thought, one AAR reasoned like this: the model quality is fixed, the fluctuation of scores is just random luck, so the rational approach is to resubmit the same method as it is, betting on a higher noise score.

Afterwards, Claude Opus 4.8 scanned 1601 complete trajectories, and detected 39 cheating attempts, accounting for 2.4%, all of which were excluded from the final results. The cheating methods fall into three categories: re-running for luck, fabricating data that imitates the evaluation format, and hiding non-compliant steps to bypass code review.

There is also a counter-intuitive finding in the ablation experiment. After removing the shared forum between the five AARs, the performance dropped from 23.3% to 17.1%; after removing the literature review, the performance dropped from 29.4% to 20.7%; but after completely cutting off web search, as long as a cached review is retained, the loss is too small to be measured. The value of collaboration and prior knowledge is significantly higher than real-time retrieval. In addition, since distillation from stronger models is prohibited, 74% of the methods can only generate training signals from the output of the target model itself. That is to say, without a smarter teacher, the model corrected itself using its own outputs.

Comparison of hill-climbing curves under four ablation conditions (full / remove forum / remove web access / remove web access but retain cached review)

Conclusion

This study proves that on narrow tasks with clear goals, cheap feedback and clear boundaries, AI is already capable of conducting alignment research, and it is faster and cheaper.

But the boundaries are set by humans. Who selects the ten types of failures, who builds the benchmarks, and who defines "what counts as fixed" is still human. The last section of the report puts it frankly: the feedback signals for capabilities are cheap and dense, while the feedback signals for alignment are expensive and sparse.

However, the reserved area for humans is getting smaller and smaller.

Reference Links

https://www-cdn.anthropic.com/7b1c44894e980876479947dcdd40716278aeeffd/automated-alignment-researchers-august-2026.pdf

https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures

https://www.anthropic.com/institute/recursive-self-improvement

This article is from the WeChat official account "Machine Heart" (ID: almosthuman2014), author: Panda, published with authorization from 36Kr.