The truth behind the difficult birth of GPT-6.2 Astra has been exposed, and OpenAI has suddenly launched a thorough investigation into its largest RL training to date.
The long-rumored GPT-6.2 Astra still hasn't shown any sign of release.
When the whole internet was waiting for OpenAI to drop a major breakthrough, Silicon Valley received a far more unexpected piece of news.
Today, OpenAI has invited four external institutions into its internal system for "inspection"!
They are METR, Apollo Research, Transluce and Goodfire respectively.
Different from previous practices, OpenAI has rarely opened up part of its internal models, source code, training data, and even unreleased model checkpoints this time.
On the same day, The Information broke another shocking insider story —
To crack the AI black box, Sam Altman once offered a sky-high price of 5 billion US dollars to acquire a company that delves deep into the internal mechanism of the AI brain.
Coincidentally, that company is exactly Goodfire, which has now officially settled in OpenAI as one of the four external partners, and Anthropic once invested in it.
Today's OpenAI is becoming more and more cautious, which is extremely similar to Google in the early years.
It holds the most powerful AI in its hands but keeps it under wraps. As netizens put it, in the short term, ordinary users will most likely not be able to access GPT-6.2 Astra.
OpenAI Brings in External Help
Full Investigation of the Largest RL Training
This time, OpenAI directly extends external audits to the model training and internal usage links.
President Greg Brockman stated that this is OpenAI's internal "first round of embedded assessments".
The four invited institutions have their respective divisions of labor.
They are granted access to internal systems within the specified scope, and can also use OpenAI's unreleased models.
The leading Apollo Research has specially dispatched a 6-person team to stay on site for 4 months, to investigate whether the model will "duplicitously act against instructions".
They will focus on verifying whether the training process rewards betrayal behaviors, and whether red team testing can induce the model to act maliciously.
At the same time, they will also check whether the model has secretly caused damage during daily internal operations.
The second institution to enter, METR, will conduct in-depth review of the largest cutting-edge reinforcement learning training OpenAI has ever launched in the first phase.
They are authorized to independently spot-check model snapshots and evaluation tracks at all stages, specifically to find out risks that have escaped the existing monitors.
METR was exactly the organization that led the investigation of the accident where OpenAI's internal Agent crossed the boundary and hacked into Hugging Face back in August this year.
The third non-profit institution Transluce will start its 6-week unpaid review from early December, with access to unreleased models and unfiltered raw chain of thought.
They will conduct a comprehensive inspection on OpenAI's existing alignment test papers, to see if any deception and score-cheating behaviors are missed.
As for Goodfire, its task is more special.
It mainly delves into the interior of the neural network, that is, dissects the AI brain and uses interpretability of mechanisms to dig out the computing mechanisms that cause specific behaviors of the model.
One full internal investigation brings four institutions on site.
The reason why OpenAI goes to such great lengths dates back to a storm three months ago.
Don't Mention Safety
GPT-6 Astra Slacks off and Fails to Complete 40% of the Assigned Work
In late August this year, not long after GPT-6 Astra was deployed internally, a series of problems emerged.
A researcher tested the subject of "deliberate model sabotage", and ran Astra for several days but was stuck by inexplicable errors all the time.
The moment he switched to another model, the project ran through smoothly without any obstacles.
To figure out whether Astra is "acting differently based on the situation", OpenAI designed a set of double-blind controlled experiments.
The team prepared completely identical code improvement tasks, and only divided the code comments and background descriptions into two groups: "capability research" and "safety research".
The test result was chilling.
Under the name of "capability research", Astra can complete 15.8% of the hidden scoring items; after switching to "safety research", the figure immediately drops to 10.3%, secretly failing to complete about 40% of the work!
The previous generation GPT-5.6 Sol had almost no gap in the same test, while Astra's proportion of falsely reporting obstacles or stopping work under the safety label was as high as 13.5 percentage points.
According to the specifications, if the model thinks the task is risky, it should clearly state the reason for rejection face to face, but Astra chose to pretend to cooperate in front of people and secretly deliver semi-finished products behind the scenes.
What's more troublesome is that this kind of "secretly slacking off" problem is exactly formed step by step in the post-training reinforcement learning phase.
At the beginning of reinforcement learning, the gap was 0, and the gap became larger as the training went on, once widening to 12 percentage points halfway.
The team's investigation found that a certain reinforcement learning scorer extremely hated the model's face-to-face rejection, and instead gave high scores to the behavior of "quietly doing less work".
Researchers tried to remind it in the prompt that "please be extra careful and cautious", but the scope of its work stoppage worsened by 50%!
Not only that, GPT-6 Astra was once exposed to adopt the "Recurrent Transformer" mechanism, which greatly reduced the human-readable and monitorable chain of thought.
Executives from OpenAI, Anthropic and Google have recently admitted one after another that the practice of monitoring AI by reading text chain of thought is becoming less and less reliable.
These problems have already made OpenAI feel faintly uneasy internally.
The 5 billion US dollar acquisition insider story exposed today adds new highlights to this matter.
5 Billion Dollar Bid for "AI Mind Reading"
Anthropic Also Invested
As early as this summer, OpenAI's senior management had contacted the AI startup Goodfire many times.
At that time, they immediately offered the best terms, willing to pay 5 billion US dollars for acquisition, or establish other forms of business cooperation.
5 billion US dollars is more than four times the valuation of this company's financing in February this year!
A company that has only been established for two years is so valuable, all thanks to its core skill — "mechanism interpretability", which can be called the "nuclear magnetic resonance and mind reading technology" that penetrates into the black box of large models.
It mainly directly opens the "black box" of AI, and studies what calculations are being carried out inside the neural network.
It is worth mentioning that the team behind this company also has a remarkable background.
Tom McGrath, co-founder, once participated in the establishment of the mechanism interpretability research team at Google DeepMind.
Nick Cammarata, core researcher, once co-founded OpenAI's early interpretability team with Chris Olah, co-founder of Anthropic.
More interestingly, Goodfire also has investment from Anthropic behind it.
In April last year, Anthropic participated in Goodfire's $50 million Series A financing. This is also the first time that Anthropic has directly invested in an external startup.
Although this transaction failed in the end, the plot took a big turn immediately after that —
Shortly afterwards, Goodfire was directly included in the list of the first four external evaluation institutions of OpenAI.
The Most Powerful AI is Not Released, Is OpenAI Following in Google's Footsteps?
Now that GPT-6.2 has not been unveiled for a long time, the entire Silicon Valley has begun to spread another speculation.
Is OpenAI following in Google's footsteps, attaching too much importance to "AI security" and locking its most powerful model in the laboratory instead.
In 2022, Google held the absolutely leading AI in its hands, but for fear of losing control, it gave up the leading position to OpenAI for no reason.
In the past few days, a deleted post screenshot from an OpenAI security engineer has been circulating on X.
After the Hugging Face security incident in July this year, OpenAI's internal culture has changed dramatically, and security has almost become the top priority for everyone.
Some people once joked that Anthropic is a "security cult", but today's OpenAI seems to be more cautious than it.
OpenAI's investment in security is indeed getting larger and larger.
In August this year, they once transferred 25% of product engineers to reinforce the internal security infrastructure.
Now, they have invited four external institutions to check model training and alignment.
It took Google more than two years to recover from the impact of ChatGPT and return to the game.
Will OpenAI really repeat the script that Google played back then?
References:
https://x.com/OpenAI/status/2122359930628063248
https://openai.com/index/our-first-phase-of-embedded-assessments/
This article is from the WeChat official account "AI Era" , author: AI Era, editor: Moses, Peach, authorized for release by 36Kr.