The PPT that was dismissed as "nonsense" by an MIT professor 5 years ago predicted the core ideas of OpenAI o1 and o3

The AI circle is no stranger to gossip.

The protagonist of this story is Giambattista Parascandolo, a leading researcher in the reasoning model track at OpenAI.

In 2020, he went to MIT for a professor position interview, and gave a presentation on conducting reasoning with GPT. As it turned out, most professors on the interview committee dismissed this direction as "nonsense".

https://x.com/turingbook/status/2084003612687249836?s=20

This researcher directly called out the incident publicly, posted this experience on his personal homepage, and attached the introduction of that presentation and the link to the slides.

https://sites.google.com/view/giambattista-parascandolo/home

What was the content of that presentation once labeled as "nonsense"?

According to MIT's official website, the theme of this presentation is how to make artificial neural networks break through the training distribution and obtain generalization and planning capabilities closer to humans.

Parascandolo believes that humans can recombine existing knowledge, identify key invariants, build abstract models, and complete long-term planning. Artificial neural networks still have much room for improvement in these aspects.

At the end of the presentation, he proposed three future research directions: open-ended reasoning in neural networks, unexplored degrees of freedom in artificial neural networks, and using language as a reasoning carrier in reinforcement learning to improve sample efficiency.

The most critical concept among them is "open-ended reasoning".

Parascandolo defines it as: the model can invest more time and computing power to continuously revise its answers. The more difficult the problem is, the more steps the model should think through, and turn additional computing power into better results.

This sounds extremely similar to the compute-time scaling in today's reasoning scenarios.

The standard Transformer at that time had a fixed network depth, and the amount of forward computation each token went through was basically determined, while the difficulty of the problem did not have a stable relationship with the input length. A problem can be very long but with a very simple answer. Another problem may only have one sentence, but require multiple rounds of decomposition and verification.

RNN seemed more suitable for this kind of task. It can run repeatedly, and theoretically can get thinking time of any length. But the curve Parascandolo showed in the PPT indicated that RNN usually performs best near the number of reasoning steps it has seen during training. If the number of cycles continues to increase, the accuracy rate may even decrease instead.

This means that increasing the amount of computation is only the first step, and the model must also learn how to make use of these computations.

The strongest multi-step planning methods at that time relied heavily on model predictive control and Monte Carlo Tree Search. The neural network was responsible for predicting the environment or evaluating values, and the external search algorithm was responsible for expanding future paths. Parascandolo hoped to further integrate long-term reasoning capabilities into the neural network.

His second assumption is to make language the carrier of reasoning.

Parascandolo took the classic game *Montezuma's Revenge* as an example. A reinforcement learning agent trained from scratch needs to try a large number of state and action combinations. Many correct operations are not complicated in themselves, what the agent really lacks is the judgment of "what behavior is more reasonable".

GPT has absorbed a large amount of world knowledge from text. Language can help the model describe the environment, understand goals, decompose tasks and generate high-level plans, and can also greatly narrow the search scope.

In today's context, this line of thinking is easily reminiscent of Chain of Thought, language planning and Agent workflow.

The third direction proposed by Parascandolo is that artificial intelligence systems can reset tasks, return to any state in memory, construct counterfactual scenarios, and adjust time, gravity and observation results in the simulator. The system can even directly read, copy and modify its own activation values and neural network weights.

This is equivalent to including the learning process itself into the operation space of the Agent. It can carry out deliberate practice, generate special training scenarios, transfer existing knowledge, and re-learn for failed trajectories.

The PPT from five years ago almost outlined the roadmap of today's reasoning models.

We also found a blog post he wrote in June 2021, titled *Backpropagation, Evolution, and the "Two Dogs" Misconception*.

This blog post mainly refutes the view that "neural networks need massive amounts of data, so they are not like the human brain".

Parascandolo believes that the fact that humans can learn to recognize dogs after seeing only a few of them does not mean that no experience has been accumulated before. Long-term evolution has precipitated the experience of ancestors contacting the world into the structure and inductive bias of the human brain.

From this perspective, large-scale pre-training of neural networks can be analogized to biological evolution, and model fine-tuning and in-context learning are closer to learning in an individual's lifetime. The text that GPT-3 has read far exceeds any individual, but its pre-training undertakes the function of compressing massive experience and shaping efficient learning capabilities. Therefore, we cannot directly compare all the pre-training data of the model with a small number of learning samples of a human after birth.

This understanding also means that continuing to expand data and computing power may still bring significant improvements. He only analogizes the roles played by the two, and does not think that the specific mechanism of gradient descent is the same as that of biological evolution.

What's the background of this researcher?

He is quite low-key. His latest post on X dates back to November 2024, when he was recruiting two research engineers (RE) and software engineers (SWE) for o1.

According to his personal homepage, Parascandolo's research experience has always focused on generalization, planning and reasoning.

In 2017, he entered the Max Planck Institute for Intelligent Systems and ETH Zurich to pursue his doctorate, under the supervision of Bernhard Schölkopf and Thomas Hofmann. His doctoral thesis focused on OOD generalization in deep learning.

During his PhD, he did internships at Google X in Mountain View and DeepMind in London respectively. The former participated in automated design research on ultra-large-scale simulators, and the latter participated in divide-and-conquer Monte Carlo Tree Search research.

After graduating with a doctorate in September 2021, he joined OpenAI directly. He first joined the reinforcement learning team led by John Schulman, then moved to the algorithm team where Mark Chen worked, and later joined the 🍓 (Strawberry) team led by Jerry Tworek. The Strawberry team is exactly the internal code name of the o1 project.

In 2023, he participated in the R&D of GPT-4 and formed a new team to continue researching reasoning. Later, he participated in the basic research of OpenAI o1 and o3, promoting the expansion of reward modeling, environment construction and general reasoning algorithms. Part of the algorithm description is still covered with black blocks on his personal homepage.