Altman declares the death of Transformer. AGI will arrive within two years, and the next-generation architecture is on the way.

The architecture that will end Transformer is about to be born! In a recent interview, Altman boldly claimed that the next - generation AI architecture will completely subvert Transformer, and the fate of LSTM may repeat itself.

The biggest beneficiary of Transformer has personally pronounced its death sentence!

In recent days, Sam Altman returned to Stanford and dropped a bombshell in front of a group of sophomores -

In the future, a brand - new underlying architecture will surely emerge, with a performance leap no less significant than Transformer's overwhelming advantage over LSTM back then.

It should be noted that the GPT empire is built on Transformer.

ChatGPT, GPT - 4, o1, and Codex are all fruits of this architecture.

Now, the one who reaps the fruits says in person: The lifespan of this tree is almost up.

Moreover, Altman said bluntly, the AGI we are pursuing may just be a "warm - up"!

The breakthrough of the next - generation new architecture is on the way - The existing high - order LLMs already have sufficient cognitive ability and can serve as a lever for human intelligence to open the door to another technological paradigm.

Use AI to find the next Transformer

People say that brute force can create miracles, but brute force itself has its limits.

Transformer has an inherent computing power black hole: when the text length increases by 10 times, the computational complexity increases by 100 times.

This is why running a GPT - 5.4 - level model today burns money at an astronomical rate.

Altman obviously sees this wall. But he doesn't think there is no way out. On the contrary, he believes that the tool to break down this wall is already in hand.

There is an extremely crucial sentence in the interview: Today's models are finally smart enough to assist humans in this level of scientific research.

This means that AI can already be involved in finding the next - generation architecture.

Using the current AI to discover a new architecture that can replace it, the logical chain is clear:

The stronger the model → the higher the scientific research efficiency → the greater the probability of discovering a new architecture → the new architecture in turn makes the model stronger.

A self - accelerating flywheel is thus formed.

Altman's confidence in making this judgment is related to his unique sense of paradigm shift along the way.

During his freshman summer vacation, he went to work in Stanford's AI lab. His conclusion was "These things won't work at all", and then he went to start other businesses.

However, his attention to AI has never ceased. In his own words, it is a habit of "looking at the big picture" and not getting stuck in a narrow view.

In 2012, when AlexNet came out of nowhere, like most people, he thought it was "pretty cool" but didn't pay much attention.

In the following years, as deep - learning models became larger and stronger, Altman kept an eye on them. Until a certain critical point, the feeling completely changed - it was like an approaching asteroid, extremely crazy, but few people in the world took it seriously.

So OpenAI was founded in 2015. The core belief was simple: push the scale of deep learning to the extreme and see what would happen.

But at that time, when they said they wanted to establish an AGI lab, the veterans in the industry thought they were crazy and even called them frauds.

However, the results are obvious to all.

GPT - 2 made Altman see for the first time what a computer could do that had never been done before. GPT - 3 amazed the world, and GPT - 4 took it to the next level. When you stick to a correct paradigm, the rewards are exponential.

Now, the same intuition is projected onto the next paradigm.

Transformer is not the end, just as LSTM is not the end.

Altman even gave specific advice:

If you are a researcher now, you should stick to this direction to find "where a nuclear - level breakthrough can be made", and you should rely heavily on large models as scientific research assistants.

The whiteboard in Greg's apartment

A night that changed the world

The most interesting part of this interview is Altman's recollection of the early days of OpenAI.

On the first day of work at OpenAI, everyone gathered in the apartment of co - founder Greg Brockman.

At around 9:30 and 10 o'clock in the morning, eight or nine people arrived one after another, sitting on the sofa, looking at each other.

Then someone said, "Okay, what should we do?"

Someone suggested writing a few papers. Another person said they needed a whiteboard first. Then someone placed an urgent order on Amazon for it.

Altman said he felt a moment of panic: This won't work. It doesn't look like a serious startup or an organization that can achieve anything.

But then he said a very "Altman - like" thing: At that moment, you just need to take a deep breath and believe that if you are surrounded by the best people, things will always work out.

He bet right.

During that first week, most of the ideas that later became the core concepts of OpenAI in the first four years were written on that whiteboard. Even though they themselves thought these ideas were unreliable at that time.

They didn't even think about making products at the beginning.

Altman repeatedly emphasized that they thought they were just a pure research lab and that publishing papers was enough.

But later, two things became clearer:

• First, the economic value contained in this path far exceeds imagination;
• Second, the required funds are not billions but hundreds of billions.

The real turning point that made Altman build his faith was GPT - 2.

He said he couldn't remember the exact date when GPT - 2 was released, but he would always remember the night when he had his first conversation with that model.

It did things that I had never seen a computer do before.

At that moment, he thought, it's settled.

As for why the release of GPT - 2 was postponed? Altman admitted that in hindsight, it was a bit overly cautious. But he believes that when facing each new ability level of AI, it's not a bad thing to be a little more cautious.

Of course, you can't be too timid. If companies don't embrace AI fast enough, they will be eliminated by fully autonomous AI companies, which would be a real disaster.

The full view of the Stanford interview

Ten judgments of Altman

Besides the architecture prediction and startup stories, Altman also shared a large number of views during this interview, and almost each one is worth discussing separately.

1. AGI will arrive within two years.

Altman directly told the sophomore students in the audience:

By the time you graduate, you will step into a world where AGI already exists.

Of course, the underlying driving forces of human beings won't change. You still have to move, find a job, and consider starting a family.

But scientific research will be highly automated, and the meanings of starting a startup and working for a big company will be completely rewritten.

2. Programming agents will be the next ChatGPT moment.

What will be the next big hit? Altman didn't hesitate: Programming AI agents.

Closely following but not yet fully booming is the equivalent ability of AI to perform tasks in all knowledge - based jobs.

However, this day is not far off.

3. One person can do the work of a medium - sized company.

In the future, there will be a large number of micro - startups with one person or six partners, whose influence and revenue can even rival today's medium - and large - sized enterprises.

Altman said that the advent of the iPhone was the last such opportunity, and this time it's even more powerful.

Not only can you do things that were previously unthinkable, but you can also build products and companies extremely quickly with very little manpower.

4. An AI CEO? It's not impossible.

When talking about the impact of AI on society, Altman said something thought - provoking:

He will never deceive himself into thinking that in the not - too - distant future, there won't be an AI CEO more suitable to lead OpenAI than him.

If some companies or countries embrace AI while others don't, the competitiveness gap will be overwhelming.

He admitted that he hasn't fully figured out the political, economic, and social impacts behind this.

5. But don't panic. Human adaptability is severely underestimated.

Altman is not an AI doomsayer.

He repeatedly emphasized the view that although AGI sounds like it will completely subvert society, the actual experience won't be as terrifying as it sounds. At most, you'll feel a bit confused in the first few days.

Humans long to be valuable to each other, to compete, to create, and to express themselves. These underlying driving forces won't disappear.

Maybe the occupations 100 years from now will be completely different from today's, but people will always have something to do and will always care about human connections.

6. Don't be afraid to compete with OpenAI.

Someone asked what would happen if OpenAI became an ultimate giant.

Altman's answer was unexpectedly honest: Back then, everyone said it was impossible to compete with Google, but we did it.

One day, there will be a company bigger and more successful than OpenAI, and they definitely won't take the same path.

He even said that if Google hadn't been so "pathetic" back then, OpenAI would never have emerged.

Big companies have their common problems.

7. It burns money fast, but don't panic.

In response to the sharp question about "OpenAI's terrifying money - burning speed", Altman was very calm: It does burn money fast, but if investing 1 billion this year means making 3 billion next year, there are plenty of capitals in the world queuing up for this deal.

8. Self - developed chips are serious business, but building data centers is out of the question.

OpenAI has a huge custom - chip plan and is extremely excited about its in - house inference chips.

As for building its own data center, in Altman's own words, he really doesn't want to do this hard - labor job at all.

He will do it if forced, but it's best to design the server racks to the extreme and let others do the dirty work.

9. There will be a breakthrough in social products.

Altman believes that the opportunities of AI are far more than just "putting an AI into existing software".

He gave the example of social products: Imagine a bunch of AI agents representing their respective users chatting and exchanging information autonomously in a virtual space. This is a real subversion of the underlying logic.

10. Knowing is easy, but doing is even harder.

This is what Altman wrote in his first blog post.

Is it still valid in the AI era? He said it's even more valid than before.

It's getting easier and easier to acquire knowledge, and it's also easier to achieve things, but this is the case for everyone - you have to compete with the whole world.

He said that the top experts he knows who are most proficient in using AI tools all feel that their work has never been more difficult than it is now.

The tools are incredibly powerful, but it's extremely difficult to use them well to maintain top - notch competitiveness.

Sam, are you really happy?

The last unexpected moment in the interview was the soul - searching question from a student.

You know, this is a CEO whose life gets completely out of control after 8 a.m. every day.

He works for a few hours, spends an hour with his kids, and then goes to the company. After that, it's just chaos.

In his words, no company runs as fast, is as chaotic internally, and is under as much scrutiny as OpenAI.

But Altman said that he is one of the happiest people he knows.

He shared a life - changing cognitive shift.

Most people think that the opposite of a bad experience is a good experience, so they suffer when bad things happen. But he reframed the problem. The opposite of a bad experience is actually the complete loss of the ability to experience.

One day, you won't even have the qualification to experience, and at that time, you'll even miss those tough days.

The post - Transformer race

The revolution has already begun

Altman's prediction is not a castle in the air.

The "post - Transformer" competition has long been underway, and the progress is much faster than most people think.

The most high - profile challenger is Mamba.

This architecture, proposed by Albert Gu and Tri Dao at the end of 2023, completely bypasses the "attention mechanism" and uses the state - space model (SSM) to process sequences.

Simply put, Transformer needs to "look at" every word with all other words when reading a passage, while Mamba only maintains a fixed - size memory state and can complete the task in linear time.