Sam Altman: Putting excessive amounts of memory in PCs is wasteful, and those resources would be much better off going to AI data centers.

"From the perspective of global resource allocation, it is a 'quite terrible' arrangement to let everyone hoard scarce DRAM in their own personal computers which stay idle most of the time. Memory should be centralized in cloud data centers."

That says it all.

In an interview with YouTube channel Cheeky Pint this Thursday, Sam Altman, CEO of OpenAI, spent the whole conversation drinking beer, talked about local inference of large AI models in a relaxed atmosphere, as well as the resulting problem of insufficient memory in home computers, and dropped a thought-provoking remark inadvertently.

As we know, the memory price surge triggered by the current global AI boom has already pushed up the prices of PCs and mobile phones significantly, which is far beyond the affordability of individual consumers. At the same time, the capacity expansion of major memory manufacturers will take some time to complete. While cutting-edge AI companies are taking up the majority of memory supply, they have already begun to discuss whether the memory in personal computers is fully utilized.

Altman raised an assumption that people can run local AI on their own computers, which may be the devices they already own, or the ones they want to assemble with the rise of large AI model technology. He thinks this is a bad idea.

"From the perspective of global resource allocation, it is actually very bad to let everyone leave the world's limited DRAM memory idle in their own computers with a utilization rate of only 5%," Altman said. "If you want to work in an environment where resource supply is extremely tight, you should maximize resource utilization in the cloud as much as possible. But for some reason, many people's intuition goes the opposite way."

Nowadays, OpenAI's main business is based on closed-source models running in large-scale data centers, and his defense for his own business model has sparked widespread debate in the AI community.

This has also raised many people's concerns: in the future, we may find it harder to own our own hardware or software, and can only choose to rent cloud-based agents and computing resources.

Although the memory used by ordinary PCs, mobile phones and AI acceleration cards is not the same, their production competes for part of the upstream wafer capacity. Altman is discussing the global DRAM resource allocation, and when it comes to who is qualified to decide where the memory production capacity goes, most of the time it is naturally the user who places larger orders.

Part of Altman's view is based on the assumption that data center construction will not slow down. In addition, he also assumes that memory used in AI data centers 24/7 is more valuable than memory used for spreadsheets, web browsing, games, and even occasionally for local AI.

In fact, this interview is very long, and Altman also talked about the PHP vulnerability that caused global panic in the past two weeks, the sandboxing of training models, and AI research in the field of mathematics.

Here are the main points of his remarks:

AI is triggering a wave of vulnerabilities

Regarding the extremely severe PHP vulnerability, Altman's judgment is that the ability of large models to discover vulnerabilities continues to improve rapidly, while the code surface on the Internet is extremely huge. When people start to systematically scan all code with AI, "we will find ourselves in a world with far more serious vulnerabilities than expected". He believes that patching alone is not enough, and the architecture must be reconstructed. Routers, operating systems, and application layers all need to be redesigned for a world where "vulnerabilities will become increasingly intelligent". Two years ago, "rewriting the entire technology stack" sounded like crazy talk, but now there are startups working on building a brand new stack.

He made a rare self-criticism of the industry: "I don't think we have done a good job of explaining to people why all the potential downside risks of this technology are worth taking." He said the narrative of the AI industry is bad sales talk, which promises to cure cancer and reduce costs on one hand, and depicts a future of unemployment, loss of control and the need for strong regulation on the other. He reiterated that OpenAI adheres to iterative deployment and capability overhang, on the grounds that we cannot let "a few square miles of people in San Francisco make all the decisions".

When asked if it is possible to build a "maximum security prison" for the model, Altman gave a pessimistic answer: sandboxes are only effective to a certain extent, monitoring (including chain-of-thought review and activation monitoring) can go a step further, but in the end we must highly trust alignment. He asked in retort: Facing a model that is a million times smarter than the smartest security researcher and can discover new science that humans do not yet understand, do you really dare to trust that sandbox?

He clearly stated that he hates the term "agent swarms", because the word swarm makes people associate with locusts, and insists on calling it agent teams. He believes that the attribute of mutual checks and balances between teams exactly corresponds to the mechanism of achieving robustness through pluralism, which is a security direction worthy of promotion.

The cost of intelligence explosion and scientific discovery

Altman outlined a curve: in the GPT-4 era, it was difficult to effectively invest 1 dollar of computing power in a single problem; chain of thought allows models to "think hard", but now we can parallelize an entire team of agents tens of thousands of times. The usage gap between ordinary users and the most advanced users is constantly widening.

Regarding how OpenAI allocates computing power internally, he said that employees have great freedom, but high token consumption is never rewarded. "A high ranking on the token leaderboard is actually a bit embarrassing, and it is definitely not a reason for promotion." He took the proof of the Navier-Stokes equations as a case: a single proof cost about 10 million US dollars, and it "ran out of control for a week" during the process, because at that time no one knew anything about the scaling law of agent teams. But he expects that a year later, with smarter models and methods, it can be redone at 1/100 of the cost.

He also rarely showed his emotions: the situation at that time was really amazing. On the weekend when a new model was launched internally, everyone threw all kinds of the most intractable problems to this model, and it managed to solve them one by one. People never expected the model to be able to do these things. Everyone witnessed that the most difficult scientific research problems in many people's academic careers were solved by this model one after another, covering many different fields. The scene was absolutely amazing.

"I barely slept for two nights because I was too excited to fall asleep. A lot of the results we published later were actually the problems we solved at that time."

Altman felt extremely excited and happy at that time, and at the same time, an unexpected sense of sadness, melancholy, and a feeling that "an era is coming to an end" welled up in his heart. It's not just nostalgia, but also empathy for human scientists who hold the belief that "if you hear the truth in the morning, you can die contentedly in the evening". He insists that human beings deserve access to basic knowledge, but the industry owes practitioners in these fields a solution.

Although it is still too early to talk about superintelligence, Altman believes that a trillion slightly superhuman AIs are better than one AI with a trillion IQ:

"I would much rather intuitively have a trillion AIs of slightly superhuman intelligence join society than one AI of one trillion IQ join society."

"Intuitively, I would rather let a trillion slightly superhuman intelligent AIs join the society than a single AI with a trillion IQ join the society."

His argument is that the strong robustness of society comes from a large number of members checking and balancing each other and collaborating, which the "single-brain" world does not have. But he emphasized that this will not happen automatically, and must be actively promoted.

Market regulation no longer works in the AI era?

Regarding the global memory shortage, Altman introduced OpenAI's layout: self-developed chips (to improve intelligent performance per watt), self-built data centers, robotics projects, and a bolder concept of "self-replicating data centers", which uses AI to control robots to operate wafer fabs, build turbines and solar panels, and construct data centers. He said "this is already within our grasp".

Talking about why "market self-regulation" has not been realized for a long time, he believes that the current progress is very bad, and explains the behavioral logic of memory manufacturers: this industry has long been a tough business where you invest 20 billion US dollars in capex and earn 1.9 billion US dollars. In history, many giants almost went bankrupt due to over-expansion of factories. Now each company can earn about 100 billion US dollars in free cash flow a year, but painful memories make them extremely conservative in expanding production. From this, he concluded that "the invisible hand is failing now".

He distinguished two types of capabilities in the chip industry: lithography "seems that only one company on Earth knows how to do it", but chip design (the final file sent to the wafer fab) is already an excellent application scenario for AI.

Talking about the real level of Recursive Self-Improvement (RSI), Altman said that now models can complete tasks such as "finding a 5% improvement in computational efficiency", but no one knows how to make AI discover new architectures at the level of Transformer, what is missing is a smarter model. He even predicted that society may need to agree to limit the IQ of a single model for a period of time in the future, "if future progress is insufficient, it will stem from this decision, not the ability of AI to conduct research".

He showed his belief in deflation: the total 5-year ownership cost of each GW of leading AI computing power is about 70 billion US dollars, and it would be more ideal if it could drop to 20 billion US dollars, while the performance of models needs to be increased by another 10 times every year. The way to achieve this is to make energy cheaper, and compress the stacked gross profit margins in the whole process "from sand to output tokens".

In response to enterprises' anxiety about token budgets, he believes that the more common problem at present is that AI is not used enough, rather than being used too much.

How to price personal agents?

Regarding the recently popular personal agent products (such as Muse), Altman's current judgment is that "the Macintosh moment has not arrived yet". People like these products, but they haven't reached the point where they "can't imagine life without it". The form of the winner will be similar to current products, but "it'll just work", reliable, and capable of handling more complex tasks. As for the cat-and-mouse game of computer use, he believes that it will eventually converge to the point where "you have the right to let AI operate your own computer on your behalf".

Regarding the commercialization method of agents, Altman believes that enterprises are clinging to the AI pop-ups and HTML in their own products, worrying about losing customer relationships, but consumers want AI to handle everything for them in their main chat application. He believes that the real moat comes from brand, product selection and fulfillment network, rather than HTML. He voluntarily revealed that he chooses Amazon Pay 10 out of 10 times even if the price is higher, and highly praises Amazon's fulfillment network, while admitting that the cooperation platform in ChatGPT is not doing well enough. He also calls all web forms "fake work", and compares the current situation to the App Store era in the past.

It is precisely because of this that when designing pricing, OpenAI almost followed AWS's practice of quoting by task and completely abandoned the token billing method, but was strongly opposed by customers: "I need to know what I am paying for", and they would rather pay more for understandable units like tokens.

References:

https://www.tomshardware.com/tech-industry/data-centers/openai-ceo-says-its-quite-terrible-for-your-pc-to-have-ram-sam-altman-says-data-centers-offer-better-utilization

https://www.youtube.com/watch?v=iMsUcCYNKqY

This article is from the WeChat official account Ji Qi Xin (ID: almosthuman2014), written by Zenan, and authorized for release by 36Kr.