"Altman" Stanford Interview: How One Person Can Accomplish the Work of a 100-Person Team With the Help of AI

In a system design class at Stanford, OpenAI CEO Sam Altman talked about an ongoing change: As long as you can afford a reasonable token cost, one person can complete the work that used to require an elite 100-person engineering team to finish. This option never existed in the playbook of startups before, but it has now become a reality. He also talked about the magic of "scale", how ChatGPT and Codex were "discovered" by users, how AI is turning into a new utility like electricity, and why betting against the scaling of LLMs is the most misguided thing to do right now.

The following is the full transcript of Sam Altman's talk:

I. One person with a reasonable token cost can do the work of a 100-person team

Now, as long as you are willing to pay a reasonable token cost, one person can accomplish what a very excellent 100-person engineering team used to be able to do at a startup. This was completely impossible in the past, and was never part of the startup option set; now it is a reality.

As a result, the level of ambition you can take on, the speed you can reach, and the number of things you can push forward at the same time are all completely different. This is not a gradual change, but the option set itself has changed.

I don't know why the following observation holds, and I don't have a theoretical explanation that satisfies me, so I feel a little nervous when recommending others to follow it. But empirically, it does hold true: All the most interesting things I have observed in my career are related to the emergent properties brought by scale.

When something already works in interesting ways at a smaller scale, and you can push it to an unprecedented scale, in most cases this is a great idea, and most people don't push it nearly far enough. This applies to the scaling laws of AI models, to gathering more smart people to work on one problem, and to the scale of companies and economies.

I really learned this at YC. Back then everyone said YC was too big, that it should shrink, and that the best state was to invest in only 10 companies per batch. Many very smart people said so. It was tempting, because the workload would be much smaller. But a large part of YC's magic comes from the network effect within the batch, which is an emergent property that only appears at scale. No one had ever invested in startups on a large scale in the same way before, so no one had discovered this point.

II. The difficulty of scaling: things break in unpredictable ways

One of the reasons people don't do scaling often is that as scale expands, things break faster and in more unpredictable ways. If you really want to scale something up, it will always have some minor flaws. And there will always be very smart people telling you: don't be too ambitious, don't make it too big, let's try a smaller version.

So scaling is essentially a system design problem. Take our scaling of AI models as an example, the questions we had to answer at that time included: Is it technically possible at all? No one had tried to run training across 10,000 or 100,000 GPUs, which required a huge number of engineering talents; how to solve the capital requirement? Where is the business model? How can you afford such a risk? Researchers would also say: with so much computing power, why bet everything on one project? Why not split it across multiple projects?

It is very important to break down these difficulties one by one and solve them one at a time. This is a general approach in almost all scaling fields.

III. The hardest part for humans to restructure: clear goals, clear plans, clear decision-making methods

When scaling, the hardest part to restructure is often not the technology, but the human part. What experience do you have in organizing large numbers of people to participate in the system?

My answer is: clear goals, clear plans, clear instructions on how to get there, and clear rules on how to make decisions along the way. This is extremely important.

Go back to the example of our decision to scale deep learning. Many people said it wouldn't work, that there would be these problems, and that we needed a more diversified portfolio. But once we said: "No, we are going to bet on scaled deep learning. This is our direction. If we are wrong, we will fail, but this is what we are going to do. Here are the reasons, and we believe what the world will look like if we get there." This clarity itself is extremely powerful.

Moreover, humans have not evolved to be good at thinking about exponential growth. It is hard for people to imagine that scaling laws will continue to grow exponentially, that revenue will grow exponentially, and that organizations can withstand exponential complexity. From my experience, it takes a very long time to truly reason with people from first principles about "why this will happen".

IV. ChatGPT was not planned out, it was "discovered" by users

We built GPT-3, and we needed to make money, because we wanted to expand to computers worth billions or tens of billions of dollars. GPT-3 was a cool demo, but we couldn't find products to build around it. We tried a few things, none of which worked out. We knew the model would get better, but we also wanted to start the revenue engine as early as possible. So we said: since we can't come up with a product, let's make it an API first, and hope others can come up with products.

In the summer of 2020, we released the GPT-3 API. It had almost no traction at all at the very beginning. About a month later, as far as we know, it went viral randomly on Twitter. On the same day, several different developers built cool things with it and posted them, and others started to try it.

But if you go back and use GPT-3 or 3.5, you will be shocked at how bad the model was back then relative to the excitement it generated. People tried all kinds of things, and the only business that really worked out in a significant way was copywriting. That wasn't very good, nor was it very exciting. We thought at the time: we can only wait for a better model.

But even though only copywriting worked, developers had already learned to put in a prompt and chat with it. We saw that many people couldn't use the API for business, but used their API keys purely to chat. We said: People obviously want an easy-to-use chatbot.

We had GPT-4 at that time, but there was another ready model in between called 3.5. We also found a new post-training method that made the model follow instructions better and chat more easily. We said: the API might only be a $10 million to $20 million run rate business, but there is something people really love. Following the YC principle — see what users love, then go build it — let's build a chatbot around it.

When we released it, we still didn't think it would be that successful. It was more like a research demo, meant to convince others to build chat products and buy the API. It ended up going wildly viral.

Another thing I learned from YC is: When something starts growing fast, and it's still not good enough, you basically have a product that is guaranteed to blow up. For five days, the traffic spiked and then fell back, and everyone said "it's just a hype cycle"; but the next day the peak was higher, then it fell back again, and people said it was hype. By the fourth or fifth day, I knew: We know how this will go, we have a potential killer product in our hands.

On the fifth day, we gathered everyone together and said: This is an emergency, but a good emergency. We have to build both a company and a product at the same time. Then came two months of crazy expansion. We will figure out the business model later, start charging first, at least don't let the computer bills burn through us. That was obviously not the long-term answer, but at the time we just had to get it running.

V. Codex was originally the main plan: programming is the way models control the world

Before ChatGPT, our plan was actually to go all in on code. We knew these models could write code, and we knew that would be a valuable field. But our internal belief at the time was: Programming is the way models control computers, and robots are the way models control the physical world. If you build a smart enough model, give it the ability to write code, control robots, and drive robot actuators, you can make intelligence do things for you in the real world.

It took us a while to get there. Codex became very good earlier this year, but 5.5 was the real inflection point, when people started using it to do incredible things.

The current capability pipeline is roughly: pre-training, mid-training, post-training, then RL and supervised feedback loops. I think this is indeed the current pipeline, but I expect we will go through a major rewrite. I don't know when or exactly how it will happen, but the very fact that it exists as a "pipeline" is a bit odd, and it doesn't feel like the optimal solution.

What is the optimal solution? I think this is the problem that AI will have to research and solve. We have set a goal: by September this year, use computing power equivalent to 500,000 A100s to act as AI research interns; by March 2028, have a complete, end-to-end, very talented AI researcher that can discover entirely new architectures on its own.

So, with the current pipeline and architecture, we will probably cross that line: AI can do incredible work.

VI. AI is becoming a new utility: selling "intelligence" is hard to resonate with people

I think we are creating a new utility. This doesn't happen very often. Electricity is a utility, the internet is a utility, water is too, there are not many things like this. So we don't have many ready metaphors or experiences to teach us how to explain it to the world.

I recently studied what happened when electricity became a utility. Early power companies did not sell "electricity", because no one knew what electricity was, or why they would want it. Electricity sounded scary: it would come into your house, could kill you, and felt completely different from the world before. Maybe they tried to sell electricity and market it, but it didn't work.

Later what they started selling was: light at night. What you get from us is not electricity, but light at night. By the way, the same thing can do a lot of other things. But people would ask: why would I want that? They would say: one day it will also help you wash your clothes. People said: no, that's too far away, I can't accept that.

I don't know what the corresponding "light at night" for AI is. But even if we are completely right, that intelligence will become a new utility that every company, every customer, every government will need to access, that you will have some kind of OpenAI token subscription, plug it into everything, keep running for you and do all kinds of amazing things — at least for now, I don't think the analogy of "we are selling intelligence" is the right way, because people don't resonate with it.

If we want to become the new utility, we need to find a way to explain to the world: what exactly it means to have such an intelligence pipeline that you can use to do anything you want.

As a consumer, enterprise or individual, you will eventually think in a way that is closer to tokens, or even one level higher than tokens. You won't care too much about where the hardware is, what specific chips they are, what is powering it, these will be abstracted away. What you care about is: when I interact with the system, can I use it a lot? Is it cheap? Does it work well?

Right now, it's like tokens. As we enter a world where everyone has a continuously running agent, always serving us, always useful, you will probably think at an even higher level. Just like when you pay your mobile phone bill, you think of "buying call time and data to access the whole system", and you don't think about the specific hardware in the base station or how it connects to the internet. The underlying hardware will be abstracted away.

VII. Betting that LLM scaling will fail is the most misguided judgment right now

Regarding Yann LeCun's view that "LLMs are a dead end": first of all, in terms of reaching human-level intelligence, these models have far surpassed humans in some aspects, and are far inferior to humans in others. For example, on very long-term, high-judgment tasks, they seem to be much worse than humans.

But on the other hand, just yesterday, one of our models discovered and falsified a conjecture of an Erdős problem, which is a problem that smart people have been researching for a long time. Many smart scientists, I don't know if LeCun is one of them, even said recently that this wouldn't happen. Then the model did it. Now mathematicians are asking: is mathematics coming to an end? What does this mean for our field?

So, LLMs clearly have the ability to discover new knowledge, and can clearly complete some intelligent tasks that humans cannot. They will continue to scale. We will see how large the distribution of tasks they can do better than humans can get, but I suspect it will be a lot.

Moreover, as we talked about earlier, the "lack of belief in exponential growth", this field is indeed held back by a generation of scientists who are overly certain that "scaling won't produce much". Then others just look at the graphs and say "it looks like it's still continuing beautifully, so let's keep going".

World models are obviously important, and things like robots need them. But at the current juncture, betting that LLM scaling will fail, I find it very misleading.

VIII. Advice for "one-person frontier labs": go research inference, make intelligence cheap and abundant

If I were in this class, working on a "one-person frontier lab" project, what would I do?

First of all, your projects are great. A lot of very smart people are researching training ideas right now, and we will have incredible models soon, I promise. But I think we are underinvesting in "delivering massive amounts of cheap intelligence at scale".

So, maybe I would go research the inference stack: how do we make this amazing intelligence cheap and abundant? I think this area is underinvested. And all frontier labs will eventually have to become inference companies to a large extent.

It might be a little late to change the project now, but better late than never. Do what you want to do.

Conclusion

This interview outlines several core judgments from Sam Altman about scale, products and the future of AI: One person with a moderate amount of tokens can complete the work of a 100-person team; scale often brings emergent properties beyond consensus, but most people don't push it far enough; the birth of both ChatGPT and Codex came from "see what users love, then go build it"; AI is becoming a new type of utility, and we haven't found the "light at night