Meta CTO reveals the truth behind Llama 4's underperformance: We deliberately held back our own progress to force Llama 3 into the spotlight

In software R&D, the most dangerous mistake is burning through all your future reserves in one go. Meta's CTO Andrew Bosworth (referred to as Bos below) recently revisited this unintended misstep during the Big Technology Podcast, discussing the passive situation that followed the launch of LLaMA 4.

When delivering LLaMA 3, the Meta team was so eager to deliver a resounding success that they fully exhausted all the pathfinding work and forward-looking research originally reserved for the next generation. As a result, LLaMA 3 earned widespread acclaim, but it left the R&D pipeline completely empty when work began on LLaMA 4, causing Meta to immediately fall behind competitors in core areas of Reasoning and Mixture of Experts (MoE). This technical gap forced Mark Zuckerberg to shift strategies entirely, pushing the company into an all-out "founder mode" focused on aggressively securing computing power and top talent.

Even for a large organization like Meta, such a drastic pivot inevitably caused internal friction. To make up lost ground quickly, Meta launched an extremely urgent "wartime mobilization": thousands of senior engineers from non-AI departments were ordered to pause their core projects and reassigned overnight to label expert code data for AI training. This heavy-handed top-down mandate sparked deep internal resentment, with some employees even complaining to the media that the work felt like "forced labor in a concentration camp." Bos made no attempt to whitewash the poor internal communication, acknowledging in a leaked internal email that the management of this initiative was "atrocious." Yet he insisted the strategic direction was correct — to teach large language models the fundamental skill of "how to use computer software normally like a human," Meta began tracking and collecting low-level interaction traces of employees typing and clicking through interfaces, because in the reinforcement learning era, this seemingly unglamorous "dirty data" is the most scarce and valuable fuel for progress.

Bos argued that these internal management tensions and growing pains align with a concept he once wrote about in a neurobiology blog — "Pain is rehab": you cannot rebuild the system without going through the withdrawal-like agony of breaking old habits. During the conversation, he discussed why the era of universal monolithic large models is over, why he stubbornly refuses to install the app for his home toaster, and his vision for a future of smart glasses that requires no app stores at all:

LLaMA 3 burned through every available technical path in advance, leaving an empty pipeline for LLaMA 4. When developing LLaMA 3, Meta drained all its technical reserves to guarantee the success of that release. This created a technical gap when work began on LLaMA 4, forcing the team to start from scratch in the now-critical fields of reasoning and Mixture of Experts (MoE).

The era of universal monolithic large models is dead, and the future belongs to specialized small models and multimodal systems working in tandem. Previously the industry was fixated on building a single trillion-parameter monolithic model to solve every possible problem. The current trend is that extremely expensive high-capability models are only used for the most complex tasks, while the vast majority of routine daily work is delegated to cheaper, lower-latency, vertically specialized small models.

Large tech companies follow their own survival logic: without full control over your core assets, you are forced to accept whatever prices competitors dictate. While Meta does not rule out renting models from OpenAI or Anthropic to run tests for internal and external products, it must ensure its self-developed foundational models maintain absolute competitiveness — this is the only leverage that prevents you from paying exorbitant "rents" to rivals at the negotiation table.

Meta acknowledges that the internal management and communication around the "code data labeling" initiative was atrocious. To seize the strategic window for code fine-tuning, Meta forcibly reassigned thousands of engineers working on conventional software into AI labeling teams. Leadership's rush for results without sufficient explanation sparked extreme pushback, including employees publicly describing the work as "concentration camp labor," and the CTO admitted the management of this process was a clear failure.

Tracking employee operation traces is meant to fix AI's long-standing weak spot: using computers. While AI has demonstrated remarkable skill at writing poetry and solving complex problems, it remains clumsy when interacting with real-world complex software interfaces and underlying operating systems. Meta's monitoring of top programmers' keystrokes and button clicks aims to teach AI the subconscious behaviors of human-computer interaction, enabling it to master how to use various office tools.

AR glasses will never need software stores, and the App era will eventually become obsolete. There is no need to install a dedicated app for every household appliance, such as a toaster — the CTO still stubbornly refuses to install the one for his own toaster. In the future AR glasses ecosystem, AI will instantly compile (Vibe-coding) and generate any application you need in the background, with services automatically matched and invoked entirely behind the scenes.

The Path to Frontier AI, Model Leasing, and the Dilemma of Consumer AI

Host: Bos, it's great to meet you. As we were chatting before recording, the tech industry is in an incredibly chaotic period right now. As far back as I can remember, I have never seen development move at this pace.

At the core of all this are AI models. AI models are the foundation of everything — without a capable, even leading, AI model, it is extremely difficult to build meaningful products on top of it.

For a long time, the industry consensus was that to build a top-tier AI model, you need massive computing power and leading researchers refining algorithms. Meta already had massive computing power and a top-tier research team working on algorithms, yet a truly industry-leading AI model remained elusive.

Can you share what you learned from this experience? Were the core assumptions about what it takes to build a top-tier AI model fundamentally flawed?

Andrew Bosworth: I would add one more critical element to that list: high-quality data. And that is something you already have access to.

Host: That is definitely an asset your team possesses.

Andrew Bosworth: I believe that too.

The story actually unfolds across two parallel narratives. The first is that looking back at Llama 1, Llama 2, and Llama 3, we were clearly at the cutting edge of the field, driving the entire industry forward. As you know, Facebook's Fundamental AI Research group (FAIR) has existed for over a decade.

I first got my firsthand glimpse into the momentum of AI there, when an AI chatbot popped up in my feed. Later I met Yann (LeCun) and started connecting with people at FAIR, and I realized just how rapidly this technology was advancing. So Meta got involved very early on.

The real gap — which I think is now an open secret — was that we failed to recognize the issue at the time. When we were integrating Llama 3, we poured almost every research achievement we had into it, mobilizing every available resource, and inadvertently cut off our own subsequent R&D pipeline.

The system is supposed to operate this way: you build a foundational model, with one team working on incremental improvements on top of it, and another team exploring entirely new technical paths outside that framework. But we did not realize — and this shows we were not paying enough attention at the time — the only reason Llama 3 reached that standard and became such a widely acclaimed model was because the team bet all of our future technical reserves on delivering that single release.

As a result, when we started working on Llama 4, we no longer had access to the exploratory research results that other labs were still advancing. So we fell behind on reasoning capabilities, fell behind on Mixture of Experts (MoE), and fell behind on a whole host of key technologies that underpin the industry's continued progress.

I think this was a very public setback for us roughly a year ago, and it prompted Mark (Zuckerberg) to reposition AI from "AI is one of our many bets" — which was our prior mindset, treating AI as just one of several investment areas — to "AI is a bet that underpins the entire company, and we must completely transform how we approach it."

The term sounds a bit cliché, but I cannot find a more fitting description: founder mode. Mark truly shifted into his unique, fully dedicated state, focusing all his energy on securing all the computing power and top talent we needed — the researchers you mentioned who joined us officially came on board around that same time last year.

Alexander Wang just celebrated his first work anniversary with the company, and I have thoroughly enjoyed working with him and learned a great deal. We are now starting to see the returns on all those investments.

Take Muse Spark as an example — our newly released model, which is not yet our most cutting-edge offering, but has already received extremely positive feedback. Its performance varies across different benchmarks, but it delivers exceptionally strong results in the areas we prioritize most, the areas we believe best demonstrate our product's unique value.

So you are completely correct — in terms of public perception around models, we are currently positioned exactly as you described. But we have built a team I truly trust, we have all the computing power and data we need, and I am very confident we will reach the position we deserve to occupy.

I want to add a second point that I believe is strategically critical: models themselves can be "rented." You can use Anthropic's models, OpenAI's models, or Google's models. They are all excellent models, and using them directly works perfectly well.

But the real value we aim to create for the world lies in products. Our vision for "personal superintelligence" is something I believe only we are uniquely positioned to deliver. This is not just because we have access to data — though that is certainly valuable — but more importantly, compared to almost any other company, we have a far greater ability to understand you, what you want to achieve, who you are in the world, and what truly matters to you.

So owning your own model is one piece of the puzzle; you need to own it for strategic reasons to avoid being held hostage by external providers, and more importantly, to maintain full control over your own destiny. But the model itself is not the end value.

I believe we will very soon enter a world where consumers simply do not care. They will not want to specify which model they are using, or whether it is version 4.7 or 4.8 — just as you do not care whether I am using an Oracle database or a SQL database. You just want the functionality, you just want the product to work well. I believe that will ultimately be the standard by which we are all judged.

So the fact that everyone is still debating models today shows that we are not yet focusing enough on the user side, on the actual tangible benefits people can gain. Beyond demonstrating our technical achievements, this is the real story we need to communicate effectively — we need to clearly prove that value to ordinary consumers.

The Era of the "Monolithic Large Model" Dominating Everything Is Over

Host: I want to start from a technical angle. The initial question I raised was whether you could brute-force build a competitive model simply by throwing unlimited resources at it.

From what you are saying, that approach no longer works, because new technologies like Mixture of Experts and reasoning mean you have to refine the foundational pre-trained model at a granular level to create the top-tier models we see today. That is exactly the challenge Meta is currently tackling.

Andrew Bosworth: That is right, and it is not just us — this applies to the entire industry, by the way.

The era of the "monolithic model" effectively ended around the time LLaMA 3 was released. The old philosophy of "we just have one model, we test how smart that single model is, and its performance on every task is represented by that one metric" no longer works. We now live in a world where whether you are using OpenCode, Claude Code, or Codex, these "harness" orchestration frameworks dynamically invoke different underlying models based on the type of task.

For example, it might call on a multimodal model. If you are using Gemini and encounter an image generation task, it will offload that work to Nano Banana. We have long since left the world where "one model rules all."

What you actually need is one extremely capable, resource-intensive model that you can distill into smaller variants for all kinds of use cases, only calling on the full high-intelligence version when you absolutely need that level of sophistication, because running such a model is prohibitively expensive. For all other scenarios that do not require genius-level intelligence, you use cheaper, faster, lower-latency smaller models.

If you think about all the tasks humans need to complete, I do believe in the scaling law — as computing power expands, the raw intelligence of models continues to rise along that upward curve. But human tasks do not have infinite demand for intelligence; many human tasks can be perfectly completed with ordinary levels of intelligence.

So I truly believe the future will be a layered ecosystem: the question will no longer be "OK, which single model will rule everything," but "how can we combine a suite of models to solve these problems with the perfect balance of performance, cost, and value."

Owning a Competitive Self-Developed Large Model Is Your Safeguard Against Being Extorted for Exorbitant "Rent" at the Negotiation Table

Host: You raised several very interesting points. First, I agree that the product itself is what ultimately matters. And owning your own model is critical to maintaining independent control.

Let us talk about that. You have certainly seen Apple's approach — they struck a deal with Google to distill Gemini, or create some kind of Gemini-derived variant. Early reports suggest that Siri's performance with this new technology is quite good.

Did you ever consider striking a similar deal with Google, while simultaneously developing your own model in parallel to maintain independent control? At least in the short term, using that approach to advance your products as quickly as possible?

Andrew Bosworth: There are two parts to that answer.

We do use many different models today, and similarly, you want to provide consumers with the model that is best suited for their needs — price, performance, and latency are all important factors.

But owning your own model not only lets you control your own destiny, it also gives you far stronger negotiating leverage when discussing all kinds of terms with other providers, ensuring you can deliver the best possible experience to your consumers.

Spending that much money — roughly a billion dollars to Google — it is too early to draw conclusions. I do not know what that end experience will ultimately look like, and I do not have access to it myself, so we will have to wait and see how it plays out.

At least for us, we are focused on "personal superintelligence." We want to deliver an extremely powerful, highly targeted capability to the products we build, not just generic intelligence. That is critically important to us.

We do not see this as a minor incremental improvement to an existing system, but as an entirely new way for people to interact with their computers.

This traces back to the years of work we have done at Reality Labs. We have always tried to follow in the footsteps of pioneers like Xerox PARC, SRI, and Bell Labs — constantly asking: how do we transfer information from our brains into machines? That is why we research neural interfaces and a whole range of related technologies. Conversely, how do we transmit information back from machines to our brains? That is why we invest in augmented reality and virtual reality research.

AI might be the best tool we have ever seen for moving information from our brains into machines, especially when it can observe the vast amount of information around us. I believe these are exactly the unique capabilities we want to leverage. This is not just about the model itself, but about the model's ability to process these entirely new input signals and integrate them into a closed-loop system.

So we are fully committed to building excellent models, and I have enormous confidence in the team we have assembled. The key point I want to make is that a model alone is not enough. Whether Apple's strategy of simply "renting" that model is sufficient, I do not know — I am not sure if they have a broader vision for integrating it fully into people's daily lives.

Host: So you are not planning to "rent" models as your core strategy?

Andrew Bosworth: No, we do "rent" models. As I said, we use other companies' models too.

There is no reason not to — while we are investing heavily in self-developed models internally, we also do some development work on models provided by Google, Anthropic, and OpenAI.

And the entire concept of being "model-agnostic" only makes economic sense if you yourself own a competitive model that you can fall back on when you need to. That also sets a hard upper limit on how much "rent" other providers can charge you.

But it is also worth noting that whether we are talking about internal developers or ordinary consumers, I do not want people to have to worry about models long-term. Right now they have to, because everything is still tightly coupled. But in the long run, they should only need to focus on the goals they want to achieve, which is what really matters.

So owning a model that is absolutely leading, at the cutting edge of the industry, is an important strategic move, very critical. But it does not mean you win the moment you have it.

You still have to connect it to a whole host of other elements: products, distribution channels, and consumer experience. I believe these four components — models, products, distribution, and consumer experience — combined together, are our real competitive advantage over rivals. Most of our competitors,