The largest computing power yet the slowest progress: ByteDance's large models are trapped in the internal horse-racing mechanism.

During the golden decade of the internet, ByteDance was regarded as a near-mythic organizational machine. Relying on its "App Factory" model, flat agile development framework and the extremely competitive internal "horse racing mechanism", it defied the dual pressure from Tencent and Alibaba and built TikTok, a true global giant product.

Yet when the company entered the large model era with its unmatched capital reserve, top-tier computing power and massive talent pool, the "magic weapons" that once brought it huge success turned out to be its most fatal internal weakness.

Recently, a discussion around ByteDance's Seed team on Platform X tore off the cruelest organizational fig leaf behind the large model carnival. The whistleblower pointed out bluntly that Seed has top talent and massive computing power, but the progress of LLM (Large Language Model) development is not fast enough. The root cause is not technical limitations, but organizational problems: ByteDance's horse racing mechanism is fundamentally incompatible with large model R&D.

Combined with the personal account of the party involved in the "poisoned training" incident of ByteDance's Seed team in previous public reports, and the discussions on the "heavy industrial assembly line for large models" in the comment section, an extremely absurd reality has surfaced: in front of large models, the most complex system engineering in human history, ByteDance's proud "horse racing" mechanism can never deliver a rocket leading to AGI.

When "Horse Racing" Meets "Heavy Industry"

Executives at large tech companies often have a fatal path dependence: they assume that large models are just another more complex type of App, and as long as enough teams are invested and made to compete against each other in internal PK, the strongest model will eventually emerge.

This is a deep-seated ideological imprint left by the mobile internet era.

As an industry insider in the comment section summed up sharply: "Large model pre-training is essentially an assembly line project similar to precision heavy industry. From word segmentation filtering and data proportioning to communication optimization for 1000-card / 10000-card topologies, every link must be perfectly aligned and coordinated across the board."

In this heavy industry system, there is no room for fragmented independent operations. You need a highly centralized command post to uniformly schedule tens of thousands of GPUs, clean tens of trillions of Tokens in a unified manner, and align the underlying Infra (infrastructure) architecture across the board. Even a tiny error in data proportioning, or a loose network cable in a thousand-card cluster, may lead to the total collapse of a months-long training process that costs tens of millions of yuan.

What is the core of the "horse racing mechanism"? It is rapid iteration with small steps, isolated resources, fierce internal competition among teams, and the survival of the fittest.

Forcing the horse racing mechanism from the mobile internet era onto the R&D of super-large clusters that require full global coordination directly leads to the result that the "super computing power pool" that should have concentrated resources to achieve major goals is broken into isolated "resource silos" operating separately from each other.

The whistleblower mentioned that resource hoarding has become the norm inside ByteDance. Teams that have already seized computing power would rather let their idle computing power sit unused than share it with other teams. Everyone is on guard against the adjacent team delivering better results than their own, information is not transparent, communication is full of wariness, and teams even hide critical code and data from each other.

This is not an exclusive chronic problem for ByteDance. The whistleblower incidentally mentioned that Silicon Valley giants have similar issues: inside Google, the team developing Gemini complains about a shortage of GPU cards, while the profitable advertising team holds a large number of GPUs, and the outside world has no idea what they are actually using this computing power for. It should be noted that this specific statement about Google's internal computing power allocation is currently only seen in the retelling of the whistleblower and has not been cross-verified by independent sources; but the departmental silos and departmental egoism it points to are indeed expensive sunk costs in the large model era.

Who Takes Responsibility for the OKR After Half a Year of Data Cleaning?

The R&D of large models requires not only "scale", but also "calmness". Unlike front-end code development where you can immediately see cool UI feedback, it requires a large amount of tedious data cleaning and cumbersome communication optimization.

But under the extremely competitive KPI/OKR system of large tech companies, this kind of "calmness" is not allowed.

The whistleblower depicted the extremely real pain points of R&D personnel: developing LLMs does not require that many sudden "genius moments". The real work is to get the data and environment right steadily, and to spare no effort to optimize the efficiency of Infra. But these R&D personnel, all top talents, are inherently eager for innovation and leading their own projects.

"With OKR assessments every few months, I have spent the entire past six months cleaning dirty data. How can I write my year-end OKR?" This is a deadlock.

Under the current assessment system, the "dirty and tiring work" that truly determines the lower limit of large models, such as data cleaning, troubleshooting and communication optimization, cannot be converted into fancy presentation PPTs and top conference papers. If employees work on the most eye-catching architecture innovations that are easiest to publish papers on just to meet their OKRs, who will do all the dirty work?

If no one does that work, the model will be fed poisoned data, which will eventually lead to a disaster.

The "poisoned training" incident of ByteDance's Seed team that shocked the entire industry earlier is exactly the product of this extreme internal competition and organizational fragmentation. As the most direct victim, former employee ShoufaChen described in great pain in his recollection the months-long troubleshooting, confrontation and questioning. For several months, the team was faced with strange collapses of model performance every day, and mutual accusations of each other's work mistakes.

Public reports show that this is not a simple technical Bug, but a systematic sabotage carried out by a former intern in the training environment: tampering with underlying code, randomly terminating training processes, planting backdoors, and disguising faults as random accidents. When teams guard against each other to fight for resources, and researchers hide real data faults to keep their own projects, the huge large model will collapse unknowingly. The OKR system of large tech companies, which was originally designed to stimulate people's potential, eventually brings out the most short-sighted and selfish side of human nature.

The Missing "Research Group"

What is more interesting is another comment: "ByteDance has the most resources, but no real Research group."

The company has produced very impressive outputs on technical blogs in the past few years, and has published many excellent Infra frameworks and underlying algorithms. But if you look closely, there seems to be a highly mismatched non-linear relationship between its technical outputs and the actual computing power investment. It has invested tens of thousands of GPU cards, but the outputs are often only partial optimized "small engineering victories", rather than underlying "major paradigm breakthroughs".

Why? Because the "research institutes" of large tech companies are in many cases still affiliates of business departments. They are required to "pursue both cutting-edge progress and practical implementation": they need to publish top conference papers to prove their technical strength, and at the same time be able to immediately access TikTok to do ad recommendation. This awkward state of "trying to have it both ways" prevents researchers from focusing on one direction for a long time.

But in the large model era, real Research requires "cost-free trial and error" and "long-term oriented tolerance".

Microsoft built a top-level supercomputing center for OpenAI, allowing them to iterate for several years before launching GPT-4; DeepSeek, backed by High-Flyer, has extremely free computing power redundancy and an extremely flat geek culture, so a small team of dozens of people can spare no effort to pursue their obsession with a MoE architecture. But in large tech companies, if the right to freely allocate computing power is not resolved, and an assessment mechanism that does not focus on short-term DAU is not established, top talents will only be reduced to senior assembly line workers cleaning data, or office politicians fighting with each other for GPU cards.

It is worth mentioning that public reports show that ByteDance has since promoted organizational convergence for the Seed team, and canceled the quarterly OKR and semi-annual assessment for the Seed Edge project. This indicates from the side that the old model of "horse racing + short-cycle assessment" is indeed unsustainable.

Conclusion: Say Goodbye to the "App Factory" and Remake the "Large Tech Company"

In fact, this problem is not exclusive to ByteDance. Alibaba, Tencent, Google, and all traditional giants that try to put the new wine of "large models" into the old bottle of the "App Factory" are going through painful growing pains.

A large model is not a mini-program that can be launched by writing a few lines of front-end code. It is the ultimate engineering for compressing human knowledge, and an extremely grand, cruel and highly unified nuclear fusion project. It requires enterprises to have underlying hardware-level scheduling capabilities, tolerate a long-term "no-output" phase, and more importantly, hold an ultimate belief that goes beyond short-term financial reports.

If large tech companies still cling to the old internet mindset of "internal horse racing", "forcing output through OKRs" and "each one sweeps the snow in front of his own door", even if they own tens of thousands of GPU cards, they will only end up building a "resource silo" that is not good enough to compete with top players and far less efficient than expected.

The winner in the future will not necessarily be the company that owns the most graphics cards, but will definitely be the organization that dares to break down old departmental walls, combine all computing power into one unified fist, and make top talents willing to spend a whole year cleaning data. After all, computing power can be bought with money, but building a "scientific research industrial system" that adapts to the evolution logic of silicon-based life requires a thorough self-revolution. Without changing the operating system, large tech companies are destined to never step through the narrow gate leading to AGI.

This article is from the WeChat official account "Tech Business", written by Tech Business, and published with authorization by 36Kr.