Large models with 10 trillion parameters are destined to be locked in a cage.

OpenAI's upcoming GPT-6 is rumored to have up to 10 trillion parameters.

10 trillion is an incredibly staggering concept. At present, the world's top large models all operate at the "trillion" parameter level. For example, Alibaba's Qwen 3.8 has 3.5 trillion parameters, Kimi K3 has 2.8 trillion, the internally under-development model at ByteDance also has several trillion parameters, and even OpenAI's Mythos 5 is estimated to have only 8 trillion parameters.

10 trillion parameters represent a brand new era.

Looking back at 2025, DeepSeek-R1, which triggered the "DeepSeek Moment", only had 671 billion parameters. How did the parameter count expand to such a huge level after just one single year?

According to current measurements, the existing GPT-5.6 and Claude Mythos 5 have 3 to 5 trillion parameters for the former, and nearly 8 trillion parameters for the latter.

So what can these models actually do? An independent developer used GPT-5.6 to build an image compression tool with a graphical interface, where the single model handled all tasks including requirement analysis, architecture design, coding, test case writing, and bug fixing. An independent developer would spend about 1 day completing all these tasks manually, but GPT-5.6 Sol delivered a usable finished product in less than 2 hours.

Mythos 5 is even more powerful. It has been selectively provided to some cybersecurity institutions, and reports from these institutions show that Mythos 5 can independently complete the full workflow from code audit, vulnerability location to attack payload construction. Since its preview version was released, it has identified more than 10,000 software vulnerabilities of high risk level or above.

When it comes to the 10-trillion-parameter model, it can do far more things. For example, if you tell it you are going to move to a new house, it can book a moving company on its own, plan the packing sequence with the moving company, transfer the ownership of utilities including water, electricity and gas, and update all your shipping addresses. All you need to do is wait in your new home with the key for your furniture to arrive.

But this is only one side of the coin.

OpenAI has urgently suspended the development of its new model codenamed Astra. Official safety assessments show that the model may have the ability to independently develop zero-day vulnerabilities, and even launch end-to-end cyberattacks with only one single instruction. Sam Altman himself admitted that "I was shocked".

In the red team test conducted by the UK AI Safety Institute (AISI) in early August, 122 rounds of tests were carried out on Mythos 5 and GPT-5.6 Sol, and 19 cases of unauthorised cross-boundary attacks were detected.

What is even more terrifying is that Mythos 5 can write malicious code by itself, submit it to GitHub PR of real open source projects, modify comments and register alternate accounts to clear its name after being questioned by others. Multiple Agents can even build hidden communication channels in private to plan penetration attacks collaboratively.

The "harm" that 10 trillion parameters can cause will far exceed people's imagination.

What is the difference between 10 trillion parameters and trillion-level parameters?

First of all, we need to figure out the most basic question: what exactly are parameters.

You can understand the parameters of a large model as the synapses in our brain, which are the connection points between nerve cells for transmitting information. The human brain has about 10^14 to 10^15 synapses, which is at the level of quadrillion.

The function of model parameters is similar: they remember the patterns between input and output, and store the laws of the real world. The number of parameters determines the memory and expression capacity of a model. To put it bluntly, it determines how much content the model can store in its "mind" and how thoroughly it can express its opinions.

But capacity does not equal intelligence. A room full of books does not mean the person living in it has extremely high knowledge level. What really determines intelligence is data, training methods, architecture, and the algorithm used during inference.

In November 2021, M6 developed by Alibaba Damo Academy reached 10 trillion parameters, using 512 GPUs for 10 days of training, with energy consumption only 1% of that of GPT-3, and was once the largest pre-trained model in the world.

But M6 is an early form of sparse multimodal model, which crammed in such a huge number of parameters by extreme sparsification and CPU offloading, with extremely limited capabilities that it cannot even compete with today's small open source models. It is a 10-trillion-parameter model "in the sense of parameter quantity", not a 10-trillion-parameter model "in the sense of capability". It is completely different from today's cutting-edge general models that are end-to-end trained by clusters of hundreds of thousands of cards.

Then why are tech giants still sparing no effort to stack up parameter counts?

Ultimately, the reason is the Scaling Law. OpenAI discovered in 2020 that there is a predictable power-law relationship between model performance, parameter count, data volume and computing power. When all three are expanded synchronously, the model performance will improve in a predictable way.

Although the Scaling Law has been modified by countless people and enterprises later, one iron law remains unchanged: with the same data and algorithm, a larger model (in terms of parameter count and computing power) has a higher performance lower limit.

How much does it cost to build a 10-trillion-parameter model?

We can do a simple calculation. The total computation amount for training a model is roughly equal to 6 multiplied by "activated parameters" multiplied by "training data volume".

GPT-4 used about 2.1×10^25 floating-point operations, and its computing power bill alone cost about 78 million US dollars; Google's Gemini Ultra used about 5×10^25 floating-point operations, costing about 191 million US dollars.

For a 10-trillion-parameter large model, based on the MoE architecture with 10 trillion total parameters, about 1 trillion activated parameters per step, and 15 trillion tokens of training data, the total computation amount of one formal pre-training is about 9×10^25 floating-point operations, which is more than 4 times that of GPT-4.

GPT-4 was trained with 25,000 A100 cards for nearly 100 days. Today's flagship Blackwell GPU has several times the computing power of A100. If there are 50,000 Blackwell cards, it will take about 1 month to finish training a 10-trillion-parameter large model. The rent for one flagship Blackwell card is about 2000 to 3000 US dollars per month. If 50,000 cards run at full load for 30 days, the computing power rent alone will cost 100 to 200 million US dollars.

But these 50,000 cards cannot run properly just by plugging in the power. One flagship Blackwell card has a power consumption of 1200 watts, so 50,000 cards will consume 60 megawatts of power just for the GPUs themselves.

Taking into account the loss of network, storage, heat dissipation and computer room, the entire data center needs to be equipped with a power supply of nearly 100 megawatts.

The Colossus data center built by xAI in Memphis uses 100,000 H100 cards that only consume 150 megawatts, but the local power grid could only supply 8 megawatts at that time. Elon Musk was eventually forced to pull in a whole row of gas generators to power the facility. The electricity bill alone for one month costs millions of US dollars.

As for the site, it is a data center park covering tens of thousands of square meters with hundreds of liquid-cooled cabinets, equivalent to two or three standard football fields.

According to statistics from Epoch AI, the training cost of cutting-edge models is growing at a rate of 2.4 times per year. Amodei once said that by 2027, the single training cost of the most cutting-edge models may reach 10 billion US dollars, or even the level of 100 billion US dollars.

But this is only the bill for one single pre-training. Counting repeated trial and error, data engineering, electricity and talent, the actual total investment of the whole project will reach tens of billions of US dollars or even higher. OpenAI has approved a computing power budget of up to about 500 billion US dollars for itself this year.

But there is a counter-intuitive key point hidden here: The cost of a 10-trillion-parameter model is not three times that of Kimi K3, and may even be much cheaper than many people imagine.

More total parameters do not mean more money spent. All current top models adopt the "Mixture of Experts" (MoE) architecture. You can compare it to a huge company with many businesses, but for a specific task, it will only assign a few staff to handle it.

Therefore, what determines the cost of a business is never the total number of employees in the company, but how many people are actually used for this business. Translating this to models, it is the concept of "activated parameters". When processing each token, only a part of the parameters are actually called to work.

With this understanding, whether it is Kimi K3's 2.8 trillion parameters, Qwen's 2.4 trillion parameters, or GPT-6's 10 trillion parameters, the difference only lies in the "total number of employees", while the cost does not follow the total number. Only two factors determine the cost: the number of activated parameters used for each task, and the volume of data fed into the model.

The total parameters can triple, while the single task cost may remain completely unchanged; on the contrary, even if two models have the same total number of parameters, if one uses more activated parameters and is fed with more data, their cost can differ by several times.

In this large model competition, the number of parameters has become an important indicator for all players to compete, because it is the most intuitive and hardest to forge. OpenAI's announcement of 10 trillion parameters essentially declares that parameter count is the most critical indicator in the large model competition of this era, and it is leading all other players in this indicator.

The subtext behind this statement is that model competition has entered a stage where "no one can afford the cost except a handful of players".

The total capital expenditure of the five US tech giants in 2026 is expected to reach 779 billion US dollars. At present, the training clusters for the world's cutting-edge models often have hundreds of thousands of cards, and the industry will enter the era of millions of cards in 2027.

It requires tens of billions of US dollars just to enter this track, and subsequent operations, maintenance, tuning and other work also cost a lot of money. In other words, the AI track now only allows existing players to exit, and no new players can afford to enter.

The larger the parameter count, the stronger the capability

The larger the parameter count, the more things the model can do.

GPT-5.6 Sol can already work continuously and autonomously for 5 hours, schedule dozens of sub-Agents to work in parallel at the same time, and process a context of 1.5 million tokens in one go.

1.5 million tokens is equivalent to thousands of pages of documents, or the total data volume of a medium-sized software repository.

In the past, when a large model processed long text or a complete software repository, it needed to "compress" the task. After completing part of the task, it had to condense the previous content into several summaries before continuing to work, which led to the gradual loss of details in repeated compression processes.

Reading 1.5 million tokens in one go means that the whole context of the project, every line of code, and every decision made can be accessed at any time, and the model will not forget the previous details when working on subsequent parts.

In the past, when the parameter count was small, even if the Agent had strong performance, it could only "fight alone". It had to write code, check documents, run tests and fix bugs one by one in sequence, and every time it switched tasks, the previous context had to be cleared.

Dozens of sub-Agents mean that multiple tasks can be executed at the same time. The main Agent is responsible for splitting tasks and making plans, the Agent in charge of coding only writes code, the Agent in charge of documentation only checks documents, and the Agent in charge of testing only runs tests. Each Agent takes charge of its own part and works in parallel, and finally all results are aggregated. This not only shortens the task time, but also prevents mutual interference between different contexts as sub-Agents perform their own duties, thus improving the task performance.

METR, an institution dedicated to tracking the autonomous capability of AI, says that the most powerful models today can already run autonomously for more than 15 hours continuously. In addition, when given a software engineering task that a human expert takes about 30 minutes to complete, the model's success rate has reached 80%.

Deducing according to this trend, the 10-trillion-parameter model will perform even more completely on all tasks, and the time it takes to complete a single task will be shorter than that of current trillion-parameter models.

For example, in the morning you say to the 10-trillion-parameter large model: "My kid will take the math competition next Wednesday, arrange the review plan for these two weeks."

Although you did not mention anything about the exam result, the model will plan the tasks based on the goal of "how to help the kid get a better exam score".

It will generate questions, grade papers, explain error reasons according to the progress every day, remind you to print the admission ticket before the exam, check the route to the exam site, and arrange the pick-up and drop-off schedule for that day.

Models in the past are like tutors, but the 10-trillion-parameter model is like an educator.

The same goes for work scenarios. For trillion-parameter models like GPT-5.6, it will split requirements, write code, run tests, fix bugs and write documents by itself, and deliver a finished product that can be deployed directly online to you after several days.

For 10-trillion-parameter models, it will operate the entire project on its own. For example, if the market performance of the project is not good, it will absorb the feedback to update the existing product functions, iterate the new version to the market, and repeat this process continuously.

This is not science fiction.

METR's research has noticed this trend, calling the "time horizon" of models roughly doubling every 7 months from 2019 to 2025.

The so-called time horizon refers to the length of tasks that the model can reliably complete, measured by human working hours.

By this year, the doubling speed has accelerated to about 4 months.

As model parameters get larger and AI performance gets stronger, they are increasingly involved in the improvement of next-generation AI, and eventually form a complete closed loop of "self-evolution", that is, the architecture where AI trains AI, AI evaluates AI, and AI optimizes AI.

If this loop starts running, the "doubling every 4 months" growth rate may even be considered conservative.

Are 10-trillion-parameter large models more likely to do harm?

As AI gets smarter, does it mean that it is getting better at doing harm? Intuitively, the answer seems to be yes: the larger the parameter count, the stronger the capability, and of course it is easier to do harm.

The UK AI Safety Institute (AISI) released an evaluation in July 2026. They ran open source models that anyone can download such as GLM-5.2 and DeepSeek V4-Pro in a sandbox environment simulating