Tencent's Yao Shunyu team responded to all relevant issues late at night: insufficient computing power, models gaming the benchmark rankings, and the profound strategic thinking on Leiting.
Report from Zhidx on November 21, in the early hours of today, Tencent Hunyuan Team held an AMA event on the social media platform Reddit, communicating with global developers around the Hy4 series models. Shunyu Yao also appeared in the comment section, though we have no idea which account is his alternate identity.
▲ Shunyu Yao announced this Reddit AMA event on X (Source: X)
This AMA is taking place at a very interesting point in time.
Just yesterday, Tencent just opened the weights of its flagship model Hy4 and released the model security assessment card (System Card). A few days earlier, Tencent also launched Hy4 flash, which focuses on low cost and high efficiency. The release of two consecutive models has made Tencent Hunyuan's technical route the focus of the community.
At the same time, competition in the AI industry is also changing. Recently, the third-party model evaluation organization Artificial Analysis has significantly adjusted its evaluation system, leading to score fluctuations of some Chinese models. Against the backdrop of limited access to high-end computing power and continuous iteration of cutting-edge global models, how Chinese models can narrow the gap with top-tier models has also become a concern for developers.
Zhidx also participated in this AMA and raised a question to the Tencent Hunyuan Team: As access to advanced AI chips is increasingly restricted, can Tencent's Hy models continue to narrow the gap with the world's cutting-edge models in the future?
A member of the Tencent Hunyuan Team responded: "We are making the fullest possible use of existing resources to catch up with the cutting-edge level."
▲ The Hunyuan Team answers the question raised by Zhidx (Source: Zhidx)
In addition to the computing power issue, the Tencent Hunyuan Team also revealed a lot of new information during this exchange.
In terms of model scale, the Hunyuan Team clearly stated that the next-generation Hy model will further expand the parameter scale, while continuing to invest in large-scale reinforcement learning.
For smaller models, the team revealed that products such as Flash-Lite and models for devices with 24GB video memory are under planning.
At the technical level, Tencent also disclosed some architecture selection details of Hy4 for the first time in this exchange. The team repeatedly weighed between the two attention mechanisms of GQA and MLA, and finally decided to adopt the MLA architecture because DSA reduced the computational overhead of MLA in the decoding stage.
In addition, the Tencent Hy Team also responded to questions such as model evaluation, security restrictions, AI programming ecosystem, and whether the model can achieve recursive self-improvement (RSI).
Hy4 is Tencent's latest generation flagship model, with enhanced capabilities in long-range software engineering, tool invocation, visual understanding, and scientific research. Hy4 flash further optimizes inference cost, with a total parameter count of 224B, including 64B Engram parameters, and only 12B parameters are activated per Token.
According to the data disclosed by Tencent, in the self-built daily work task evaluation WorkIndex, the single-task cost of Hy4 flash is only 1/20 of that of Hy4. In a blind test involving 85 internal users and 259 work tasks, its single-task cost is about 0.26 yuan.
However, Hy4 still lags behind the world's top closed-source models in some highly difficult tasks. How to continuously improve model capabilities under limited computing power, while taking into account inference cost, product experience and open source ecosystem, is still a problem that the Hunyuan Team needs to face.
The following are the key contents sorted out by Zhidx from this AMA. For the convenience of reading, we have made adjustments that do not change the original meaning:
Full Q&A address:
https://www.reddit.com/r/LocalLLaMA/comments/1y15w4a/tencent_hy_team_shunyu_yao_here_ask_us_anything/
01 .
Catch up with the cutting edge with limited computing power
The next-generation model will be greatly expanded in scale
Small-sized models are on the way
(1) Question from Zhidx: As access to advanced AI chips is increasingly restricted, what is the biggest bottleneck Tencent Hy is currently facing? Is it computing power, training efficiency, or algorithm? Can the future Hy series models continue to narrow the gap with the world's top models?
Tencent Hunyuan: We are making the fullest possible use of existing resources to catch up with the cutting-edge level.
(2) Question from netizen: Is Hy4.5 planned to expand the model parameter scale to 1.5 trillion to 2 trillion in the future?
Tencent Hunyuan: The next-generation Hy model will definitely further expand its scale significantly.
(3) Question from netizen: Do you think the current Hy series models have approached the upper limit of capabilities? For future more powerful models, is it necessary to increase the parameter count, or can we maintain the current scale and further improve capabilities through post-training?
Tencent Hunyuan: We believe that Hy4 still has a lot of room for improvement, especially in post-training. We are vigorously investing in large-scale reinforcement learning (RL), while continuously improving user experience and model security. Stay tuned for subsequent updates. We also plan to train larger-scale models. We firmly believe in the importance of Scaling and will continue to adhere to this direction.
(4) Question from netizen: Companies such as ByteDance and Kimi seem to be moving towards super-large-scale models with 3T or even 5T parameters, while Tencent Hy series is still at a scale of around 700B. Why did you choose such a route? How do you view the impact of parameter count on the intelligence level of the model? Do you plan to further expand the model scale in the future?
Tencent Hunyuan: Hy3 and Hy4 have proved that smaller but well-trained models can often outperform larger models. More importantly, smaller models have lower costs, and we can learn more and accumulate more experience during the R&D process.
(5) Questions integrated from multiple netizens: Will Tencent launch Hy4-mini and dense models with a scale of 24B-31B in the future? Is there a plan to launch small-sized models that can run on devices with 24GB video memory and 64GB-128GB memory? What are your plans for models with parameters below 100B that are specially optimized for programming tasks?
Tencent Hunyuan: Hy4-mini is already on the development roadmap, and more small models will be launched in the future. For dense models with a scale of 24B-31B, the team is considering it; for smaller dense models, it is possible to launch them, but currently we may not specially develop models with parameters below 100B optimized for programming tasks.
In terms of local deployment, the Hunyuan Team confirmed that it will launch models for devices with 24GB video memory in the future, and will continue to develop models suitable for running on devices with 64GB-128GB memory. The team also gave a positive answer to the open-weight models that are similar in scale to the current models and are suitable for running on devices with 128GB memory after Q4 quantization.
In addition, Hy4 flash will continue to be updated, and the team is optimizing the pre-training and post-training schemes for Flash-level models. As for the positioning of Hy4 and Hy4 flash, the team said that both models are general-purpose models, not specially trained for programming tasks.
(6) Questions integrated from multiple netizens: In addition to language models, will Tencent launch smaller open-source image models, upgrade video generation and translation models? Can Hy4 support dialect recognition and real-time speech translation? What progress has been made in continuous learning and Artificial General Intelligence (AGI)?
Tencent Hunyuan: For image generation, the team said it is considering launching smaller image models, and suggested that developers pay attention to the latest developments of video models recently, but did not disclose the specific release time.
In terms of translation, the team confirmed that the Hy-MT model is about to be upgraded, and said it will seriously consider the user's suggestion on translating Chinese online novels.
Regarding dialect recognition and real-time speech translation, the Hunyuan Team frankly admitted that Hy4 does not currently support audio understanding or audio generation, but users can try the speech recognition (ASR) and speech synthesis (TTS) functions provided by Tencent Yuanbao, which may meet relevant needs.
Regarding continuous learning, the team believes that it is a very promising research direction, and is actively carrying out relevant research, but no specific results have been disclosed.
In addition, a netizen cited the AGI capability grading system and asked what level Hy4 has currently reached. A member of the Hunyuan Team gave a personal judgment — "It is about level 2.5."
▲ The AGI capability grading system proposed by Google DeepMind (Source: Reddit)
02 .
Repeated trade-offs were made for architecture selection
MoE is not necessarily better when it is sparser
(7) Question from netizen: Does Tencent plan to launch a small-sized Flash-Lite model for low-latency tasks? What are the advantages of iHC compared with mHC and Gated Residual (GR)? Why did Hy4 flash choose Sliding Window Attention (SWA), and will other attention mechanisms such as QSA be tried in the future?
In addition, will the license of the old version of the open-weight model on Hugging Face also be updated to support users in the European Economic Area (EEA)?
Tencent Hunyuan: Great, there are a lot of technical questions.
First, yes, stay tuned for an advanced Flash-Lite model.
Second, compared with mHC and GR, iHC is much simpler and can achieve similar performance at lower cost.
Third, SWA can reduce the KV Cache size by 4 times, thus significantly reducing the inference service cost.
Fourth, of course! We are open to all kinds of architectural innovations.
(8) Question from netizen: What is it like to participate in the R&D of open-weight models and contribute to the open source community?
Tencent Hunyuan: Contributing to the open source community is a very rewarding thing, especially when seeing so many people paying attention to our work.
(9) Question from netizen: What was the most difficult moment you personally experienced during the R&D process?
Tencent Hunyuan: For me, the most difficult part was the process of scaling from Hy3 to Hy4.
We did not carry out enough detailed ablation experiments on the training data to the extent I hoped, so there was still a lot of uncertainty when we started training. To be honest, I was very nervous when the training just started.
Fortunately, the final training process went unexpectedly smoothly. It felt like we took a desperate gamble and actually won.
(9) Question from netizen: If you could make one wish, what would you want future large language models to do for you?
Tencent Hunyuan: As a total food lover, I hope that future large language models can create brand new recipes, and really ensure that the taste made completely meets my preferences.
(10) Question from netizen: Apart from the finally adopted Hy4 architecture, what is the most competitive alternative architecture that Tencent Hy Team has seriously considered but finally abandoned? Why was it abandoned?
Tencent Hunyuan: We previously weighed repeatedly between GQA and MLA.
The computational intensity of MLA is significantly higher than that of GQA, which leads to the fact that computational power becomes the performance bottleneck when performing Speculative Decoding on inference GPUs.
However, with the same KV Cache size, MLA can provide better model quality than GQA.
Fortunately, with the introduction of DSA, we were able to reduce the computational intensity of MLA in the decoding stage. This finally changed the trade-off between the two architectures and made us decide to adopt MLA.
(11) Question from netizen: When evolving from the original Hy4 architecture to Hy4 flash, how did you decide which network layers or architectural components to retain? What is the main basis for every decision to adjust the neural network layers?
Tencent Hunyuan: Hy4 flash inherits many technologies of Hy4, including iHC, gated MLA and DSA, while introducing new architectural components such as hybrid SWA, LatentMoE and Engram.
Behind these design choices, there are a large number of results accumulated by the team in Scaling experiments of different model scales, as well as the co-design of model architecture and inference system.
(12) Question from netizen: How important is additional parameter density to the intelligence level of the model? Have you tested adjusting Hy4 flash to 156B total parameters with 8B or 5B activated parameters? How much will the model quality drop?
Tencent Hunyuan: We have carried out Scaling experiments with different MoE sparsity, similar to the research in the MAI-Thinking-1 paper, and reached roughly the same conclusion.
When the number of activated parameters is fixed, increasing sparsity can improve computational efficiency. Therefore, this technical route is effective.
However, 156B-A8B or A5B is a completely different trade-off. This approach reduces the number of activated parameters while keeping the total number of parameters unchanged. I expect this will significantly damage the model quality.
The number of activated parameters essentially determines the computational budget of the model. Increasing sparsity can recover part of the computational efficiency, but cannot fully make up for the impact of reducing activated parameters.
In the final analysis, reducing the number of activated parameters means less computational investment and a weaker model.
(13) Question from netizen