Just after leaving Kimi, 17-year-old Chen Guangyu went to Jeff Dean's new company.
As learned from Synced, 17-year-old AI researcher Chen Guangyu has joined the Discovery Loop team founded by Jeff Dean and other researchers.
On October 15, Chen Guangyu announced his departure from Moonshot AI on X, stating that the past year has been the best time so far, and thanked his former colleagues, especially Du Yulun, who was willing to hire him when he was 16.
Regarding his next move, he only left one sentence: "There are more things worth exploring."
After the announcement of his departure, Sholto Douglas, head of scalable reinforcement learning at Anthropic, interacted with him.
Douglas said that the two met the week before, asked if he still planned to go to Carnegie Mellon University, and invited him to work together in San Francisco.
Chen Guangyu replied: "Thank you Sholto! I'm heading to the Bay Area."
This interaction once led the public to speculate that he would join Anthropic, but Chen Guangyu later denied the claim with a "no :)".
Joined Kimi at 16 and participated in the R&D of trillion-parameter models
When Chen Guangyu joined Moonshot AI, he was a senior high school student.
During his time at Kimi, he participated in the R&D of multiple core technologies, including optimizing the calculation kernel of Kimi Delta Attention (KDA), researching Attention Residuals, and exploring optimizers for large model training.
This March, Chen Guangyu, together with Zhang Yu and Su Jianlin, published the paper Attention Residuals as co-first authors.
Paper link: https://arxiv.org/pdf/2603.15031
This study improves the residual connection mechanism of Transformer, enabling the model to selectively use information from different previous layers and improving information transmission in deep networks.
After the paper was released, Elon Musk reposted it and commented that it was "impressive".
In July, Moonshot AI released Kimi K3 with a total parameter count of 2.8 trillion. Attention Residuals became one of its core architecture technologies, and Chen Guangyu also participated in the case production before the model was released.
In terms of personal research directions, Chen Guangyu focuses on model architectures, optimizers, as well as efficient algorithms and computing kernels adapted to hardware characteristics.
Before joining Kimi, he was also engaged in applied model interpretability research at Tilde Research.
In addition, he listed some concepts he adheres to on his personal website:
- The ability to act independently is a superpower.
- Taste can be cultivated.
- Write down your ideas.
- With friends, solutions to any problem can be found.
- You can explore and transform the world in a positive and beneficial way.
- Don't get stuck in local optima.
Interestingly, he wrote in the "Things I Like" section: "Learning from my teacher Gemini (this sentence seems a bit out of place now)".
Next stop: Discovery Loop, valued at 50 billion USD
Discovery Loop was founded in August this year by four former core Google researchers: Jeff Dean, Sanjay Ghemawat, Quoc Le and Oriol Vinyals.
Among them, Jeff Dean once served as Google's Chief Scientist and participated in leading the R&D of Gemini. Oriol Vinyals is also an important technical lead of the Gemini project.
The research goal of the new company is to use AI to accelerate the discovery process in the fields of machine learning, science and engineering.
Specifically, Discovery Loop hopes that AI can automatically propose hypotheses, design and run experiments, analyze results, and carry out the next round of research based on feedback, forming a continuously iterated automated experimental system.
The company plans to first promote the automation of machine learning research, and then expand to other scientific and engineering fields.
On September 11, according to a report by Business Insider, Discovery Loop is seeking a new round of financing with a target valuation of about 50 billion USD.
Chen Guangyu, who has just stepped down from the front line of large model training, will have the opportunity to cooperate with former core R&D personnel of Gemini to continue to explore the automation of AI research.
References:
https://x.com/ArfurGrok/status/2112332568616120406
This article is from the WeChat Official Account "Synced" (ID: almosthuman2014), author: Niu Lai, published with authorization from 36Kr.