Google's Dreamer guru leaves the company and reveals missing out on Transformer

Just now, the great "Dreamer" Danijar Hafner announced his departure from Google, where he had worked for nearly a decade.

Before leaving, Danijar served as a Staff Research Scientist at Google DeepMind's San Francisco branch.

His research goal is to "build general agents that can understand and interact with the world."

As a leading expert in Google's world model, Danijar led/co-led the development of the Dreamer series (Dreamer, DreamerV3, Dreamer4, etc.).

Danijar Hafner

He wrote in a tweet, "Today is my last day at DeepMind."

Looking back on his nearly 10-year work experience at Google and DeepMind, Danijar believes that "an important chapter has come to an end."

In the early days at Google, Danijar mostly participated in the work of teams such as Google Research, DeepMind, and the Brain Team as a researcher.

From his educational background, his career development trajectory can also be clearly seen.

Since 2016, he has been participating in internships at the Brain Team.

According to Danijar's recollection, in 2016, he first participated in an internship at Google's Brain Team in Mountain View and worked with James Davidson and Vincent Vanhoucke.

Since there was no runnable PPO implementation at that time, the team conducted debugging and experiments under TensorFlow's static graph framework.

From 2017 to 2018, Danijar pursued a master's degree at the University of London. During this period, he worked at DeepMind in London.

From 2018 to 2023, Danijar pursued a doctorate at the University of Toronto. Therefore, he worked at the Brain Team in Toronto for a long time.

He co-developed multiple versions of Dreamer with another Staff Research Scientist at the team, Mohammad Norouzi.

First Encounter with Transformer

But "Didn't Pay Attention"

Danijar still remembers when Łukasz Kaiser first showed them a Wikipedia page generated by a "large" LSTM.

Łukasz Kaiser

Łukasz is a machine learning researcher. He initially worked at the Google Brain Team and later joined OpenAI.

The Google Brain Team published "Generating Wikipedia by Summarizing Long Sequences" in 2018. Łukasz Kaiser is one of the authors, and the theme is end-to-end generation of Wikipedia entries.

https://arxiv.org/abs/1801.10198?utm_source=chatgpt.com

Danijar also mentioned an episode where he "missed" the Transformer.

At that time, Ashish Vaswani excitedly introduced to him the advantages of a new architecture in computational efficiency and suggested that he apply it to reinforcement learning, but Danijar didn't pay attention at that time (I didn't try it then).

Later, this architecture was named Transformer.

Ashish Vaswani

https://arxiv.org/abs/1706.03762?utm_source=chatgpt.com

Ashish Vaswani is an influential scientist and entrepreneur in the field of AI.

He is the co-founder of Essential AI, but he is better known for co-authoring the famous groundbreaking paper "Attention Is All You Need."

This paper proposed a new and simple network architecture, Transformer, which "completely abandons cyclic and convolutional structures and only uses the attention mechanism," enabling stronger parallelism and faster training speed.

The aforementioned Łukasz Kaiser is also one of the main authors of this groundbreaking paper.

Deep Learning Research

From 2017 to 2018, Danijar pursued a master's degree at the University of London.

During this time, he worked at DeepMind in London and had the opportunity to participate in research on deep reinforcement learning and generative models, and collaborated with Timothy Lillicrap (DeepMind) and Ian Fischer (Google Research).

This experience led to their joint completion of PlaNet (a model-based reinforcement learning algorithm that can learn a latent world model from pixel observations and perform planning and action selection in the latent space).

Timothy Lillicrap

Subsequently, Danijar had long-term exchanges with Nicolas Heess and the Adaptive Agents team at DeepMind, systematically sorted out, and practiced Karl Friston's Free Energy Principle (FEP).

Developing Dreamer

Once Guided by Hinton

From 2018 to 2023, Danijar pursued a doctorate at the University of Toronto. During this period, he also conducted research as a jointly-trained doctoral student at the University of California, Berkeley.

During his doctoral studies at the University of Toronto, Danijar's main supervisor was Jimmy Ba, and he also received guidance from Jeffrey Hinton.

During this time, Danijar also worked at the Brain Team in Toronto for a long time.