AI vision field star "Luma AI" completed a ten-million-dollar financing round, with investments from Amazon and AMD. | Exclusive from 36Kr.
Written by Zhou Xinyu
Edited by Su Jianxun
"Intelligent Emergence" has learned that "Luma AI", an AI vision company in the Silicon Valley, USA, has recently completed a new round of financing, with an amount of tens of millions of US dollars.
The investment lineup of this round includes four European and American companies or funds: Amazon, AMD, Factorial Funds, and LDV Capital. At the same time, the old shareholders A16Z, Amplify Partners, and Matrix Partners continue to increase their investments.
It is understood that this round of financing is mainly used to accelerate the development of visual artificial intelligence basic models and products.
Founded in 2021, Luma AI is a technology company focused on computer vision content. Its self-developed models cover video generation, 3D generation, and image generation. In January 2024, "Intelligent Emergence" reported that Luma AI completed a $43 million Series B financing, with the investor being A16Z.
Globally, the resource allocation in the AI track has entered the "midfield". According to the statistics of the technology media Techcrunch, the number of monthly financings of over 100 million US dollars in the second half of 2024 is 10% less than that in the first half. At the same time, hot money is concentrating on the AI application layer, especially in the fields of AI search, AI sales, robotics, and AI programming.
The model layer is the infrastructure. The AI model layer cannot become a product alone, and the final traffic needs to be undertaken by AI applications - this consensus has been formed among both investors and AI practitioners.
On November 26, 2024, Luma AI, which mainly focuses on the model layer, also released the first AI application product, Dream Machine AI Creative Platform, after the video generation model Dream Machine became popular.
"Compared with language models such as ChatGPT, the video model is still a relatively niche field." Jiacheng Yang, the product designer of Luma AI, found that the users of Dream Machine are mainly professionals with AI or film and television production experience. He explained to "Intelligent Emergence" the reason for releasing the AI creative platform focusing on image design:
"Compared with video generation, the user base in the image field is larger, which is conducive to expanding our user base. Our goal is to create an AI vision tool that both AI beginners and design beginners can easily use."
The Dream Machine AI Creative Platform can be understood as a design platform that integrates functions such as text-to-image design, AI brainstorming, subject/style reference, and design drawing to video conversion.
The subject/style reference function of the Dream Machine AI Creative Platform. Source: Luma AI
Compared with text-to-image products such as Midjourney and Stable Diffusion, the Dream Machine AI Creative Platform has a stronger understanding ability of natural language prompts, and can also generate higher-definition and design-sense captions in images.
High-definition captions generated by the Dream Machine AI Creative Platform. Source: Luma AI
The reason why the Dream Machine AI Creative Platform is easy to use and has strong performance still lies in the underlying model capabilities. Currently, the platform's language understanding ability comes from the Agent built by Luma AI based on a third-party language model; the image generation ability comes from Luma AI's self-developed image generation model Luma Photon; and the image-to-video ability comes from the self-developed video generation model Dream Machine released on June 16, 2024.
At that time, video generation models such as Sora and Vidu of Shengshu Technology only stayed at the demo release stage and were not publicly tested. Dream Machine became popular on social platforms for a time by taking the lead in being "free" and "publicly tested", as well as having good performance and the gameplay of "meme images".
Within 4 days of its launch, the number of users of Dream Machine exceeded 1 million. At the same time, Barkley Dai, the head of data products at Luma AI, told "Intelligent Emergence" that the promotion cost of Dream Machine is 0.
Currently, the Luma AI team has about 50 people. According to Barkley, after deciding to start the video generation project in December 2023, the team size expanded from 10 to 50 people, mainly introducing top talents in the video generation field.
The effect of high talent density operations is reflected in the performance of Dream Machine. Currently, Dream Machine can generate a 5-second video in about 20 seconds. At the same time, extremely realistic camera movement trajectories, natural light and shadow changes, and a rich camera position are the characteristics of Dream Machine. In the 1.6 version released in September 2024, users only need to input a text prompt to adjust the movement direction of the camera.
At the same time, Luma AI, which started with 3D generation technology, also has the Text to 3D tool Genie. At that time, Genie was the only tool on the market that could generate a 3D model within 10 seconds.
In terms of commercialization, on the one hand, the model products in the video, image, and 3D fields of Luma AI provide APIs to the outside world; on the other hand, application layer products such as the Dream Machine AI Creative Platform will adopt a charging model of limited free + paid subscription.
Currently, Luma AI has also become a rare AI startup that has a comprehensive layout in the multi-modal fields of video, image, and 3D. In a public interview, Jiaming Song, the chief scientist of Luma AI, mentioned that the amount of Tokens required for multi-modal model training is much larger than that of language, and the Scaling Law of multi-modality can enable the model to better understand the world.
Extended Reading:
Welcome to follow!