After DAU exceeded 100 million, Doubao is making Picasso "talk" | The Frontline

Recently, the Shanghai Pufa Art Museum has seen a denser flow of visitors than usual.

In the exhibition halls, some people hold up their phones to have real - time video calls with Doubao on the screen, listening to the interpretation of the patterns on the ceramics from the Louvre's collection; others take photos of Picasso's paintings and send them, and receive text replies with layered explanations within seconds.

When an AI application with over 100 million daily active users (DAU) meets 300 cross - continental artworks from the Louvre and 80 masterpieces by Picasso, a cross - border experiment on technological barriers, cultural empowerment, and user growth is taking place at the Shanghai Pufa Art Museum.

On January 20th, Doubao, under ByteDance, reached a cooperation agreement with the Shanghai Pufa Art Museum and became the official AI docent for the museum's two international exhibitions - "The Wonders of Pattern: Masterpieces of Indian, Iranian, and Ottoman Art from the Louvre" and "Picasso and Friends: A New Perspective by Paul Smith". Through exclusive data cooperation and targeted search optimization, the accuracy of Doubao's recognition and interpretation has been further improved.

This is the first time an AI product has become the official interpretation tool for an art museum.

As the official AI docent for the two major high - profile exhibitions at the end of the year at the Pufa Art Museum, Doubao's appearance stems from a profound industry insight: For a long time, art interpretation resources have always been scarce. In museums and art galleries, the number of professional docents is limited. A single interpretation session usually involves dozens of people listening together, which cannot meet individual needs. Moreover, interpretations from art history experts are even harder to come by. Facing art treasures spanning time and space, ordinary visitors often only have a superficial "window - shopping" experience. Cultural inclusiveness not only means getting more people into the exhibition halls but also enabling everyone to obtain high - quality art interpretations at an affordable cost.

The museum interpretation scenario has always been full of challenges. Docents not only need to distinguish highly similar exhibits such as the 15th - century Iranian "Peony - Patterned Plate" and the Ming Dynasty Yongle blue - and - white peony - patterned plate but also accurately identify niche treasures with limited public information. They also have to deal with complex situations such as multi - angle shooting, shaky images, and changing lighting when visitors are moving around the museum.

Through exclusive data cooperation with the Pufa Art Museum and targeted search optimization, and leveraging the visual reasoning and stable recognition capabilities of the Seed1.8 model, Doubao has successfully achieved "detective - like" accurate recognition. It can even respond accurately when visitors whisper questions, completely solving many pain points of traditional interpretation services.

The relevant project leader introduced that currently, Seed1.8 has reached the state - of - the - art (SOTA) level in video perception, streaming, and video understanding. It is no longer limited to the mode of "taking a picture, sending it, asking a question, taking another picture, and asking another question". Instead, it can continuously understand the constantly changing perspectives and scenarios in front of the visitors, thus achieving continuous, natural, and human - like dialogue interactions.

Zhu Jun, the vice - president of ByteDance, said that the interaction between AI and users is essentially a dialogue experience. During the museum visit, it is hoped that Doubao can mobilize the users' existing feelings and experiences through empathetic questions and heuristic dialogues, forming a more participatory understanding process.

During the museum visit, users can ask Doubao to interpret the artworks from multiple dimensions such as artistic style, historical background, creative techniques, and cultural significance. For example, when visitors are in the "Picasso and Friends" exhibition hall and facing Picasso's "Reading", they can ask Doubao, "How is the serene atmosphere in the picture created?"

Doubao will combine the specific background of the painting's creation in 1932, analyze how the soft curves and bright color blocks together create an intimate and serene reading scene, and elaborate on the characteristics of Picasso's creative style inspired by his muse, Marie - Thérèse Walter, and how he achieved a balance between figurativeness and deformation during this period.

Doubao's exploration in the art field has not been achieved overnight.

Previously, it had reached cooperation agreements with seven first - class national museums in China, including the National Museum of China and the Henan Museum. Through exclusive data cooperation and targeted search optimization, it has continuously improved the accuracy of recognition and interpretation.

For the two exhibitions at the Pufa Art Museum, Doubao has further optimized its algorithm model. It can not only accurately recognize 80 masterpieces by Picasso and 300 precious artworks from the Louvre's collection but also provide more context - specific interpretation services based on the exhibition space design and visitors' viewing habits. Whether it is real - time interaction during video calls or detailed text analysis after taking a picture to recognize an artwork, it aims to make art interpretation more accessible.

From museums to art galleries, from ancient cultural relics to modern art, Doubao's AI interpretation is building a more inclusive and accessible public cultural space. It not only solves the pain point of the scarcity of art interpretation resources but also redefines the relationship between art and the public - art is no longer an exclusive pursuit for a small group in the hall but a cultural nourishment that everyone can equally enjoy.

When technology breaks down the barriers to understanding and art interpretation becomes accessible, cultural inclusiveness has more concrete practices.