The data annotation business of AI music
To learn music, you must first understand rhythm, chords and instruments. The same applies to AI.
In October, an AI-generated song "Veteran" became a smash hit. A number of historical and mythological figures including Du Fu, Su Shi and Liu Bang talked about housing loans, workplace struggles and midlife dilemmas in rap, which struck a chord with many people. Some people said bluntly that it was obviously AI-generated, making them feel extremely uncomfortable while listening.
AI has also entered the professional music creation stage. The latest "Music Bond Project 3" on Jiangsu Satellite TV focuses on original works, and its selling point is the music stories behind these original creations. However, in the first episode, a musician confessed that he used AI in his work, but guaranteed that the lyrics were created by himself.
After listening to the song, Joker Xue said: "The lyrics are totally understandable to me, and they also give me a sense of deja vu." Many audiences believe that the lyrics are fused from several songs of Joker Xue.
The song itself is not unpleasant to hear. The controversy among the public is that in a program that highlights the value of original creation, a song whose creation process relies heavily on AI appears, so why can it compete on the same stage with other original works?
Many people are also curious about how AI learns to write songs. Wang Kunzhe, the creator of "Veteran", did not hand over the entire creation process to AI. He first built the skeleton of the lyrics, then let AI try different combinations, generated multiple music versions, and then filtered, edited and adjusted the arrangement by himself.
AI can generate music, but that does not mean it truly understands music. To make the model learn to identify the structure of a song and judge the quality of the generated results, a group of people who understand music are needed behind the scenes.
They are not responsible for writing songs, they act as music teachers for AI. This group of people are AI music annotators.
Who is teaching AI to listen to music?
In 2018, it was 4 years before the birth of Suno, and much earlier than the time when AI-generated music became a hot topic.
Liu Xiaoguang had already started thinking about one thing: how to make machines understand music.
This entrepreneurial direction is a bit counter-intuitive. The most attractive stories in AI entrepreneurship are usually about letting machines create something. What Liu Xiaoguang did is exactly the opposite: he wants to teach machines to recognize music first.
Liu Xiaoguang obtained his bachelor's, master's and doctor's degrees from the Department of Chemistry of Tsinghua University, and he is also a music producer. The company he founded, DeepMusic, first targeted music data processing.
Hubei Daily recently reported on Yaosheng Digital Intelligence, a music data service enterprise in Wuhan. Huang Binyao, the person in charge, introduced that annotators need to listen to the tonality and speed first, judge the harmony trend, then use tools to measure the beat, cut the song into different sections, and identify the instruments section by section.
For example, one section uses violin, another section has mixed timbres of brass and woodwinds, and the background is also equipped with string music. Whether the lyrics are sung wrong, whether the instruments are used correctly, and whether the rhythm matches can all become information that needs to be marked.
This job is music annotation.
Strictly speaking, this is a set of professional positions generated around AI music training and evaluation, and its boundary is not completely fixed.
Long before the emergence of generative AI, music platforms needed to tag a huge number of songs. The classifications such as pop, sad and rock we see in music software are all supported by the sorting of music data. But these tags mainly serve for song search, classification and recommendation.
After the rise of generative music tools such as Suno and Udio, the things that models need to learn have become more complex: in addition to knowing what style a song belongs to, they also need to understand its chords, rhythm, orchestration and structure, and even judge whether there is any problem with the music they generate.
As a result, music annotation has gradually developed several different types of work.
Some people are responsible for disassembling basic information; some people specifically judge whether the melody, lyrics and arrangement are reasonable; others are responsible for classifying errors and sorting them into feedback that can be used by the model R&D team. Someone needs to tell AI exactly what is wrong with the music it generates.
Overseas, the US AI talent platform micro1 has also begun to recruit music teachers, performers and producers to participate in model training, with the hourly salary of the positions marked at 30 to 45 US dollars.
In addition to high salaries, for such positions in China at present, the monthly salary of basic annotation and internship positions at several thousand yuan is the common level.
From selling labor to selling data, who is making money from AI?
In 2023, Liu Xiaoguang mentioned that large enterprises purchased data from them when doing music generation and music recognition.
Tencent Music is an important strategic investor of DeepMusic. DeepMusic connects its AI arrangement technology to WeSing and launches the function of AI style conversion. At that time, WeSing users used AI accompaniments more than 300 million times. Now, DeepMusic's official website lists this as an important technology implementation case; WeSing's AI layout continues to expand.
Behind music annotation, several different types of businesses have also developed.
The first type is selling labor. This is the most basic business, and also the most direct source of employment for music annotators.
If an AI company needs professional annotation data for 100,000 songs, but does not have enough music professionals, it is obviously not cost-effective to recruit a separate music team. Thus, third-party data service providers emerge.
Service providers are responsible for recruiting annotators, formulating work specifications, assigning tasks, checking quality, and finally handing over the sorted data to customers. AI companies do not need to maintain a music team for a long time, and only need to purchase services when there are projects. This is very similar to traditional data annotation outsourcing.
The second type is selling professional judgment, which is more valuable than simple annotation.
When an AI company is about to release a new version of the music model, it needs to find professional musicians to do blind listening tests to confirm whether the new model has made progress.
For the same section of lyrics, let two models generate songs respectively, and then let musicians score from dimensions such as melody fluency, arrangement rationality, vocal performance and style matching. After finding problems, analyze whether the error comes from melody generation or the model's failure to understand the creation instructions.
What is finally delivered to the customer is a set of evaluation results, error analysis and optimization suggestions.
This has become a formal link in AI music R&D. The 2025 ICASSP conference included a study named MusicEval. The research team collected 2748 pieces of music generated by 31 AI music models, invited 14 music experts to score, and finally obtained 13740 evaluations.
The third type is selling data assets.
If a company has both annotation capabilities, a large number of music materials, and solves the corresponding authorization problems, it can package audio, music structures, professional tags, text descriptions and usage rights, and authorize them to AI companies for model training.
The GCX platform under the overseas company Rightsify, which specializes in AI music data authorization, authorizes sorted, annotated music datasets with corresponding rights arrangements to AI R&D enterprises and research institutions. The dataset scale of GCX has exceeded 4.4 million hours of audio, 32 billion sets of text data, and the total capacity exceeds 3PB.
Its fee is quoted according to the data scale, complexity, usage scenario and authorization period. GCX also emphasizes that the dataset obtains corresponding rights through commissioned creation and other methods, and provides commercial authorization.
The imagination space of this business is obviously much larger than simply selling labor.
Problems also follow: where does this music come from?
For large record companies, the use of unauthorized music to train models means damage to copyright and commercial interests; they still have strong resources to deal with it, but for ordinary musicians, they often cannot afford the consequences.
An original musician in Shandong revealed that he once received a call from an AI sampling company, hoping to collect his original music for model training. Some musicians sent new works to exchange groups, and soon the works were intercepted, trained by AI and regenerated. Many people have filed lawsuits for this.
Works created with painstaking efforts have become nourishment for AI training, but the products of AI may not bring any benefits to the original authors.
The teacher hasn't finished the class, but the student has already learned the skills
There is a paradox in this business: the harder people work to teach AI to understand music, the more opportunities AI will have to learn to listen by itself.
The UMP Board developed by DeepMusic has long tried to automatically extract music information with technology. By 2023, this system has been able to independently complete automatic annotation of 400,000 songs.
However, the early automation mostly helped people process some standardized information. The real change took place in the past two years.
On May 1 this year, MOSI.AI, the OpenMOSS team and Shanghai Chuangzhi College jointly open sourced MOSS-Music, which includes two 8B models and a supporting large-scale music data annotation pipeline.
The pipeline supporting this model can already start from the original audio, automatically complete feature extraction, song segmentation, lyric recognition, metadata cleaning, and then generate music descriptions and training samples. Even the evaluation of music descriptions has begun to try to use another large model to score.
For the content that people used to listen to over and over again with headphones, machines can now complete a considerable part of it instead of humans.
However, there is not only one technical route for music AI.
One route is to disassemble music into recognizable structured information, such as the UMP system of DeepMusic and MOSS-Music mentioned above.
The other route is end-to-end music generation represented by products such as Suno. Users input a piece of text, and the model directly generates complete music.
The difference between the two routes determines that the demand for music annotators cannot keep growing in the same direction. The former requires more refined music data, while the latter may reduce the dependence on manual annotation of each song.
But the common point is that both require data quality control and evaluation of generated results.
Looking at the entire AI music market, the speed at which machines generate music has far exceeded the speed at which humans consume music.
Data released by the overseas music platform Deezer shows that in June this year, the platform received an average of 90,000 AI-generated songs per day, accounting for more than half of the new music uploaded daily during peak periods.
By October, Deezer disclosed that from January 2025 to August 2026, the platform had excluded more than 42 million AI-generated songs from algorithmic and editorial recommendations.
The actual play volume of such a huge number of AI songs only accounts for 1% to 3% of the total play volume of the platform. How much of the massive content produced is worth being heard?
For music annotators, the focus of their future work will also change: basic recognition is handed over to machines, and professional musicians will be more involved in formulating evaluation standards, judging complex problems and correcting model errors.
Looking at the entire music field, from annotating music for AI, to helping AI judge quality, and then worrying that their own works will become nourishment for AI training, the relationship between musicians and AI is far more complex than simple substitution and being substituted.
When Zhou Shen released his new song "Chronicle of the Moon", he specifically stated that unauthorized use of the work for AI training, imitation, learning and generation is strictly prohibited.
Algorithms can replicate melodies and imitate styles, but that does not mean they have the creator's life experience; models can produce music in batches, but cannot produce cultural depth.
AI has opened up an unprecedented broad world for music. What kind of music deserves to be preserved, and how creators can get the due value, ultimately needs to be answered by humans. The core of art is always rooted in human beings.
This article is from the WeChat Official Account "Digital Intelligence Surge", author: Tian Yu, published with authorization from 36Kr.