ChatGPT's advertising performance is unsatisfactory

It is undeniable that ChatGPT Ads has been expanding rapidly since the start of this year.

In February, OpenAI began testing ChatGPT ads in the United States; in May, it opened the self-service Ads Manager and introduced CPC bidding; in August, ChatGPT Ads further expanded to 31 European markets, adding capabilities including conversion optimization, geotargeting, custom audiences, Pixel, Conversions API and third-party measurement. By the end of August, OpenAI announced that ChatGPT Ads had covered more than 40 countries, with tens of thousands of advertisers on board and reaching an annualized revenue scale of 1 billion US dollars.

From the perspective of advertisers, this seems to be a very attractive new channel.

However, the problem is that after advertisers actually invest their budgets, the results are not as impressive as expected. In recent weeks, several advertising agencies and marketing practitioners have successively published their own ChatGPT Ads test data. Impressions can be obtained, and clicks do exist, but the conversion performance is not ideal, and there have even been multiple cases of zero conversions.

9,000 Impressions in Exchange for 0 Buyers

Recently, MediaPost reported a ChatGPT Ads test conducted by Canadian digital marketing agency Choice OMG. Peter Jaffray, Managing Director of the agency, launched three sets of ads in Ontario and Alberta, investing a total of 415 US dollars over two weeks.

Originally, he expected to get about 4,000 targeted impressions, but in the end he received nearly 9,000.

Judging from the impression volume alone, the result seems good. But problems soon emerged, "My impression volume is much higher than expected, but the click-through rate is significantly lower, only about 0.6%, and the estimated cost per click is about 7 US dollars," he said. "If you compare it to search ads, I can usually achieve around 2%." At the same time, none of these clicks ultimately led to any purchases.

Jaffray believes that these clicks most likely came from real users, as the relevant IP addresses appeared normal. But after these users entered the website, they did not further browse or interact with the website content.

In his view, ChatGPT Ads at the current stage is more similar to display ads, and the conversion performance of high-intent users has not yet been demonstrated. This judgment is somewhat different from OpenAI's previous market expectations for ChatGPT Ads. OpenAI has been emphasizing the user intent of ChatGPT, which is also an important bargaining chip for it to attract advertisers.

Many conversations in ChatGPT itself take place during the consumer decision-making process. Some users ask about sofas suitable for small apartments, some compare computers suitable for video editing, and others ask ChatGPT to recommend marketing agencies. Users put their own needs, budgets, preferences and even concerns into a conversation. Compared with a simple search term, the user information it can provide is far richer.

From the perspective of advertisers, such scenarios naturally lead to expectations of better ad performance. Users have already stated their needs, the platform has mastered more context, and ad matching has a better foundation.

However, the several sets of test data that have been made public so far have not fully reflected this advantage. More notably, the problem has begun to extend from conversion performance to the data itself.

Nicholas Verity, CEO of B2B lead generation agency Cleverly, recently tested ChatGPT Ads. He found that after the same set of ads was placed on Google, Meta and ChatGPT, the CTR of ChatGPT was lower than the other two platforms, and the ads ultimately did not bring any conversions.

Then a problem arose that was even more unacceptable to him. The OpenAI backend showed that this ad campaign received 57 clicks, but Google Analytics recorded less than 20 visits. Verity also added UTM parameters to further confirm the source of this traffic, but the data between the two platforms still did not match.

For advertisers, this is more troublesome than a simple low conversion rate. How many real visits and valid clicks an ad placement actually brings is the most basic data for performance ads.

Verity therefore suspended this test. He stated on LinkedIn that when the platform data cannot correspond to the actual website data, it is difficult to judge whether it is worth continuing to increase the budget afterwards, and it is also impossible to rule out the possibility that clicks are counted repeatedly.

Similar feedback has also come from other advertisers. After Daniel Johnson, founder of We Scale Startups, tested ChatGPT Ads for several clients, he also found that the number of clicks on the platform does not fully correspond to Google Analytics. He believes that before the platform's own reports build credibility, advertisers are more suitable to treat this part of the budget as test expenses.

Some marketing practitioners have also shared similar experiences on social platforms, including cases where the platform shows clicks but third-party analytics tools do not record corresponding visits, and no conversions are obtained after hundreds of dollars of investment.

Some other public tests gave similar results. In August this year, marketing practitioner Yasha Boroumand shared a ChatGPT Ads test. He invested 460 US dollars, got about 15,000 impressions, with a CPC of about 2.22 US dollars, and the final number of conversions was 0. Another marketing agency, Grow My Ads, also made public the results of a test. The agency invested about 1,000 US dollars, got 92 clicks, with an average CPC of about 13 US dollars, and ultimately also had no conversions.

The sample size of these cases is not large enough to represent the overall performance of ChatGPT Ads. But at the very least, it shows that for advertisers who are just starting to test ChatGPT Ads, both the effect and data credibility need further verification.

ChatGPT's Ad Business May "Lose Out to AI Recommendations"

There is another more subtle phenomenon.

The Data-Driven Trades once conducted a sample analysis of 20 home service companies that placed ChatGPT Ads. The data shows that these ads brought a total of 68 independent leads, but the overall performance is still weaker than the organic ChatGPT recommended traffic. This sample size is not large, and cannot represent the entire ChatGPT Ads market, but it provides a noteworthy perspective for observation.

In the ChatGPT scenario, users brought by ads are not necessarily more valuable than users brought by active AI recommendations.

When users actively ask ChatGPT for advice and the brand appears in the answer, what the user receives is an AI recommendation; when users see an ad, they first know that they are facing commercial content before deciding whether to click. Both types of traffic occur in ChatGPT, but the way users establish connections with brands is not the same.

This may become a relationship that OpenAI needs to handle for a long time in the process of commercialization. How ads enter the conversation scenario while maintaining users' trust in AI responses, and whether advertisers can get a sufficiently high commercial return from this new reach method.

Of course, looking only at these current tests is not enough to draw a final conclusion on ChatGPT Ads.

ChatGPT Ads only started testing in February, and its commercialization time is not long, with many advertising capabilities being iterated rapidly. From opening the self-service Ads Manager in May to adding conversion optimization, geotargeting, custom audiences and more complete conversion measurement capabilities in August, OpenAI has been continuously improving the infrastructure required for performance advertising.

These actions basically correspond to the capabilities that a performance ad platform needs to have, including finding the right people, delivering ads to the right scenarios, tracking whether users click and convert, and continuously optimizing based on the results. Google and Meta have accumulated more than ten years on this system, while ChatGPT Ads has only just started.

Therefore, the cases of low CTR, zero conversion, and mismatch between platform and third-party data that we see now are more appropriate to be regarded as problems exposed in the early stage of ChatGPT Ads. They indicate that the platform has not yet fully run through the entire link from user reach to final conversion.

But the user resources in ChatGPT do have their own characteristics. Google knows what users have searched for, and Meta can judge interests based on users' past behaviors. What ChatGPT faces is an ongoing conversation. Users may tell it why they want to buy something, what their budget is, which products they have compared, what problems they are worried about, and even continue to ask about the differences between different solutions. For the advertising system, the value of this information is far higher than an isolated keyword.

OpenAI's current ad matching mechanism is also using this information. According to the official introduction, the system will refer to the context and intent in the current conversation, and combine information such as ad titles, landing pages, and context hints provided by advertisers for matching. OpenAI has also made it clear that the ad system will match relevant ads based on signals such as the topics users are discussing.

In the past, ad targeting was mostly focused on "who this user is", including age, interests, behaviors and search records. In the AI conversation scenario, the platform has the opportunity to further understand "what problem this user is currently solving".

This is also the most promising part of ChatGPT Ads.

But user intent itself cannot be directly converted into ad value. For advertisers, it ultimately comes down to customer acquisition cost, conversion rate and revenue. Assuming that with the same 100 US dollars of investment, Google brings 10 clicks and 1 customer, Meta brings 20 clicks and 1 customer, and ChatGPT only brings 8 clicks but 2 customers, advertisers will naturally increase their budgets. Conversely, if ChatGPT has a large number of high-intent conversations but the final conversion performance does not show advantages, these user scales and context information will be difficult to convert into real ad value.

Moreover, for B2B advertisers, there is another practical problem to consider.

Currently, ChatGPT Ads is mainly available to Free and Go users, and paid users such as Plus, Pro, Business and Enterprise will not see ads. For B2B brands that need to reach professional users and enterprise decision-makers, this group of people may have higher commercial value in the first place.

Conclusion

Therefore, what ChatGPT Ads needs to prove next is actually very clear. OpenAI has already proved that advertisers are willing to buy, and that ChatGPT can become a new ad entry. The 1 billion US dollar annualized revenue figure announced by OpenAI yesterday further shows that commercial demand already exists.

But ad revenue growth and ad performance are two different things. The former shows that the market is willing to try, while the latter determines whether advertisers will stay for the long term. For OpenAI, the real test is whether it can convert the user intent, conversation context and decision-making scenarios that ChatGPT has into results that advertisers are willing to continuously increase their budgets for.

For the entire advertising industry, a more noteworthy issue is also emerging. When users start to find answers, compare products and make decisions in AI, will the most valuable traffic in the future come from ads or from the AI's own recommendations?

This article is from the WeChat official account "wj00816" (ID: Morketing), written by Alan Wang Jingxing, and published with authorization from 36Kr.