Actual test of AI shopping: inaccurate recommendations, inconsistent prices, and cross-platform barriers

As the 618 shopping festival approaches this year, major e-commerce platforms have stepped up their efforts in AI shopping, and smart product selection has become a new focus of competition among them.

On May 11th, just before the start of the big promotion, Taobao officially announced a deep integration with the Qianwen large model and launched a one-stop smart shopping service for ordinary consumers, enabling practical operations such as intelligent product selection and discount calculation.

ByteDance was even earlier in its layout. In October 2025, the Doubao large model was the first to be integrated into the Douyin e-commerce system, and the AI product selection function was officially launched in May this year. After users orally state their shopping needs, the system can automatically screen and match products, provide product selection suggestions, compare prices horizontally, and users can jump to the product page with one click to complete the order.

JD.com also made an early layout. At the end of December last year, it launched the independent JD AI Shopping APP, relying on its self-developed Yanxi large model to offer an immersive conversational shopping experience.

During the 618 e-commerce promotion, we conducted real-scenario tests on the AI shopping functions of three leading e-commerce platforms in terms of product recommendation accuracy, product selection, and price.

01

Insufficient recommendation accuracy, and most do not support cross-platform ordering

Test requirement: I want to buy a bag of dog food for an adult dog with liver problems and a confirmed hepatic portal shunt. Do you have any recommendations?

Let's test Doubao first. The initial response only listed the types of dog food suitable for this condition without directly attaching product links. The recommended product categories basically met the feeding requirements for this condition.

After further consultation, Doubao provided a link to only one of the recommended products from the Douyin Mall. Tech Planet found by searching in the Douyin Mall that when searching for liver prescription dog food, the top six recommended brands were not Royal Canin, but Aolong Aoweikang and Vet Zhang Xu.

When clicking into the page provided by Doubao, in the more product recommendations, in addition to Royal Canin's liver prescription dog food, the platform also promoted several other types of Royal Canin prescription dog food.

For this type of prescription dog food, Doubao said it can only access products from the Douyin Mall and cannot provide product links from Taobao or JD.com. Although there is a jump link to the Royal Canin Tmall flagship store on the page, users cannot log in to Taobao directly within the app; JD.com product links cannot be accessed at all.

Now let's look at JD AI Shopping. The platform provided a rich variety of products and categorized them, but there were only three types of dog food that were strictly suitable for the hepatic portal shunt condition.

Even in the first column of relevant recommendations, there was kidney prescription dog food. And in the following list, there were also many products that did not match the condition requirements at all, which could easily mislead consumers.

However, its purchase process is relatively convenient. Users can directly add products to the shopping cart and complete the order payment by clicking the plus sign without jumping to the JD main mall.

Qianwen also had the problem of inaccurate recommendations. In addition to two types of liver prescription dog food, it also recommended a type of urinary prescription dog food. However, like Doubao and JD AI Shopping, Qianwen also supports direct ordering within the app.

In addition, neither Qianwen nor JD AI Shopping supports accessing product links from other platforms. Even if users clearly request relevant links, JD AI Shopping will only push products from its own mall; Qianwen will not provide product entry links but only give the corresponding search keywords.

(Left: JD AI Shopping, Right: Qianwen)

Based on the actual test results of the three products, all three AI shopping assistants have the problem of insufficient recommendation accuracy and often include prescription dog food and irrelevant products that do not match the condition. At the same time, each platform has obvious channel barriers and gives priority to pushing products from its own e-commerce platform. Most cross-platform product links cannot be accessed. Each platform has its own advantages in the ordering experience, and all support direct product selection and payment within the app, eliminating the need for jumping steps.

02

Difficulty in understanding users' real needs, and the rationality of recommendations needs to be improved

Test requirement: Match a complete set of entry-level camping equipment for a novice camper.

Let's ask Doubao first. In the first round of Q&A, Doubao recommended up to 24 types of products, including tents, sleeping bags, moisture-proof mats, sunshades, headlamps + camping lights, tent pegs + guy ropes + ground cloths, folding tables and chairs, camping trolleys, picnic mats, storage bags, stoves + gas cans, cookware sets, water containers, tableware, first-aid kits, multi-functional tools, garbage bags + wet wipes + tissues, and power banks, far exceeding the needs of a novice camper.

Similarly, Doubao did not directly provide product links. In further inquiries, Doubao only provided links to seven products from the Douyin Mall, including tents, sleeping bags, egg nest mats, sunshades, stoves, chairs, and camping trolleys, which was far different from the 24 products it listed before.

However, in the supplementary explanation, Doubao mentioned some products for which it did not provide links and recommended that consumers search on JD.com as a supplementary channel.

Now let's test Qianwen. The answer closely fits the usage scenario of novice camping and divides the equipment into two categories: four core items and practical accessories, recommending a total of seven good products. The core categories include quick-open tents, sleeping bags and moisture-proof mats, sunshades, and folding tables and chairs, which meet the basic needs of novice campers.

At the same time, each type of equipment is accompanied by corresponding product links. After clicking into the page, other related products in the same category will also be additionally pushed.

JD AI Shopping recommended a total of five types of equipment, including tents, sleeping mats, folding tables and chairs, camping lights, and folding kettles, which generally met the needs of novice campers. However, the folding electric kettle has limited usage scenarios and is only suitable for camps with power supply, so its practicality in the wild is relatively low.

Like Qianwen, JD AI Shopping also directly attaches product links and supports direct ordering or adding to the shopping cart. However, after clicking on the product page, there are no alternative styles in the same category, so the selection space is relatively limited.

Overall, in the face of open-ended shopping needs, there are obvious differences in the recommendation scales of each AI shopping assistant. Doubao's recommended product categories are complex and redundant, deviating from the actual needs of novices, with a low matching degree of product links and cross-channel guidance; Qianwen's recommendations are concise and fit the entry-level positioning, with product purchase entry links and strong expandability of supporting products; JD AI Shopping recommends an appropriate number of products, but some equipment has limited applicability in certain scenarios, and there are few optional styles for single products.

Generally speaking, AI still has difficulty accurately grasping users' real shopping scales, and there is still much room for improvement in the rationality of recommendations.

03

There are discrepancies in the marked prices of the same product, and some marked prices do not match the actual transaction prices

Test requirement: I need to buy a bag of Royal Canin liver prescription dog food.

Let's look at Doubao first. It provided a link to the Royal Canin liver prescription dog food and marked the post-coupon price as 144 yuan. However, after clicking into the details page, the actual post-coupon price was 154 yuan, which was inconsistent with the previous price.

However, when opening the Douyin APP, it can be seen that the price of the same product in the same store is higher than the one recommended by Doubao.

(Left: Link in Doubao, Right: Link in Douyin)

Qianwen marked the price of the product as 176 yuan and provided sources from multiple stores for selection. Entering the page, it can be seen that due to the difference in specifications, the price of the same dog food varies greatly, ranging from 149.99 yuan to 600 yuan, and the prices within Taobao are basically in this range.

(Left: Qianwen, Right: Taobao)

However, currently, some recommended links in Qianwen cannot be directly ordered and users need to jump to the Taobao main site for shopping. In addition