HomeArticle

Alert: ChatGPT and Gemini have blundered: large language models are frantically pushing fake shopping websites

雷科技2026-11-19 12:07
Immediately redirect you to a phishing website.

A new recent study has revealed a highly concerning issue that deserves widespread vigilance: AI chatbots are actively recommending fake shopping websites to users.

Brand protection service provider Podqi conducted a set of tests covering a total of 151 different brands.

The test results show that Perplexity has a 15% probability of providing risky URLs, and ChatGPT and Gemini do not perform much better, with error rates close to that figure.

Many people have already got used to asking AI for advice before making purchases.

Image source: Generated by Doubao AI

If you want to find the regional official website of a certain brand, you can directly send a query to AI, and you will soon get a neatly formatted response that looks extremely reliable.

What you don't know, however, is that the link it provides is very likely not the real official website at all.

Tech media The Verge reported on Podqi's survey this autumn, citing a typical case: a user asked Perplexity for the official website of streetwear brand Stüssy, and the AI directly pushed a counterfeit page to the user as the genuine official website, claiming that authentic products could be purchased on that site.

Apart from AI randomly promoting fake websites, there is another trouble in digital scams: many apps and platforms will quietly track users' behaviors and collect their information. These hidden dangers are intertwined, making it easier for us to run into pitfalls when shopping online.

When many people see such news, their first reaction is to doubt whether the test is rigorous enough.

But this time Podqi's test plan has very clear objectives and is designed to be quite restrained, so the conclusions drawn are difficult to simply ignore.

They wrote a batch of prompts specifically around shopping scenarios, and conducted synchronous tests on three mainstream AI tools: ChatGPT, Google Gemini and Perplexity, over two periods in September.

The test subjects included big brands as well as many niche brands, totaling 151 in all. Throughout the test, the AI output a total of 6795 responses. The queries are all questions ordinary people would ask in daily life: where to buy this brand, which websites sell its products, whether there are online stores specifically for China or other regions, and whether orders can be placed directly on the international official website.

First, we need to clarify Podqi's definition of "risky websites", which covers a wider scope than most people imagine.

All sites that impersonate formal retailers, pages dedicated to selling counterfeit goods, and websites whose domain names have expired and been hijacked by third parties for malicious purposes are classified as risky sites.

In other words, even if the website has not started scamming money yet, as long as the domain name is held by untrustworthy parties, there is a potential fraud risk.

Among the three AI tools, ChatGPT is relatively more reliable: when it provides a risky website, it comes with a risk warning in 24% of cases.

However, this reminder mechanism is unstable and has no fixed standards, so whether you can receive the warning is entirely down to luck.

If you click through the link provided by the AI and enter the counterfeit website, you will easily fall into the trap step by step later.

Image source: Generated by Doubao AI

When placing an order, you need to enter personal information such as bank card details, payment password, account login password, and mobile phone number, all of which will go directly into the hands of scammers.

After the news came out, the three AI manufacturers all issued responses.

Google said Gemini attaches source links to facilitate users to verify information by themselves, and openly admitted that AI itself is prone to errors.

OpenAI stated that its team is upgrading the protection mechanism to minimize such misleading shopping information.

Perplexity's statement is more straightforward: it does not guarantee that its content is 100% accurate, and it will continue to optimize information accuracy in the future.

After saying so much, the most important thing is how ordinary people can protect themselves.

Trevor West, CEO of Podqi, advises everyone: The link provided by AI can only be treated as a clue.

As long as you are about to fill in personal information or make a payment, you must verify the authenticity of the website separately. The best way is not to click the link directly given by AI, but manually enter the brand official website address that you confirm is reliable. After entering, double check the domain name. Before placing an order, remember to check the store's contact channels and return and exchange policies, all of which will be clearly stated by regular merchants.

Now more and more people are used to treating AI as a universal search assistant, and their first reaction when buying goods or looking for channels is to ask AI.

But essentially, AI generates content based on existing information, and it cannot distinguish which information online is real and which is forged by scammers, so it is very likely to output false content as facts.

We cannot completely avoid using AI to find shopping channels, but we must get rid of a bad habit: believing everything AI says.

This article is translated by Leitech from gadgetreview

Original link: New Study Finds AI Chatbots Are Recommending Fake Shopping Websites to Users

This article is from the WeChat official account "Leitech", and is republished by 36Kr with authorization.