In the next 70% of my life, I will probably keep asking: Is this made by AI?

The issue of AI-generated forgery really needs to be properly regulated.

On the morning of August 26, a mudslide tragedy just occurred at the Jilong Port, and fake images created by AI spread that very night.

The high-traffic post on Weibo has received more than 8,000 likes, and many readers who have no knowledge of the truth are still expressing their emotions over the fake images up to now.

The only person who put forward evidence to refute the rumor in the comment section even used a screenshot of the conversation with Doubao as proof...

What is more, not only ordinary people, but even professional media can be tricked.

In early March this year, a classic incident of AI fake image pollution took place during the US-Iran conflict.

Starting from the source supplier SalamPix, the AI-generated fake scene photos first obtained the credibility endorsement of a regular French photo agency, and then spread to all major mainstream media in Europe.

Germany's long-established media Der Spiegel, the largest Dutch news agency ANP and many other institutions were all involved in the incident, and they urgently released statements to remove the suspected images from their platforms.

If things go on like this, when we see on-site reports, the first thing we care about may no longer be the news itself, but to question whether the content is generated by AI in the first place.

So is there any feasible way to regulate this problem?

To this end, we tested some common detection methods on the market, and also consulted researchers from Vermilion Lab. The result can only be described as: the law is one step ahead, the devil is ten steps ahead, and there is really no perfect solution that satisfies both sides for now...

For example, the most commonly used official method is to add watermarks in advance, such as marking the words "AI Generated" clearly on the generated images and videos.

This method is conspicuous and intuitive, but it is far too easy to remove, so much so that people joke that the only use of PS now is to erase these words.

To put it plainly, this is just a formal statement that cannot solve the problem of authenticity identification at all.

Therefore, more hidden invisible watermarks came into the spotlight.

For example, C2PA, which is adopted by many large tech companies by default, will package information such as the model and generation tool into the image at the same time when the image is generated.

In an ideal state, C2PA can even record the editing and modification history of the image.

Does it sound a bit like the metadata of mobile phone photos, which can record the geographical location of the photo and the aperture size of the camera?

Exactly, C2PA is very similar to metadata, equally ideal, but also equally fragile.

Operations like screenshotting and cropping, or compressing the image when sharing it on WeChat, will make this information disappear completely.

For example, for this image generated by GPT Image 2, right after we downloaded it and uploaded it to the official OpenAI verifier, it failed to recognize the watermark at all, which is a solution that only treats the symptoms but not the root cause.

Then Google, the "benevolent tycoon", stepped forward and launched the highly praised SynthID invisible watermark.

This technology is more in line with the characteristics of the AI era. Instead of attaching the label outside the file body, it mixes some machine-recognizable signals into images, videos, audios, and even texts during the generation process.

In their technical paper, they explained that when Gemini generates tokens, it will be slightly intervened by this technology, so that the content of the final product remains basically unchanged, but presents certain traces and features in terms of signal statistics.

Simply put, there is no visible difference to the naked eye, but dedicated detectors can identify it.

In addition, SynthID is more robust than C2PA, and it can basically resist common operations like cropping, adding filters, and compressing.

Seeing such good performance, OpenAI also joined the camp. For example, the extremely realistic GPT Image 2 now supports SynthID.

We also used the same test method and found that it does have excellent performance. The watermark can still be identified after most images are cropped, compressed, or even added with filters.

A few days ago, Anthropic also added SynthID watermarks to Claude's text output to comply with EU regulations. The team of supporting enterprises is getting larger and larger, so does that mean the problem will be completely solved after full coverage of this technology?

Well, the situation is not that simple.

Even if both use SynthID, their detectors are not interoperable. Images generated by Gemini cannot be recognized by OpenAI's verifier, and vice versa.

This technology cannot form a unified identity authenticator for now, and each company can only take care of its own products.

What is more troublesome is that for watermarks to work, they must be detectable. Once the detection scheme is made public, corresponding technologies for erasing, forging and countering it will emerge soon.

As for the text watermark launched by Anthropic, just a few days after its release, a counter watermark removal project appeared on GitHub, which got 20,000 stars in less than a month.

Google itself also admits that there is no such thing as a perfectly secure watermark. Their goal is very realistic: to raise the cost of attacks and forgery.

Therefore, third-party detectors are also very popular in the market now. They do not rely on watermarks, but can infer whether the content is AI-generated based on its features.

Unfortunately, after a series of tests, we found that the performance of these detectors varies greatly, and some of them are really unsatisfactory.

We asked ChatGPT to generate an article about pseudo-historic theory, and submitted it to different AI detectors for testing.

Three of the four foreign detectors failed to find any AI traces. The only relatively capable one, GPTZero, even identified the 2000-character Chinese article as only 49 words, which shows it simply cannot recognize Chinese at all...

From left to right: Originality.ai, winston, AISEO, GPTZero

The situation of domestic detectors is not much better. The detection result of Wanfang is still 0%, and VIP only detected 0.42% AI content after the article was expanded to 20,000 words.

Only the Vermilion AI detector has relatively better performance, with a result of 38% human-written content and 62% suspected AI-generated content. Although it cannot give a definite conclusion, it at least marks the content as "suspicious".