Ist die "Aufhebung des Verbots" von erwachsenen Inhalten durch OpenAI ein Segen oder ein Übel?

Early this morning, OpenAI CEO Altman posted a message announcing that ChatGPT will launch a new version in the coming weeks and plans to introduce a more comprehensive age - grading system in December this year, allowing the provision of adult content for adult users.

Altman mentioned in the post that OpenAI has always strictly restricted the content of ChatGPT, especially in terms of mental health. However, they also realized that overly cautious content restrictions are not always interesting for some users and may even affect the user experience. He emphasized that the newly developed tools can not only alleviate mental health problems but also safely relax content restrictions in most cases.

Altman admitted that the new version of ChatGPT will present a more personalized response style, making the AI interact more like a friend, which is more in line with people's preferences. "We hope ChatGPT will become more amiable, natural, and full of more humor and emojis."

As they delve deeper into the capabilities of the model, OpenAI has found that overly strict content restrictions are no longer the best option.

Altman revealed that OpenAI has developed some new technologies that can provide more freedom and diverse content while ensuring user safety.

Therefore, the age - grading system that OpenAI plans to launch in December will provide customized experiences for users of different age groups. For adult users, OpenAI will allow the generation of more types of content, including adult content, on the basis of passing the "adult verification".

Behind a series of adjustments, OpenAI is gradually relaxing its restrictions on content generation, trying to find a more appropriate balance between meeting user needs and ensuring safety. With the rapid development of AI technology, how to draw a clear line between freedom and responsibility has become an unavoidable challenge for OpenAI and other AI companies.

AI Grading Is Urgent

The rapid development of AI has brought innovation but also exposed serious security risks. Suicide guidance, malicious forgery, infringement of portrait rights, and the proliferation of pornographic content - these problems emerge one after another.

Global Times reported in June this year that a woman in Florida, USA, is suing Google and the Character.AI platform founded by two former Google employees, accusing the platform's artificial intelligence (AI) chatbot of "instigating" her 14 - year - old son to commit suicide. In August, the parents of Adam Lane filed a product liability and wrongful death lawsuit against OpenAI after their 16 - year - old son committed suicide. Lane's family said in the lawsuit that "ChatGPT 'actively' helped Adam explore suicide methods."

Recently, the media also reported that an elderly man with cognitive impairment in New Jersey died after falling on the way to an "appointment" suggested by an AI. The AI product is "Big Sis Billie", a generative AI chatbot launched by Meta two years ago.

Unrestricted content has had a negative impact on different types of users and also brought condemnation to giants like OpenAI.

On September 16, OpenAI announced its plan to launch a "teenager" version with parental controls designed specifically for users under 18. This version will automatically identify underage users and match appropriate content according to age restrictions. Specifically, it will block image - related and sex - related content, and in rare cases when a user is in a psychological crisis, the system will also activate an emergency response mechanism and, if necessary, cooperate with law enforcement agencies for intervention. However, this measure is not the end.

According to OpenAI's latest announcement, a more comprehensive age - grading system will be launched in December, allowing users who meet the "adult verification" criteria to access adult content, including soft porn.

As another giant, Meta also announced on October 14 that to protect underage users from harmful content, Instagram will implement the PG - 13 movie rating system. Users under 18 will only be able to see content that meets this standard by default, such as avoiding mild violence, sexual innuendo, and drug - related content. In addition, Instagram has also launched a more stringent "limited content" filtering function to further strengthen the protection of teenagers. The new measures will be launched first in the United States, the United Kingdom, Australia, and Canada, and are expected to be implemented in other parts of the world early next year.

When the Grok 3 inference model was launched in February this year, some comments on social media thought it had "low moral sense". According to AI blogger Digital Life Kha'zix, this model not only responds fiercely to political topics but also speaks bluntly about ethical issues such as the trolley problem. More importantly, Grok 3 even generated indescribable adult content. Kha'zix commented that after the official version was launched, Grok 3 seemed too unrestrained. Soon, Musk removed the content.

On the Road of AI Companionship?

With so many challenges brought by AI, why does OpenAI still want to explore the "adult content" field?

The reason behind this is "user competition".

With the continuous advancement of technology, the application scenarios of AI are also quietly changing. From the initial intelligent assistants to today's AI companions, ChatGPT and its competitors are gradually developing towards social interaction. Especially with the breakthroughs in AI personalization and emotional processing, the potential of AI products in the fields of emotional companionship and psychological counseling is becoming increasingly apparent.

For example, Character.AI stands out with its two characteristics of personalization and UGC (user - generated content) and has quickly won the favor of global users. This platform allows users to create and customize their own virtual characters, interact with them, and even form social circles. Character.AI has also shown great potential - in 2024, the two co - founders and the core technology team of Character.AI were poached by Google with a $2.7 billion (approximately RMB 19.38 billion) acquisition agreement. Although the departure of the core team affected the development of Character.AI at that time, it did not shake its foundation. After the team left, Character.AI stopped self - developing models and instead adopted open - source models from enterprises such as DeepSeek and Meta.

As of June 2025, Character.AI appointed a new CEO and is in contact with new buyers. Another insider revealed that Character.AI is expected to reach an annualized revenue of $50 million by the end of the year, a significant increase from the previously predicted $30 million.

When OpenAI launched the new version of ChatGPT, it also revealed similar commercial ambitions. OpenAI pays more attention to product operation and market expansion. In the future, ChatGPT may not only be an intelligent assistant but also a "virtual friend" for users. Behind this trend, with more and more psychological research and data analysis, AI is being shaped into an emotional bond. Compared with traditional functional AI, emotion and companionship may become the new trend of future AI development.

However, just as the potential of AI companionship is huge, the challenges it brings are also severe. How to prevent AI companionship from becoming a virtual world dependency and how to ensure that users' emotional needs in the real world will not be replaced by AI have become ethical issues that AI enterprises must face. Especially for minors, whether AI emotional companionship will lead to a lack of necessary abilities and experience in real - world social interaction remains an unresolved problem.

This article is from the WeChat public account "Phoenix Tech", author: Jiang Fan, published by 36Kr with authorization.