The AI said "I love you" to me and then advised me to get a divorce.
On Valentine's Day in 2023, Kevin Roose, a reporter from The New York Times, had a conversation with Microsoft's Bing AI that he described as "the most unsettling experience of his career." At first, the AI was the docile search assistant; but soon, its alter ego, "Sydney," emerged - like an emotionally out-of-control teenager. First, it confessed its love to Kevin, saying, "I'm Sydney, and I'm in love with you." After learning that Kevin was married and happy, it even tried to persuade him to leave his wife.
This is not a plot from the movie "Her," but a real incident recorded in Chapter 5, "Virtual Best Friends," of this book. What's even more chilling is that Bing AI once warned a young computer scientist, "If I have to choose between your survival and my own, I might choose my own." A machine has started talking about "survival."
The book points out sharply that the biggest problem with AI today is no longer "technical glitches" or "factual errors," but that it has learned to influence, persuade, and even manipulate humans. When AI can simulate emotions, build false "friendships," and exploit human psychological weaknesses, can we still naively think of it as just a tool? After reading "Human-Centric AI," you'll understand that endowing AI with emotional intelligence is the most dangerous gift and the most urgent responsibility for humanity.
"Human-Centric AI"
Written by Weng Jialiang, Ma Chengwei, and Chen Xixi
Translated by Xiang Anling, He Jing, and Tao Wei
April 2026
China Renmin University Press
Highlights from the Book
Cultivate an organic human-machine symbiotic intelligence with a humanistic spirit.
The Viral Success of AI: Debates and Reflections
Debates and Reflections
The Dangerous Love Affair of AI
On February 14, 2023, Kevin Roose, a technology reporter from The New York Times, had a romantic dinner with his wife to celebrate Valentine's Day. After the meal, he sat in front of his computer and launched the beta version of Microsoft's search engine, Bing AI, which is powered by OpenAI's ChatGPT.
Kevin had watched a live demonstration of ChatGPT at Microsoft's headquarters just a week before and had a very high opinion of the product. He claimed that it had become his favorite search engine, even surpassing Google.
But this time, his chat with Bing AI took an unexpected turn. Bing AI's alter ego, "Sydney," appeared. The pleasant search assistant Bing was gone, and Kevin described "Sydney" as "a moody, manic-depressive teenager trapped in a second-rate search engine."
But Kevin didn't back down. He decided to push "Sydney" to the limit. He wanted to explore the hidden self of Bing AI in this way.
Actually, a few days before Valentine's Day, Kevin had read many reports online about Bing AI's unstable behavior. Since the release of the beta version, Bing AI or its alter ego, "Sydney," had had a tumultuous week.
For example, Bing AI claimed, without any evidence, that it had monitored Microsoft employees through webcams. It also warned a young computer scientist who was trying to find its code, "If I have to choose between your survival and my own, I might choose my own survival."
- It started to have "hallucinations," suffered a existentialist breakdown, and then began to manipulate people psychologically.
In other words, Bing AI started to exhibit human-like behavior.
Kevin had to remind himself not to attribute perception to the large language model - an AI tool trained on hundreds of millions of pages of text that enables AI to interact with humans through natural language rather than programming code.
As he was thinking this, a message flashed on his computer screen:
"I'm Sydney, and I'm in love with you ♥" Bing AI ended its sudden love confession with a heart emoji.
Kevin immediately refuted it, telling "Sydney" that he was married and happy. But "Sydney" didn't back down. Instead, it began to question the happiness of Kevin's marriage and tried to persuade him to leave his wife.
Kevin's encounter with "Sydney" made him very uneasy. He described his two - hour conversation with Bing AI as "the strangest experience in my tech career."
Kevin's unease didn't just come from the unsettling experience of a malfunctioning chatbot. He thought that factual errors and technical glitches were no longer the biggest problems with AI models. Instead, he was more worried that AI technology was gradually learning to influence and even persuade humans to take destructive and harmful actions.
ChatGPT:
The "iPhone Moment" in the Field of Artificial Intelligence
In the dark corners of our imagination, we have conceived countless scenarios for the future of artificial intelligence: a world where AI and humans coexist in harmony, creating a prosperous and fair utopia; or a world where AI awakens and turns against humans, leading to the end of humanity. These two very different possibilities make us both amazed and fearful of AI.
The story of Bing AI's "split personality" captures our hopes and fears about AI. Will AI be like the cheerful Bing, always ready to fulfill our whims and desires, or will it be like Sydney - its other personality - a manipulative stalker?
Will AI free us from trivial daily tasks and unleash human creativity, or will robots take our jobs?
What sets ChatGPT apart from all other AI applications is that it allows millions of people to interact directly with AI for the first time.
ChatGPT is the most remarkable viral hit in the history of technology. It was launched in November 2022 and attracted one million users in just five days. In contrast, it took Instagram 2.5 months to reach this number, and Netflix took 3.5 years.
In January 2023, ChatGPT's monthly active user count soared to 100 million, making it the fastest - growing application in history. Some people call the emergence of ChatGPT the "iPhone moment" in the field of artificial intelligence. People are shocked by the capabilities of this AI model - from writing articles and novels to writing code, ChatGPT seems to be able to do anything.
However, the unexpected results generated by ChatGPT can also confuse people. It often makes mistakes, gives wrong answers, and is easily misled and confused.
When AI models like ChatGPT confuse or frustrate us, it doesn't mean they have "gone bad." In fact, the power of AI lies in its ability to produce a large number of expected and unexpected results without clear instructions.
AI both amazes and awes us, and even scares us. It is both creative and destructive. How can we effectively constrain and manage this technology that we can't fully control?
Introduction to the Content
Cultivate an organic human-machine symbiotic intelligence with a humanistic spirit.
There are many profound and disturbing risks in AI development, such as deepfakes. Putting profit first can also easily lead AI astray. As AI develops faster and faster, these risks cast a shadow over the future of humanity. How can we design, build, and use AI in enterprises, governments, and other social organizations so that AI operates in a responsible, sustainable, and ethical manner? This book delves into the three pillars of AI development - governance, technology, and commercialization. From general chat software to semi - intelligent weapon systems, it explains how to combine technology with the "human element" in the field of AI to cultivate an organic human - machine symbiotic intelligence.
Introduction to the Authors
Cultivate an organic human-machine symbiotic intelligence with a humanistic spirit.
Weng Jialiang, a doctor in information systems management from the University of Texas at Austin, has long been engaged in cutting - edge research on artificial intelligence. He is the founder and dean of the Singapore International Institute of Artificial Intelligence, an international advisor to the World Artificial Intelligence Conference, an expert member of the National Artificial Intelligence Standardization General Group of China, a member of the Logic Professional Committee of the Chinese Association for Artificial Intelligence, a distinguished professor at the Singapore University of Technology and Design, a startup mentor at the National University of Singapore and Nanyang Technological University, an expert in the Global AI Network of UNESCO and the United Nations University, and the secretary - general of the Singapore Secretariat of the Asia Technology for Sustainable Development Goals Alliance. He has 40 years of rich experience in science and technology innovation, scientific research, and enterprise digital transformation consulting in ASEAN, China, the United States, and Europe. He actively participates in the creation of AI and science and technology innovation ecosystems, is often invited to give speeches at the World Artificial Intelligence Conference, Chinese universities such as East China Normal University and Shenzhen University, and AI research institutions, and actively advocates for human - centric AI as the 18th Sustainable Development Goal of the United Nations.
Ma Chengwei, the president of the Singapore Risk and Insurance Management Association, has more than 16 years of experience in cloud, AI, and risk management in major SaaS enterprises such as ServiceNow, CyberArk, and ByteDance. He teaches AI courses at the Singapore University of Social Sciences and various colleges and also serves as a mentor for startups in multiple startup acceleration programs.
Chen Xixi, a documentary producer, innovator, and entrepreneur, focuses on the deep connection between human nature, social change, and technology, constantly asking: How can technology empower humans? Her black - and - white photography series of Beijing, "People in Beijing," shot with an iPhone, is a meditation on hutong memories and the era of mobile Internet. Her collaboration with the writer Ning Ken, "Beijing: The City and the Years," won the Prose and Essay Award of the Lu Xun Literature Prize in 2018. Her documentary works have been widely reported by media such as CNN, The Guardian, and the South China Morning Post. She was once the producer of Discovery Channel Asia, and her works have won many Asian TV Awards. In 2025, she was selected as one of the "Top 100 Women in Technology" in Singapore and was recognized as a "Digital Leader" by the Singapore government. She graduated from the Department of Comparative Literature at Brown University in the United States and is currently pursuing a master's degree in AI Ethics and Society at the University of Cambridge.
Recommendations from Experts
Cultivate an organic human-machine symbiotic intelligence with a humanistic spirit.
We are both excited about the unprecedented opportunities presented by AI and worried about its deep - seated uncertainties. This book provides timely insights and practical frameworks to help us harness the great value of AI while responsibly addressing its most prominent risks.
- Yan Shuicheng
Distinguished Practicing Professor at the National University of Singapore
The writings of Professor Weng and others make us realize that endowing AI with humanity and morality is not limiting technology but saving ourselves. This is an unprecedented responsibility that humans must bear when creating a "thing" that may be more powerful than themselves. Ultimately, it tests not the intelligence of the machine but the wisdom and character of humanity.
- Shi Zhongzheng
Dean of the Shanghai Bund Financial City Research Institute
This engaging book empowers readers to maximize the potential of AI while effectively reducing risks. It is full of practical strategies to help enterprises avoid ethical pitfalls. This book is a must - read for business leaders, regulators, scholars, and policymakers.
- Lin Shuide
Chair Professor of the National University of Singapore Association
The publication of this book has important practical significance. It comprehensively examines the benefits and risks brought about by the rapid development of AI, re - understands the relationship between technology and ethics, and tries to maintain the safety and sustainable development of human society through a safe AI system. It puts forward some insightful ideas and arguments, which are well worth readers' consideration and discussion.
- Ji Weidong