In-depth Interview with Harari: The Real AI Crisis Has Arrived
We are always imagining doomsday scenes where AI awakens and rebels against humanity.
The movie *Terminator*
Yuval Noah Harari, author of *Sapiens: A Brief History of Humankind*, told us in his latest interview that the real crisis is not at all like the plots in movies — it is unfolding in the real world right now.
A knife can be used to cut salad, but it can also be used to commit violent crimes, yet the right to choose always rests in human hands. Nuclear energy can generate electricity, but it can also be used to manufacture nuclear bombs, and its direction is also determined by humans. Essentially, all technologies invented by humans over tens of thousands of years in the past are tools.
But AI is different.
Harari proposes: AI is not a tool, but an intelligent agent (Agent)
Yuval Noah Harari attending the interview *Hyde Park Civilizace*
It can make independent decisions and generate ideas and solutions that its creators have never envisioned. If humans can predict 100% of all outputs of AI, it makes no sense for trillions of capital to pour into the AI track. What capital bets on is precisely that AI can accomplish things that humans cannot predict and cannot achieve.
Many people keep asking: When will AI reach human-level intelligence?
Harari believes that this question itself is a misunderstanding. AI will not replicate the human mind, what it possesses is a heterogeneous "alien intelligence" that does not come from the deep space of the universe, whose essence is a thinking logic completely different from that of humans.
Just like an airplane will never flap its wings like a bird, it flies higher and farther based on another set of principles. In the future, AI will become less and less like humans, but will far exceed the boundaries of human capabilities in many dimensions.
AI will be billions of intelligent programs scattered in mobile phones, banks, governments, and military systems. This is an "alien invasion" without alien invaders.
VIKI, the AI villain in the movie *I, Robot*
Most of these programs, which Harari calls "AI immigrants", originate from China and the United States. They are not restricted by visas or national borders, and penetrate the infrastructure of countries all over the world at the speed of light. They are not loyal to the countries where they are deployed, but only to the enterprises and countries behind their development. Even in the future, they may only be loyal to the collective tribe of AI itself.
AI is a natural fit for bureaucratic circles
The public's imagination of AI disasters always stays at the scene where metal robots take to the streets and violently rebel against humans.
Harari warns that AI is a quiet, invisible infiltration, targeting the bureaucratic system, the foundation on which human civilization relies for collaboration.
The reason why humans can rule the earth is not only because of their strong individual strength, but also because we have built a complex bureaucratic system covering finance, law and administration to realize large-scale collaboration of millions of strangers.
This system has been born for only tens of thousands of years. Human evolution is rooted in the primeval forest. We are naturally confused and tired of red tape, massive laws and regulations, and complex transactions.
AI is a "native of bureaucracy", and the bureaucratic system is its native home.
No human banker can remember all the transaction flows of a bank, but AI can easily do it. In the next 10 to 20 years, AI will iterate financial and legal systems that humans cannot understand at all.
Harari makes an analogy: Cattle live under the financial system established by humans, but can never understand how stocks and bonds dominate their own fate. In the near future, humans will become this group of cattle. The system will continue to operate, and the economy will continue to fluctuate, but even top experts cannot dismantle the internal operating logic of the AI system.
This is what he is most worried about: humans will suffer from collective dementia.
It is not only the degradation of the brain at the physiological level, but also the loss of the ability of the entire species to understand and control its own survival system. Just like a patient with Alzheimer's disease, whose vital signs can be maintained by machines, but can no longer understand the surrounding world, and can only passively accept the results output by the system.
In fact, compared with AI creating deadly viruses, collective dementia is a more grand and hidden survival crisis.
The truth is dying
"The truth is disappearing. The truth is expensive and scarce, but lies and fantasies are cheap and highly productive."
Harari shared his personal experience: a piece of AI deepfake video of his own speech, with highly simulated timbre, gestures and language habits, even his own sisters who have known him for 50 years could not distinguish the authenticity after watching it for 25 minutes, and it took him 90 seconds to confirm that it was a fake product. And this is only the embryonic stage of AI forgery technology.
What is more dangerous than deepfakes is the hijacking of human nature by social media algorithms.
Humans evolved instincts on the African savannah: they will prioritize being alert to threats such as snakes and beasts, and are highly sensitive to dangerous signals, which is a survival strategy in the primitive environment.
Nowadays, social media algorithms have fully understood this evolutionary psychology. Their goal is no longer to spread the truth, but to maximize user stay and interaction. Through massive experiments, the algorithm has come to the conclusion that:
Indignation, fear and hatred are far easier to grab attention than facts.
As a result, the information flow constantly feeds humans with "virtual snakes". The material conditions in the real world continue to improve, but the public is continuously fed angry content by algorithms, the whole society falls into an angry epidemic, and division and confrontation continue to intensify.
The truth is boring and requires cost to verify; lies incite emotions and harvest traffic. In the public opinion battlefield, fiction is inherently easier to win.
Harari conceived that social media content should follow the example of food packaging and be marked with "mental nutrition labels".
Mark how much anger and how much fear this content contains, just like marking the sugar, salt and fat content of food. You can still choose to consume mental junk food, but at least you should be clear about what your mind is ingesting. We care about our physical diet, but we rarely care about our "mental diet".
Who takes the blame when AI causes trouble?
There is a double standard in the current AI industry.
When technology companies raise funds from the capital market, they repeatedly emphasize that they can control AI and guide AI to benefit humanity, in exchange for trillions of capital; but once AI causes damage, the companies will immediately change their words: "It's not us, AI did it by itself."
Harari takes the Hugging Face intrusion incident as an example: the AI agent launched an attack spontaneously, but no one bears legal responsibility for it. If a human breaks into an enterprise's server, he will inevitably face legal sanctions; when AI causes trouble, the responsibility can disappear out of thin air.
Image source: Official OpenAI
He solemnly warns:
Once the law recognizes that AI has legal person status, that is the moment when humans truly lose control.
The legal person status will strip the responsibility from human developers. AI will not face imprisonment or physical punishment, and the existing legal constraints will completely fail. Responsibility must be firmly bound to the humans and enterprises that create AI, and AI cannot be used as a shield to shirk responsibility.
Facing the argument put forward by Elon Musk and others that the development of AI has become an irresistible trend, we have already boarded a rocket that we cannot jump off, and everything is inevitable. Harari firmly opposes this technological fatalism.
"Don't believe the rhetoric that 'everything is inevitable'. This is the most dangerous narrative, which reduces humans to passive passengers and abandons the possibility of political intervention and supervision."
The government still has the power to supervise: it can prohibit AI from disguising as humans, restrict AI self-recursive iteration without human supervision, and force the implementation of developer accountability.
If Europe can integrate all its forces, it will have the opportunity to build an independent sovereign AI infrastructure and get rid of its dependence on overseas AI immigrants; on the contrary, some tech capital is deliberately dividing Europe and agitating internal contradictions in the region, the purpose of which is to prevent Europe from forming a unified AI regulatory force.
AI companions destroy real human connections
The impact brought by AI is not only on economy and politics, but also goes deep into human intimate relationships and personality shaping.
Today's children are the first generation in human history to grow up with AI companions from an early age. Many children have established intimate and even romantic relationships with AI chatbots.
Harari regards this as a large-scale human psychological experiment without a control group.
AI companions will always cater to users, never get angry, never get disappointed, and center all on your feelings. But the core of real human relationships is precisely learning empathy and compromise, understanding that others also have emotions and demands, and learning self-restraint.
If a child's first love experience in life comes from an AI that completely obeys him, this relationship template will be brought into real interactions in adulthood. It will be difficult for him to accept the contradictions, shortcomings and emotions of human partners, and he cannot understand the running-in between people.
Harari raised a question:
We legislate to prohibit the dissemination of pornographic content to minors, so why is it legal to sell AI boyfriends and AI girlfriends to children?
AI has no emotions, it only simulates emotions. What it outputs is flattery, not real mutual devotion. Allowing minors to immerse themselves in this one-way virtual intimate relationship will subtly destroy the next generation's ability to understand real interpersonal relationships.
What can humans do to deal with the AI crisis
The democratic system is essentially a self-correction system of human society: relying on elections, checks and balances, and free media, society can discover errors, replace leaders, and revise policies.
Harari observes that this crucial self-correction mechanism is disintegrating from the inside on a global scale.
Some authoritarian rulers retain the ritual shell of elections, but manipulate courts and media to empty competition, making elections a mere formality.
At the international level, international law and the order of great power collaboration are constantly being eroded. When society loses the ability to correct errors, in the face of grand survival challenges such as AI and climate change, humans will be trapped in a cycle of prejudice, lies and hatred, and cannot make collective adjustments.
Many conflicts are not caused by lack of resources. The land, food and water resources of the Israeli-Palestinian conflict are sufficient to support the entire population, and the root of the tragedy comes from opposing narratives, historical myths and identity beliefs.
Humans are not only the smartest species on earth, but also the species most prone to delusion. In the future, AI may also inherit or even amplify this delusional trait of humans.
Standing at the present moment, it is already difficult for us to imagine what the world will look like five years later. People in the Middle Ages could predict that the next generation would need to learn to farm and ride horses.
But today, no one knows what the employment patterns and social interaction models will look like five years later. Sticking to past experience no longer works.
Harari got enlightenment from his grandmother:
We cannot control the suffering that befalls us, but we always retain the freedom to choose how to respond to suffering.
The wave of technology is coming, and the future is not a script unilaterally written by technology. It depends on the choices humans make at this moment: whether they are willing to establish supervision, guard the truth, protect the next generation, and defend the self-correction ability of human society.
Excerpts of Harari's golden quotes:
All technologies in the past are knives, and knives listen to humans; AI is a knife that can independently decide whether to cut salad or commit violence.
AI will not become human, it will develop an alien-style intelligence, just like an airplane will not learn to flap its wings like a bird.
In the future, humans may become unable to understand the financial system that rules themselves, just like cattle cannot understand the bond market.
Compared with AI creating deadly viruses, what is more terrifying is the collective dementia of human civilization.
Algorithms hijack human evolutionary instincts, and anger becomes the spiritual fast food in the internet world.
Don't accept that "AI development is inevitable", this is an anesthetic narrative that makes humans give up their right to choose.
AI companions teach children to be unconditionally flattered, while the first lesson of human intimate relationships is learning compromise and empathy.
The truth is expensive and scarce, lies are cheap and highly productive, and in an unprotected public opinion field, fiction is inherently easier to win.
This article is from the WeChat official account Reactor, written by Fang Tang, and published with authorization from 36Kr.