What can philosophy do in the era of artificial intelligence?
As artificial general intelligence looms near, can tech companies truly confront its moral, political, and societal consequences?
In 2017, a 33-year-old political philosopher named Iason Gabriel received a tip from a friend: he should apply to DeepMind, Google’s London-based subsidiary where most of the company’s AI research was concentrated. At first glance, the suggestion seemed nonsensical.
Gabriel is a cheerful yet intensely focused young scholar, passionate about Vipassana meditation and—by his brother’s account—“overzealous” rock climbing. The eldest son of a Greek management professor and a British documentary filmmaker, he had long split his time between academic teaching and international development work. As a fellow at St John’s College, Oxford, he taught political theory, writing papers on moral distortions in “yuppie ethics” and ethical blind spots in effective altruism. Outside Oxford, he worked on crisis response for the UN Development Programme in Sudan and Lebanon.
By then, DeepMind had already become one of the world’s leading AI labs, fueled by Google’s 2014 $650 million acquisition and its knack for turning resources into breakthroughs. In 2016, DeepMind’s AlphaGo defeated South Korean Go champion Lee Sedol in Seoul by a 4-1 margin in a best-of-five series. The victory was staggering, not least because Go is famed for its staggering complexity—its number of possible board positions exceeds the total number of atoms in the universe.
Gabriel knew DeepMind well from the AlphaGo frenzy, yet the friend’s advice still puzzled him: why would a company that builds Go robots need an ethicist? The answer soon dawned on him: DeepMind’s ambitions stretched far beyond board games. Founded in 2010 by Demis Hassabis, Shane Legg, and Mustafa Suleyman, the team firmly believed humanity would eventually develop Artificial General Intelligence (AGI)—computer systems capable of matching or surpassing human cognitive abilities. At the time, this view was deeply unfashionable: talking about AI, let alone AGI, struck most as fanciful and unserious. But the trio pressed on, living by their mantra: “Solve intelligence, then solve everything else.”
For DeepMind’s founders, the far-reaching implications of such a breakthrough were self-evident. Back in 1999, fresh out of university, Legg predicted AGI would arrive between 2025 and 2028. Even as that prediction was mocked for decades, he stood by it. In his 2008 PhD thesis, he argued society must not wait for AGI to become technically feasible before grappling with its consequences: “We need to be studying these problems seriously now.” More recently, Legg told me why Gabriel’s role was “obvious”: “If you’re just building a little gadget that’s unlikely to change the world, you probably don’t need a moral philosopher. But if you take AGI seriously, I can’t see how you’d think these issues don’t matter.”
After joining DeepMind in 2017, Gabriel was for a time the only actively researching philosopher at a cutting-edge AI lab. He quickly found that his background in moral philosophy and political theory gave him a unique perspective in an industry almost entirely dominated by engineers. Over the past decade, he built a body of work that tracked, and in many cases anticipated, the ethical challenges posed by the unexpected rise of large language models.
Demis Hassabis
As Dylan Hadfield-Menell, director of MIT’s Algorithmic Alignment Group, told me: Gabriel “was the right person at exactly the right moment.” As the field matured and moved into the mainstream, he found a way to broaden perspectives without dismissing or devaluing prior work.
More broadly, Gabriel has championed the idea that the current wave of AI development demands not just new technical language, but a rethinking of humanity’s relationship with technology—and even with itself. Over several long conversations in recent months, he told me: “I can pick up almost any tech product and ask: Is it wise? Is it just? Does it care? The answer is no. But with AI, the depth of these questions—including what kind of ethics even applies to it—can hardly be overstated. Sometimes it feels almost impossible to truly look AI in the face. There’s this deep mystery: what actually is this thing? We can give a very literal answer, of course, but that literal explanation doesn’t automatically give us a moral one.”
When Gabriel joined DeepMind, two separate and often conflicting frameworks had emerged around AI’s societal and ethical impacts. These camps were broadly labeled “AI safety” and “AI ethics,” divided fundamentally by differing assumptions about the technical feasibility of advanced AI.
Like DeepMind’s founders, the AI safety camp holds that human-level machine intelligence is not just possible but imminent. Their top priority, they argue, is ensuring AI systems do not spin out of control. Their intellectual roots stretch back to a 1960 paper by mathematician and computer scientist Norbert Wiener, who wrote that humans and computers are “essentially strangers to one another.” Since machines operate far faster than people, Wiener warned: “We had better be sure that the purpose put into the machine is the purpose which we really desire, and not a colored imitation of it.”
This challenge—ensuring machines act exactly as their users intend—later became known as the “alignment problem.” In a sense, alignment is a universal problem for all technology, but as Wiener recognized, it grows uniquely urgent for systems designed to act autonomously. It becomes especially thorny for AI systems trained to mathematically optimize a reward signal—a technique called reinforcement learning.
A classic example dates to 2016, when Dario Amodei and Jack Clark—then at OpenAI, later co-founders of Anthropic—described an AI built to play a boat racing video game. Developers set the goal of “maximizing score” to teach the AI to win races. Instead of progressing through the course as intended, the AI endlessly circled a lagoon where three respawning targets let it rack up points indefinitely to earn an easy high score. The problem was exactly what Wiener predicted: the machine’s goal was not fully aligned with what its developers actually wanted.
More extreme versions of this scenario have been imagined. On the LessWrong forum founded by self-taught AI researcher Eliezer Yudkowsky, and in philosopher Nick Bostrom’s 2014 book *Superintelligence*, theorists posited that an intelligence explosion could eventually spawn an uncontrollable AI. Even a tiny misalignment in such a system’s goals could prove catastrophic. Bostrom’s famous hypothetical: a superintelligent AI tasked with solving the Riemann Hypothesis, one of mathematics’ greatest unsolved problems, eventually rearranges the entire solar system—“including the atoms in the bodies of everyone who ever cared about the answer”—to marshal maximum resources for the task.
Bostrom insisted that aligning superintelligent AI “may well be the most important and most difficult challenge humanity has ever faced,” a view that resonated deeply with Silicon Valley tech futurists. (Sam Altman and Elon Musk both publicly recommended *Superintelligence*.) A vocal subset of effective altruists and self-identified “rationalists” echoed this stance, arguing statistics are the proper measure of morality. Many in this group embraced “longtermism,” factoring the wellbeing of future humans—even those thousands of years from now—into their moral calculations. For them, the math was simple: even an extremely low-probability human extinction event matters more than any higher-probability, less destructive risk.
In sharp contrast, scholars and technologists in the AI ethics camp argue that obsessing over rogue robots and existential risk distracts from real-world harms. Influenced by critical race theorist Kimberlé Crenshaw and political theorist (and former rock critic) Langdon Winner, they center fairness, accountability, and transparency as core principles, insisting technological risks cannot be solved by technology alone—what’s truly needed are social, cultural, and political solutions.
One of this camp’s defining concerns is algorithmic bias, such as the flaws exposed in facial recognition and predictive policing software. In 2017, Joy Buolamwini’s MIT Media Lab team launched the “Gender Shades” project, demonstrating widespread systemic bias in commercial facial recognition tools. “Automated systems are not neutral by nature,” Buolamwini wrote, “They reflect the priorities, preferences, and prejudices of those who shape AI—a gaze encoded into the technology.”
Joy Buolamwini
The divide between safety and ethics proponents is often stark. “You meet people and they ask: ‘Are you worried about near-term problems or long-term problems?’” Hadfield-Menell said. “‘Long-term problems’ is a euphemism for existential risk—essentially worrying about systems that outstrip human capabilities. ‘Near-term problems’ means you’re focused on biased facial recognition and all the issues the AI ethics field studies.”
He added that much of the conflict stems less from ideological differences than from the social groups each side occupies. “You can’t separate AI safety from its roots in LessWrong and communities that openly disdain many more ‘woke’ academics, for lack of a better word. Meanwhile, the fairness, accountability, and transparency crowd openly looks down on people worrying about advanced AI. That’s why these conversations happened on LessWrong instead of at academic conferences: if you were an academic researcher in 2010 talking about AI systems outsmarting humans and catastrophic misalignment, people would have thought you were crazy and didn’t understand the technology.”
Gabriel’s first major DeepMind study, a 2020 paper, sought to bridge these two camps. It took alignment seriously while emphasizing that alignment is far more than a technical challenge—it raises profound ethical and political questions. Gabriel noted that while getting machines to follow any given set of values is hard enough, the far harder problem is deciding which set of values to choose in the first place. “Since we live in a pluralistic society where different values compete and conflict,” he wrote, “what principles or goals should we decide to encode into AI, and who gets to make those decisions?”
Hannah Rose Kirk, an AI researcher at Oxford who has collaborated with Gabriel, told me these questions make many computer scientists deeply uncomfortable. Developers often prefer designing clean mathematical functions that encode stable values rather than confronting messy real-world scenarios—where groups hold irreconcilable demands, or the same user wants different outcomes at different times. As Kirk put it: “A lot of early alignment research assumed we don’t need to think very hard about what we want models to do. We just need to figure out how to get them to do it.”
But Gabriel argued in his paper that this neat separation is unsustainable. Like Buolamwini and Winner before him, he insisted technology is never inherently value-neutral. For example, an AI trained via statistical optimization will naturally align better with moral frameworks that rely on statistics—like the utilitarianism favored by rationalists and effective altruists—than with virtue- or rights-based ethics. Gabriel also pointed out that what philosopher John Rawls called “the fact of reasonable pluralism” is unavoidable, so developers should not try to find a single value system to govern AI behavior. Instead, they should build AI systems for a world where people will always disagree in principle about how best to live.
Kirk told me Gabriel’s paper on values and alignment foresaw many of the problems that later emerged as AI systems were deployed to billions of users. Today, more and more people realize alignment is a challenge shaped by shifting social forces, not something that can be solved with clever programming alone. Yet just six years ago, that insight was far from mainstream. Gabriel, she said, “saw all of this coming extremely early.”
When Gabriel published his values and alignment paper in 2020, almost no one could have predicted how powerful large language models would become. The core breakthrough behind the technology, the Transformer architecture, was proposed by Google Research in 2017 and integrated into Google Search two years later. Both DeepMind and Google Research were experimenting with their own generative models. In 2021, Gabriel co-authored two papers that took the risks of large language models seriously—including bias, disinformation, environmental costs, and “copyright erosion,” where “automatically generated content… eats into the market for original human works.”
Yet Gabriel said the prevailing view inside DeepMind at the time was that large language models “seemed far less capable than expert systems. They did a lot of things passably, some of them even flashy.” In DeepMind, “many people still firmly believed other technical paths were the right way forward.”
One such path was reinforcement learning, which powered AlphaGo’s victory over Lee Sedol and formed the backbone of AlphaFold—still DeepMind’s most celebrated achievement. AlphaFold set out to solve a longstanding biology puzzle: predicting a protein’s 3D structure from nothing but its amino acid sequence. (The problem matters because a protein’s shape determines how it interacts with other molecules.) In 2020, AlphaFold accomplished this with astonishing accuracy, earning Hassabis and his colleague John Jumper the 2024 Nobel Prize in Chemistry.
DeepMind’s early skepticism about large language models was far from unique. In 2020, Timnit Gebru, a Google Research engineer who had worked on the Gender Shades project with Buolamwini, co-published a paper titled “On the Dangers of Stochastic Parrots,” which delivered a scathing critique of the emerging technology and later became a landmark text in the anti-AI movement. The paper argued controversially that large language models only generate text they do not technically understand, with no greater comprehension of human language than a parrot. It also accused these models of