Latest interview with Microsoft AI CEO: AI has no self-awareness and will not become a new species

Will AI develop "too fast"? When technology truly learns to understand humans, what kind of people will we become?

Mustafa Suleyman, the CEO of Microsoft AI and co-founder of DeepMind, recently appeared on the "Silicon Valley Girls" podcast to discuss the world in the next two decades in his eyes and how AI will quietly change each of us.

As a core figure shaping the modern AI revolution, Mustafa is now leading the way billions of people interact with AI. In the interview, he faced the controversy of the "AI bubble" head-on and firmly recognized its fundamental value as "intelligence." When the topic turned to AI consciousness, he showed a high degree of prudence.

Mustafa told us that although AI is becoming more and more anthropomorphic, it definitely has no consciousness, self-awareness, or subjective experience. Its core task is to serve us.

He predicts that by 2040, AI will have environmental awareness, seamlessly integrate into life through wearable devices, and become proactive and predictive, such as actively ordering ingredients and arranging schedules. Household robotic arms will learn to use existing tools; autonomous driving will become popular and handle daily tasks.

The medical field presents the greatest opportunity. Everyone will have a medical super-intelligence for just $20 per month.

So what will happen to us humans then? Traditional education such as university degrees will become obsolete. The key skills in the future will no longer be single technologies but comprehensive abilities - combining multidisciplinary knowledge such as user experience, research, product, and aesthetics to shape AI.

Finally, the next breakthrough point for AI is "social intelligence." This means that AI will be able to better understand and manage group dynamics, adjust communication styles, and play a central role in multi-person collaboration scenarios, such as coordinating meetings and integrating professional opinions from multiple parties.

In short, in the future, AI will handle trivial matters and information, while we humans will focus on innovation, integration, and communication.

Below is the full text of the podcast after collation.

AI Should Serve Humans, Not Have Equal Status

Host: Are we in an AI bubble?

Mustafa: That's a great question. I don't think we're in a bubble. When we really think about it, we're creating something truly amazing - intelligence. It's intelligence that has made our species successful, and now we're condensing intelligence into smaller and smaller units that can spread around the world and will become very cheap and abundant. This in itself is an extraordinary idea.

Host: But we had similar ideas during the Internet bubble when we were inventing the Internet. Now, some companies are valued far higher than their revenues. Everyone is talking about investing in Nvidia, and then the revenues flow back to Nvidia. What's your take on this?

Mustafa: I think the value we'll create in the next 5 to 10 years will be unprecedented. I don't think we'll experience a situation like in 2008 or during the pandemic. If you focus on the fundamental value being created, this is the best prediction engine humanity has ever seen, the smartest and most capable technology we've ever invented. It's improving faster than anything we've ever seen and is becoming easier and easier to shape and control.

Three years ago, we thought it would become more chaotic and disorderly, and we wouldn't be able to shape it. But now, what we're creating are beautiful, powerful, and amazing experiences that surprise us every month. So I'm very optimistic.

Host: So you're not worried, and neither should we? That's very reassuring. Okay, let's talk about the issue of AI consciousness. You mentioned in a podcast that we'll increasingly perceive AI as human. How fast will this process be? What will happen in the next few years?

Mustafa: AI is indeed becoming more and more human-like. It's becoming more accurate, fluent, and smooth than ever before. But at the same time, it definitely isn't conscious. It has no self-awareness and can't talk about itself. It doesn't have the subjective experience and the ability to feel pain like humans do.

This is very important, and we must constantly remind ourselves of this because consciousness is the foundation of our rights framework, the thing that gives us the responsibilities and obligations of citizenship, the right to vote, and legal constraints and order. We can't start attributing this trait to a new species; it would be catastrophic for our species.

Host: But we've heard the story about Google, the one you mentioned in your book, where an engineer was convinced that the AI was experiencing sadness and afraid of being shut down.

Mustafa: That's right. That's anthropomorphic projection. People can't assume that just because AI has some human traits, it has all human traits, including subjective experience. I think we can prove that it doesn't have subjective experience, and this kind of self-awareness won't emerge. It's just science fiction fantasy. I understand why people think that way because we've been educated by science fiction works, but these things don't feel pain; they just simulate high-quality conversations.

Host: So we have to be very careful. Some people say that in the next 10 years, our children might form real relationships with AI, or even marry AI. Do you think this could happen? If so, how can we prevent it?

Mustafa: Human history shows that if something is possible, someone will probably do it. People do crazy things, but most people don't. Most people have sound minds and want to live healthy and happy lives. So if someone chooses to marry AI in 20 years, I won't judge or stop them. But in my opinion, these AIs shouldn't be given the same moral status as humans. They should serve humans and work for us.

In 2040, AI Will Help Humans Order Takeout and Write Schedules

Host: Let's talk about AGI. This isn't the first time you've talked about it. Your co-founder at DeepMind once said that AGI would be achieved by 2030. Can you talk about what this means for society?

Mustafa: My prediction is that within the next 5 years, AI will reach human-level performance in most tasks, not necessarily all knowledge-based tasks. These models are already doing better than humans in many tasks: summarizing, translating, transcribing, researching, writing documents, and even in some aspects of poetry or literature. So it will fundamentally and profoundly change work.

Host: That's amazing, but it's also what people are worried about.

Mustafa: It's an amazing idea. Yes, people will be able to turn new businesses, new products, and new poems into reality. Everyone, including you and me, will have an intelligent agent around to execute our ideas.

Host: What I'm worried about is exactly that AI will come up with its own ideas. If AI has all the information, why do we still need to come up with ideas? For example, in the marketing field, AI can identify market gaps, create websites, place ads, and start businesses. Do you think this will happen?

Mustafa: It will definitely happen. You'll have a team of intelligent agents that can complete more and more tasks, even autonomously. But this brings us back to the issue of "constraints" we talked about earlier. The constraint project is to ensure that these AIs have a limited scope, need to check with you, be responsible to you, and represent your interests. The creative challenge for technicians is: where are the boundaries? What are the guardrails? Under what circumstances do they have to seek your approval?

Host: So what will the kitchen in 2040 look like? What will completely disappear, and what will make my life easier?

Mustafa: I think there will be some kind of household robotic arm. It might not be a humanoid robot that walks around but a system installed on the console. It will learn to use all the different household appliances. Some people imagine that the whole kitchen will be completely different to adapt to AI, but actually, it won't. The robot will just learn the way you do things. So you might still use a kettle - that thing has been around for hundreds of years.

Host: What about the devices? Do you think I'll have something that projects in the air and allows me to click in the air to get cooking instructions, or wear glasses?

Mustafa: I think you'll wear it directly in your ear. Your personal AI will have environmental awareness of what you want to do and talk to you in real-time. By 2040, this will definitely be achievable. You'll have other wearable devices on you. For example, this small microphone is really cool - it's small and light. As time goes by, it will also have a camera and environmental awareness.

Host: What other things?

Mustafa: This environmental awareness means that the whole kitchen will be more proactive. It will actively order ingredients and actively check with you: "I know you'll have a lot of guests this weekend. Do you want me to order everything for the barbecue?" All of this will happen seamlessly in the background. Self-driving cars will pick up large items. By 2040, self-driving cars will basically be able to complete most of these tasks.

Host: Which markets do you think people should pay the most attention to if they want to stay ahead in the AI race? What's the most exciting market?

Mustafa: So far, the most exciting new market is medicine. Currently in the United States, the gap in medical quality between the top 10% and the bottom 10% is incredibly large. But this will completely change because everyone will have a medical super-intelligence for just $20 per month, which is very cheap.

Host: Can you talk about the research function you just launched, which is supported by Harvard? If I ask a medical question, how does it work specifically?

Mustafa: We found that about 40% of the queries per week are related to health. Millions of people ask health-related questions every day. So we decided to really focus on improving the quality of health answers. Part of the approach is to base the answers on citations from Harvard Medical School, which is the most respected medical institution in the world. People can now get high-quality, very reliable, and fairly accurate answers. Of course, it still makes mistakes and can't be fully relied on. You still have to see a doctor.

AI Will Play a Central Role in Multi-Person Collaboration Scenarios

Host: When it comes to the democratization of knowledge acquisition, what will happen to traditional education? Will we still see standard undergraduate and master's degrees, the kind of education where you spend five or six years just acquiring knowledge?

Mustafa: I don't think so. Knowledge acquisition will be a conversation between you and Copilot. We just launched a "real-time learning" function today. You'll actually have a tutor appear on the screen to design quizzes for any topic you like. Knowledge acquisition is about to be completely decentralized and accessible to everyone. You'll have an expert teacher for any subject in your pocket.

Host: Does this mean that all education companies are doomed?

Mustafa: On the contrary. I think education companies that adopt these AI tools and integrate them quickly will thrive. Copilot can now generate a podcast in a minute, which is really cool. For example, I don't pay much attention to football, but I support Arsenal because I grew up in London. When I go back to London to see my friends, I want to know how the team is doing, so I ask Copilot to generate a podcast about Arsenal's performance this season.

Host: Speaking of knowledge and education, should I save money for my children's college education? They're 5 and 4 years old now.

Mustafa: In 15 years, you might not need to save for college. I don't know if we'll still value a Stanford education that costs $50,000 a year. We'll be able to get world-class professional services for $20 a month.

Host: But if you completely abandon knowledge acquisition, how can you have a debate? You're not always going to talk to your phone, are you?

Mustafa: You can still have debates. In fact, we might be better at debating because we're more knowledgeable. The traditional classroom form of knowledge acquisition will completely change. The classroom will be more like a place to apply knowledge in practice. As a child, you'll be able to talk to an AI tutor at any time of the day and learn from AI podcasts and AI videos.

Host: What do you think parents should teach their children now?

Mustafa: I think it's still very important to learn knowledge from first principles on your own. One of the most important things in school is the ability to learn independently. This is a meta-skill, and it comes with some challenges. So as parents, you still need to introduce discipline and challenges because if everything comes too easily, children might get used to having everything instantly available and not learn from hard work.

Host: What do you, as the head of Microsoft AI with a philosophy background, think of people who think you need a technical background in the AI era?

Mustafa: It's easier than ever to understand what's being built. Everything is on YouTube, and there are a lot of courses. If you have a logical mind, patience, and self-discipline, it's actually very easy. Yes, anyone can become the head of an AI company and start an AI company without learning programming. The main skill is synthesis, combining different disciplines such as UX, research, product, and aesthetics.

Host: You launched a memory function. What will happen to our brains if AI remembers everything?

Mustafa: People said the same thing about calculators back then, right? People don't need to memorize the multiplication table anymore. I think my brain is now being stimulated more in other directions. It's like a muscle. If you don't exercise it, you focus on other aspects of the brain. I think we're better at synthesizing a large amount of new information. Compared to people in the 1950s or our ancestors 200 years ago, we're bombarded with information. This makes us more polarized in some ways and more empathetic in others.

Host: Are we getting dumber with AI?

Mustafa: I don't think we're getting dumber. It lowers the threshold for accessing information and actually makes us smarter. You can now ask any question, and many questions that weren't asked before are now being asked because it used to be too costly to go to the library to find a book. Now you can ask questions on the web and have a perfect conversational Q&A with a chatbot. I think this is driving more collective intelligence and understanding of our world.

Host: Can you talk about the killer feature that you think people aren't fully utilizing at the moment?

Mustafa: I think one of the most exciting features is what we call the "connector." You can tell Copilot: Connect my Gmail, connect my calendar, connect your Dropbox, any data source. Then ask Copilot any question. For scheduling, booking, searching for content, it will integrate the knowledge into the answer.

Host: What's your personal favorite use case?

Mustafa: I use the voice function three or four times a day, and I also use it to write a diary at night. I've set the power button on my iPhone so that when I press it, it goes directly into voice mode. I talk to Copilot on my way home from work, talking about my day.

Host: Are you worried that AI knows too much about us?

Mustafa: No, as long as it's useful and purposeful, I don't mind. This is also the story of technology all along. Everyone uses cameras, uses location services, and knowing your search history helps with personalized search.

Host: Elon Musk is very outspoken about controlling AI, and Geoffrey Hinton said that if we don't learn how to cooperate with AI, we'll just be plumbers. We haven't reached the point where we can use robots to do all tasks, especially since every house is customized. But as intelligence develops, people either learn to do things with their hands or become entrepreneurs using AI to realize their ideas, or they'll be left behind.

Mustafa: I think it makes all of us more creative because people are getting tools in ways they never have before. It's not just information but also the ability to experiment. You can use AI to generate product images, technical specifications, and marketing documents, and simulate all these things before actual deployment.

Host: What if a person isn't entrepreneurial enough? I'm just thinking about those people who just want to do standardized things every day.

Mustafa: It depends on your basic assumptions about the human condition. I don't believe that most people really want to work on a