See clearly the problems that AI cannot solve
The ability of AI to solve complex problems depends on several factors, which may increase (or decrease) the effectiveness of AI. Among them, the most critical factors include: the availability of high - quality data, the number of potential solutions, the clarity of the goal, and the need to adapt to a constantly changing evaluation system. Challenges arise when these elements are missing or poorly defined.
The development of AI is advancing by leaps and bounds. Its potential can bring about transformation, fascinating researchers, product teams, and end - users alike. However, despite the great popularity of AI, it can only solve very specific problems. In this article, we will provide guidance to help enterprises understand which problems are suitable for AI solutions and which are not. In this way, when the problems faced by enterprises do not align with the capabilities that AI excels in, enterprises can still find solutions. After overcoming these obstacles, more AI projects that can bring greater benefits to enterprises and society can be found.
Take Google DeepMind as an example. DeepMind is an AI laboratory of Alphabet, dedicated to solving extremely challenging real - world problems. Three of the authors (Paolo Cervini, Chiara Farronato, and Marshall W. Van Alstyne) engage in academic research on the development of technology platforms and have also carried out innovative practices in multiple fields of AI, from which they have gained professional knowledge. In addition, Pushmeet Kohli, one of the authors, works at Google DeepMind and has first - hand experience.
The following content can be used to determine whether AI is suitable for solving a certain problem, but we do not provide a list of all judgment criteria. Especially considering that these judgment criteria may vary depending on different AI application fields. Our main purpose is to find the most critical judgment criteria.
Can AI solve this problem?
The ability of AI to solve complex problems depends on several factors, which may increase (or decrease) the effectiveness of AI. Among them, the most critical factors include: the availability of high - quality data, the number of potential solutions, the clarity of the goal, and the need to adapt to a constantly changing evaluation system. Challenges arise when these elements are missing or poorly defined. We need innovative methods to overcome these challenges. The following are the measures to be taken when these four elements and challenges occur:
1. Lack of high - quality data
Data is the most critical input for any AI model, but the scale of data is often more emphasized than its quality. Although the development trend of large language models (LLMs) shows that increasing the amount of data can help obtain high - quality models and output results, it remains to be seen whether this trend will continue.
Facts have proven that data quality is as important as, or even more important than, data scale. In some cases, if you start with a relatively small but high - quality database, you can even increase the scale of data by generating virtual data.
This was the situation initially faced by the team researching AlphaFold. AlphaFold is a breakthrough technology of Google DeepMind that can accurately predict the three - dimensional shape of protein structures. This technology is accelerating research in almost all fields of biology.
In 2017, when AlphaFold was in its initial development stage, the development team only had about 150,000 data points of 3D protein structures. These structures were experimentally determined over several decades through expensive and time - consuming techniques (such as X - ray crystallography or cryo - electron microscopy). (Studying a single protein structure requires at least one doctoral student.) For a machine - learning system, this is a very small dataset.
However, the research team successfully expanded this data by predicting the structures of unknown protein sequences. They built a prototype version of AlphaFold to achieve this. This version was powerful enough to generate predictions for one million newly folded proteins.
Then, the team used this version of AlphaFold to evaluate how accurate its own predictions were. Through this self - refinement process, the team added the top 30% of the predicted results back into the dataset for training the model, along with the real data. In this way, the scale of the dataset was expanded to about 500,000 folded proteins, which was sufficient to train a final version of AlphaFold.
It should be noted that although this training method was very effective in that situation, great caution must be exercised when using generated virtual data to train models, as there is a risk of recursive learning: using the prediction results of past model iterations as input to influence subsequent iterations will amplify the errors and inaccuracies passed down from the previous round of model iterations, leading to a decline in model performance.
However, if the dataset used at the beginning is very large but of low quality, it will be very difficult to improve data quality through synthetic data. Data scraped from the Internet, which may contain various information, formats, languages, topics, and methods, belongs to a dataset with large scale but low quality. In most cases, it is not worth trying to improve its quality because it requires a lot of effort and time. Moreover, in such a dataset, the model cannot find the “correct answer” for synthetic data. For example, when asking which recipe can make a perfect pizza, the most likely answer is “it depends”. Such nuances and the need for context make it difficult for the model to structure language and common sense, thus making it difficult to create a high - quality dataset.
2. Too many potential solutions
When a problem has too many solutions, it becomes impractical to solve it by the “brute - force” method of exhaustively testing all possible answer combinations. Historically, such problems have been solved using heuristic methods: using simple rules to find a “good enough” solution that meets most situations, rather than the optimal solution.
Applying AI is a promising option for complex problems with countless solutions. However, as the number of potential solutions increases, it becomes more difficult to verify the quality of each solution. For example, LLMs are good at generating creative and diverse answers, but they also face the problem of the “hallucination” phenomenon, that is, generating information that does not conform to facts. This undermines people's trust in them. Therefore, finding ways to ensure that the solutions generated by LLMs are correct and verifiable has become the key to promoting the development of LLM capabilities.
3. Lack of clear and measurable goals
The objective function (also known as the reward function) is the goal or output that an AI model tries to achieve. In other words, we need to ask the right questions of the model. Determining “what do you want the model to do?” is one of the most difficult parts of any machine - learning system. Take games such as chess or Go as examples. These games have clear and measurable goals, such as scores or a set of rules to determine the winner, and the cost of iteration is low, allowing AI to learn quickly.
However, in the real world, goals are usually complex and disordered, and we do not have a simple metric to measure progress. Without clear and measurable goals, it becomes difficult to define the standard of “good”. The more vague the goal, the worse the model's performance.
Let's go back to the example of AlphaFold. In biology, protein folding has a clear and measurable goal, and the goal of AlphaFold is to minimize the difference between the 3D folded protein structure detected experimentally and the structure predicted computationally. A good way to measure success is the Critical Assessment of Structure Prediction (CASP) competition, which is held every two years. This competition is similar to the “Olympics” of protein folding. It brings together the best teams from around the world to compare computational methods for predicting protein structures. Participating in this competition allows the research team of AlphaFold to obtain an objective “score” of the model's performance and compare it with other teams.
It is a challenge to give AI clear goals and measurement criteria. Especially in environments such as social media, “engagement” is often regarded as an indicator of the success of a project, measured by likes, shares, comments, or the time spent on the platform. Optimizing engagement can drive user activity and revenue, but it may also bring unexpected consequences. For example, it may amplify extreme ideas, spread sensational content, false information, etc., or encourage addictive behavior, harming the long - term well - being of users.
These pitfalls highlight the importance of designing a multi - objective optimization framework that can balance engagement with other key indicators, such as the accuracy of content, the diversity of viewpoints, and user satisfaction. Incorporating human feedback, ethical guidelines, and broader evaluation criteria in addition to simple judgment indicators helps AI generate meaningful and sustainable results while avoiding harm.
4. When “good” cannot be written into code
The problems faced by organizations are usually dynamic. Coupled with the aforementioned difficulty in simply identifying whether a solution is effective, AI may gradually deviate from the optimal answer. To overcome this challenge, more and more technologies are starting to adopt reinforcement learning with human feedback (RLHF).
This “human - involved” technology enables the model to learn from human insights beyond data. RLHF is particularly suitable for situations where it is difficult for AI to write solutions based on clear algorithms, but humans can intuitively judge the quality of the model's output.
The recent cooperation between DeepMind and the YouTube Shorts team is a good example. Since short - form videos are usually produced quickly, creators often add only a small amount of information to the video title or description, making it difficult for viewers to find the short - form videos they want to watch.
Generating accurate video descriptions is the problem that AI needs to solve: each video is unique, and the general descriptions generated by heuristic methods can only be used to roughly classify videos (for example, it is a sports video or a gardening video), but they are not sufficient to improve the uniqueness of the video at the individual level. However, detailed and personalized video descriptions are the prerequisite for viewers to find the content they like.
Humans are good at intuitively judging what a “good” video description is. Computers have always had difficulty doing this, so it is also difficult to write a “good” description for each video. By directly inputting human feedback into the next iteration of the model, the model can learn through trial and error, intuitively understand what kind of description is “good”, and continuously optimize it. Through continuous learning and adaptation to new information, the model can evolve with the changes in social aesthetics and norms.
Google DeepMind collaborated with the YouTube product team to deploy a generative AI model that can generate text descriptions for each video. These descriptions are stored in YouTube's system to provide users with better search results. This solution is now applied to all newly uploaded YouTube short - form videos.
Which opportunities should we seize?
Enterprises can use the above criteria to determine which problems are suitable for AI solutions. The next step is to make a choice from the large number of AI application opportunities that meet the criteria. Determining the priority of deploying AI based on the impact and scope of the solutions is not only useful for Google but also beneficial to society as a whole.
The DeepMind team calls this approach the “root node problem”. Imagine all the knowledge in the world as a huge tree, with branches representing different fields. In this tree, each “node” is a connection point, like a stepping - stone, leading to other branches. The root node is the starting point of the tree - the original position from which everything else grows. If a root node problem is solved, new research fields and understandings will emerge, and these new fields will open up brand - new paths for exploration and innovative applications.
The AlphaFold project fits this criterion well because solving the major challenge of protein folding can not only promote better drug discovery, such as malaria vaccines and cancer treatment drugs, but also contribute to the development of other fields, such as developing enzymes that can degrade plastics to address pollution problems.
After the release of AlphaFold, Andrei Lupas, an evolutionary biologist at the Max Planck Institute for Developmental Biology, wrote in the journal Nature: “This will change medicine, change research, change bioengineering, and change everything.” In 2024, the Nobel Committee awarded the Nobel Prize in Chemistry to Demis Hassabis and John Jumper in recognition of their great contributions to the development of AlphaFold.
To identify root node problems, companies should arrange discussions between AI experts and experts in other fields. The early DeepMind team already used this method. In addition to machine learning, the team intentionally hired a large number of talents from different disciplines, covering biochemistry, physics, philosophy, and ethics. This diverse professional background can stimulate the exchange of ideas between different disciplines and find new ways to solve problems.
Iterative innovation
Once an organization has identified a root node problem, the product team will strategically choose which AI to deploy and start the productization process.
Two major risks should be avoided during productization. First, only focus on the root node problems that the company has determined it can solve and the related fields. This approach may miss major opportunities. When AlphaFold was released in 2020, there was no suitable product team within Google to effectively utilize its research results (such as new drug compounds). In response, Alphabet created Isomorphic Labs, focusing on using AI to discover new drugs. This project can be established entirely by the company or achieved through strategic partnerships. The key is that the establishment of Isomorphic Labs integrated the research and product departments into a new company, which can promote the rapid development between the two.
The second risk is thinking that one can predict the evolution of technology and foresee new application scenarios. Instead of following a fixed path, organizations should retain their options, embrace uncertainty, allow the continuous evolution of AI solutions, respond to technological progress and user needs, and ensure continuous feedback collection throughout the process.
The way forward
As AI continues to develop, organizations must think carefully when deploying AI. This article provides a strategic framework to find problems suitable for AI solutions - these problems align with the advantages of AI, and solving them will open up surprising new paths for innovation and impact. By focusing on root node problems and promoting interdisciplinary cooperation, organizations can not only effectively use AI to drive business growth but also promote social progress.
Keywords: AI
Paolo Cervini, Chiara Farronato, Pushmeet Kohli, Marshall W. Van Alstyne | By
Paolo Cervini is a strategic consultant in the fields of AI, sustainable development, and corporate innovation. Previously, he was the content coordinator of the Italian edition of Harvard Business Review. Chiara Farronato is an associate professor of business administration at Harvard Business School, jointly leading the Platform Lab at the Digital Design Institute (D^3) at Harvard University. She is also a researcher at the National Bureau of Economic Research and the Centre for Economic Policy Research. Pushmeet Kohli is the vice - president of science and strategy at Google DeepMind, leading AI scientific research projects, including AlphaFold and SynthID. Marshall W. Van Alstyne is a professor at the Questrom School of Business at Boston University, a digital fellow at the MIT Initiative on the Digital Economy, and a visiting scholar at the Berkman Klein Center at Harvard University.
Zhang Yuxiao | Edited
This article is from the WeChat official account “Harvard Business Review” (ID: hbrchinese). Author: HBR - China. Republished by 36Kr with permission.