AI "face reading" came up with a predicted death list, and surprisingly half of the predictions actually came true.
"I see your glabella is dark, you are very likely to suffer from a violent bloody disaster."
In the past, after Fox Girl heard this sentence, her first reaction would be: Where did this so-called master come from?
Nowadays, AI has mastered this set of "face-reading physiognomy".
Moreover, it will not talk to you about wealth luck or romantic fate, it directly raises the difficulty to the maximum at the very beginning: With only a single face photo, it can predict the death risk of cancer patients.
On October 19, the top medical journal *The Lancet Digital Health* published a research paper, which made Fox Girl break out in a cold sweat after reading it.
The team from Mass General Brigham developed an AI foundation model named FAHR-Face, and trained a tool on this basis to directly predict the survival risk of cancer patients.
How remarkable are the results?
Among the 10% of patients judged by AI as "highest risk", half (52.5%) did not survive for 90 days; while among the 10% judged as "lowest risk", only 2.1% passed away within 90 days.
Can a face really reveal a person's life and death record? Is this a medical miracle, or cyber metaphysics?
01
Why do we have to "read faces" to determine life and death?
This study was jointly completed by Harvard Medical School, Mass General Brigham medical system and other institutions.
The team has a highly cross-disciplinary configuration: on one side is Raymond H. Mak, a radiation oncologist, and on the other side is Hugo J. W. L. Aerts, head of the medical AI program.
One is a radiation oncologist who deals with cancer every day, and the other is an expert in medical AI.
The two met and targeted an extremely critical problem: How do doctors judge whether a cancer patient can withstand the treatment?
It should be noted that cancer treatment cannot only depend on the tumor type and stage. The patient's physical foundation and degree of frailty directly determine whether they can receive radiotherapy, surgery or systemic treatment.
But the problem is that this kind of "physical state" can only be estimated by doctors based on their subjective experience, and it is difficult to be accurately quantified.
Therefore, the team targeted a zero-cost data source that everyone has but is often ignored: The face.
They developed the foundation model FAHR-Face, whose core logic is: Let AI learn to "read faces" first, and then teach it to complete different "specialized subject tasks".
To lay a solid foundation, the team adopted the Vision Transformer architecture and used a hardcore training method called "Masked Autoencoder".
During the training process, the system will randomly mask about 75% of the area in the photo, forcing AI to "fill in the blanks" and reconstruct the entire face based on the remaining limited information.
After this "super base model" is trained, two specific tools are fine-tuned on it:
FAHR-FaceAge: Estimate age from facial photos, and study the relationship between facial aging and survival outcomes.
FAHR-FaceSurvival: Directly analyze the face of cancer patients to predict their survival risk.
02
What exactly does AI see on the face?
How exactly does AI "judge life and death by reading faces"? In fact, the team completed the work in two steps.
The first step: Let AI learn to calculate age from faces.
They developed FAHR-FaceAge and trained it with more than 740,000 photos labeled with real age. In public tests, the age it calculated differs from the actual age by an average of only 5.1 years.
But being able to calculate age alone is nothing special.
What the team really wants to figure out is: Do the aging features on the face hide codes related to health, and even life and death?
The second step: Let AI directly find the pattern between faces and life and death.
The team released their second big move: FAHR-FaceSurvival. They directly used 34363 photos of cancer patients, combined with the real survival outcomes of the patients for training.
In fact, the AI here does not have X-ray vision, it cannot see the cancer cells inside the body. It purely performs statistics on massive amounts of data: what kind of facial features correspond to higher death risk?
So where on the face are these patterns hidden?
The team called up the AI's "heat map" (which is the picture below), wanting to see where exactly it is focusing on when making judgments (the bright colored areas are the key focus points of AI).
The results showed that the two models focus on completely different areas:
FAHR-FaceAge (age calculation): It focuses on the forehead, temples and nasolabial folds. This is very consistent with human intuition, as these areas are exactly the feature zones where wrinkles grow and make people look older.
FAHR-FaceSurvival (life and death judgment): Its line of sight surprisingly moves down, and locks tightly on the under-eye area and the mid-face of the nasal bridge.
This is very interesting, why do we need to focus on the under-eye area and mid-face to judge life and death?
Although the heat map cannot directly tell us "why", this makes perfect sense when combined with common medical knowledge.
Medically, malignant tumors consume the body extremely, and patients are very prone to extreme weakness that leads to facial fat loss, sunken eye sockets and muscle atrophy.
This is actually a manifestation of decreased body reserve function and overall frailty.
However, in daily clinical consultations, it is difficult for doctors to quantify this kind of "slight facial structural collapse" at a glance with the naked eye.
With the support of life and death data of tens of thousands of people, AI keenly captures these "haggard feelings" that are easily ignored by the naked eye.
So is AI's judgment reliable? Is it just a lucky coincidence?
The research team also conducted independent verification. After excluding all confounding factors that doctors have already mastered, such as age, gender, and cancer stage, the results are still astonishing.
For the quarter of patients with the highest risk predicted by AI, their death risk is still 3.23 times that of the quarter of patients with the lowest risk!
It can be seen that an ordinary facial photo contains far more information than what we can see with the naked eye.
03
How far is it from being actually applied in hospitals?
At this point, Fox Girl is also very excited, wondering when hospitals will really be able to use this technology.
Obviously, the research team does not want this technology to only stay in academic papers.
At present, they have launched the public project FaceAge Research Study (the website shown below), where ordinary people can also upload photos to get their own research results, and help the team collect more diverse data at the same time.
Why invite ordinary people to participate? Because AI is most afraid of "partial knowledge".
The existing model may be very accurate in some specific groups of people, but the effect may not be the same when applied to people of different skin colors or different age groups.
Only by feeding it sufficiently diverse and extensive data can it become a truly universal tool.
From a clinical perspective, if this technology is successfully implemented, it will have extremely huge potential for application.
For example, whether elderly cancer patients can withstand high-intensity treatment has always been a difficult evaluation problem that plagues doctors.
If a facial photo can provide additional risk clues, it can to a certain extent prevent elderly patients from suffering unnecessary pain, which is also the original intention of this research.
Professor Hugo Aerts, a medical AI expert, has a more far-sighted vision: in the future, this set of technology may not only be used for cancer.
After all, taking a photo does not require expensive equipment, and you can take it with a mobile phone.
In the future, in primary clinics, home-based elderly care and even telemedicine, taking a regular facial photo may help medical staff spot abnormal signals that deserve vigilance.
For chemotherapy patients, continuous facial records can also allow doctors to dynamically observe the changes in their physical state.
However, there are still several huge gaps between "being able to predict" and "being truly useful".
Although AI has found the correlation between facial features and survival outcomes, it still cannot explain the underlying physiological mechanism.
In addition, apart from giving a score, can it provide additional value beyond the existing diagnosis made by doctors? All these require larger-scale, cross-hospital real-world tests.
Furthermore, if AI labels a patient as "high risk", how should doctors use this result?
If it only reminds doctors to pay more attention and intervene earlier, it is definitely a good tool.
But if an immature risk score becomes a reason to "sentence the patient to death" and dissuade them from receiving treatment, it will completely deviate from the original intention of technical assistance.
Looking back at this research, Fox Girl thinks the most fascinating part is not that AI can "tell fortunes", but that it tries to turn the previously mysterious "degree of physical frailty" into quantifiable and comparable data.
How many health secrets are hidden in a single face? AI may have only just lifted the tip of the iceberg.
This is undoubtedly a very imaginative starting point. But the more powerful the technology is, the clearer its boundaries must be.
It can become a powerful tool to assist doctors in seeing more information, but it can never replace doctors and become the final arbiter of fate.
After all, the essence of medicine is ultimately the warmth of humanity.
References: The Lancet Digital Health and other data from online platforms