Tech giants want to harvest your brain

Rafael Yuste is in his early sixties, and at first glance, he bears a striking resemblance to Picasso — if Picasso wore glasses and sported a neatly trimmed white goatee. He speaks concisely and methodically, with a thick Spanish accent. He told me about a mouse experiment he was conducting in his lab at Columbia University, focusing on the region of the cerebral cortex that responds to visual input. His mentor was the Swedish neuroscientist Torsten Wiesel, who won the Nobel Prize for his research on how the visual system processes information.

"He stumbled upon the fact that the most intense stimulus was a set of striped patterns with sharp contrasts between light and dark," Yuste raised one hand, wiggling his fingers back and forth, "Suppose my fingers are bright stripes against a pitch-black background — if I move my fingers right in front of your eyes, the entire visual cortex will be activated."

At the start of the experiment, researchers trained the mice using these moving images. They projected stripes onto a computer screen in front of the mice: when the stripes moved up and down, the mice would drink water from a water spout; when the stripes moved left and right, they would stop drinking. Through a sophisticated laser system, the researchers monitored brain activity through the mice's skulls, accurately identifying exactly which neurons fired when the mice viewed the projected images. Yuste explained: "We were able to see which neurons were encoding the visual stimulus."

After cracking this neural code, Yuste's team used a second holographic laser system to project a series of light spots into the mice's brains, each spot activating the exact same batch of neurons that represented the up-down or left-right moving stripes.

"The defining experiment was when we turned the screen off," Yuste said. "It was like playing the piano, where you press specific keys with different fingers. We were 'playing' images on the cortex; once we played them out, we could make the mice act exactly as we wished." When the team implanted the image of up-down moving stripes into the mice's brains, the mice would lick for water; when they implanted the image of left-right moving stripes, the mice stopped licking.

In effect, they first "read out" the mice's thoughts, accurately identifying what was happening in their brains when they viewed images, and then used that data to make them see things that did not exist at all.

"When the mice saw the images we implanted, they licked the water spout in exactly the same way as when they saw the images with their own eyes. I mean, the number of licks was the same, the duration of each lick was the same, and the delay before they started licking was also the same. So as far as we can tell, they couldn't tell the difference. They thought those things were actually right in front of them."

Yuste said this clearly demonstrates the power of this new technology: Researchers can "manipulate mice like puppets," making them behave differently based on the images implanted in their brains.

"What we can do in mice today, we will be able to do in humans tomorrow."

Over the past 20 years, researchers have been using functional magnetic resonance imaging (fMRI) to draw increasingly detailed maps of the mammalian cerebral cortex and build a functional inventory. fMRI typically uses the blood-oxygen-level-dependent (BOLD) signal to indirectly reflect brain activity by detecting local blood flow and blood oxygen changes caused by neural activity. At the same time, tremendous progress has been made in machine learning artificial intelligence — these computer algorithms can process massive amounts of information and use statistical methods to classify and make predictions. Today, fMRI scans can already identify a wide range of mental states, from depressive thoughts to subtle emotions like jealousy and schadenfreude. Other algorithms can reconstruct movie clips that subjects have watched with considerable accuracy just by analyzing brain scans; other researchers recorded the brain activity of swing voters in the US presidential election as they viewed photos and videos of candidates, to determine which candidates triggered anxiety or even disgust, and which ones evoked positive reactions or empathy.

In just the past few years, neuroscientists have advanced their decoding targets from images and emotions emerging in the cerebral cortex to sounds, words, phrases, and even language. In 2023, this emerging technology received a high-profile demonstration. A woman named Ann Johnson, who had been paralyzed for 18 years due to a brainstem stroke, "spoke" again through a 253-electrode array implanted on the surface of her brain. The system was able to convert her neural signals into sentences in real time, at a speed of 78 words per minute, roughly half the pace of normal conversation.

The research team at the University of California, led by neurosurgeon Edward Chang, also connected this brain-computer interface to an animated avatar on Johnson's head. The researchers reconstructed her own voice based on a 15-minute recording of a speech she gave at her wedding. When Johnson organized language in her mind, the avatar's mouth would say the words she wanted to express; its facial expressions were also affected by subtle changes in her brain activity, translating her thoughts about facial movements into emotional manifestations such as smiles, lip presses, and frowns.

"They liberated her," Yuste said. "They cloned her mind in a computer. Well, not the entire mind, just the part responsible for language. After finishing this work, Eddie" — the lead neuroscientist on the study, Chang — "called me and said, 'I can't sleep.' Because he realized the full power and all the dangers inherent in this technology. For paralyzed patients, this is incredible. But just imagine what would happen if it was used on a person for other purposes? We bear a huge responsibility. Look at what we have in our hands: we have just built a machine that can decode your language; in another 10 years, we will give you a machine that can intervene in thoughts, just like we intervene in mice today."

All "brain-reading" technologies are built on the same basic principles: first record the activity of a person's neurons when they perform specific functions such as speech, language, vision, or concentration, isolate and interpret where this function occurs — which manifests mainly in the form of electric fields, radio waves, or electrical pulses — and then determine what these activities actually mean.

The more invasive the recording device is, the richer and more detailed the data obtained usually is. Surgical implantation technology is at the cutting edge of neuroscience and remains extremely rare today: fewer than 100 people worldwide have implanted brain-computer interfaces of the kind Johnson used under their skulls. However, these capabilities are almost inevitably beginning to trickle down step by step. In the summer of 2023, a team at the University of Texas demonstrated how to convert fMRI brain scan results into words and sentences. Subjects first listened to 16 hours of narrative podcasts, including *The Moth Radio Hour* and *The New York Times' Modern Love*, to train an AI model. Later, when subjects listened to new podcasts, the algorithm could convert the activity patterns left by the story content in their brains into words, phrases, and sentences, roughly restoring the outline of the story they heard. The team leader, computational neuroscientist Alexander Huth, said in an interview with *Science*: "When it actually started working, our first thought was, 'My God, this is a little bit scary.'"

Today, non-invasive wearable brain scanning devices are moving out of laboratories, entering workplaces, and entering ordinary people's homes through the huge global consumer market.

In 2021, Jack Gallant, a professor of cognitive neuroscience at UC Berkeley, mentioned a possible future technology in an interview with *The New Yorker*, calling it a "thinking cap." Gallant is committed to building a "complete functional map of the human brain." He envisions companies paying people $30,000 a year to wear such a cap; the cap can integrate glasses with video recording functions and various sensors to collect brain data generated by the wearer when they see, feel, hear, and experience everything in daily life. From a scientific point of view, this logic is obvious: imagine what amazing new data this kind of device can generate, especially when your goal is to draw a truly comprehensive map that covers all functions of the brain. Nevertheless, the first time I read about this "thinking cap," I instinctively felt a wave of dystopian fear.

It is not hard to imagine how things will develop: first, graduate students, many of whom are eager to pay off their high student loans, will voluntarily become the first batch of subjects; then, it will gradually spread from campuses to the gig economy. For people who are already juggling multiple jobs, putting on a "thinking cap" seems like a no-brainer — or more accurately, a choice that requires handing over their entire brain. Earn an extra $30,000 a year just by using your brain? Soon, an immeasurably massive amount of brain activity data will flow into the digital public space. People move around with wearable scanners and live their normal lives, while the devices are constantly "mining" the information in their brains. At first, people may do it voluntarily; later, they may be increasingly forced by reality, or out of desperation, or for even worse reasons.

In fact, we will begin to hand over our neural activities to enterprises and data brokers in exchange for money, or even just digital assets or website access permissions, just like we trade our personal information and search preferences today. This is a dangerous transaction. As Yuste said, the mind should be the sanctuary of our identity. "You must protect it, and not let others break in, hoard and sell brain data."

This may sound alarmist, like something out of science fiction, but it is built on technologies that already exist and are actually operational. There is no universal wearable "thinking cap" today, and it may still take a long time to truly realize it, but early versions of this concept have already emerged. Most current devices are worn on the scalp, using electroencephalography (EEG) technology to detect the weak electric fields generated by the discharge of millions of neurons in the cerebral cortex that pass through the skull. In recent years, as part of brain-computer interfaces, EEG has become a tool for assessing and even changing mental states, covering multiple states such as concentration, calmness, stress, and drowsiness.

For example, neurotechnology company Emotiv has been testing a system consisting of an EEG head-mounted device and headphones. After office workers put it on, the system will monitor their attention levels throughout the day when they process tasks based on brain activity, including when their attention drops, when they get distracted, and their corresponding cognitive stress. Its goal is to improve efficiency and productivity: if stress stays high for a long time and attention decreases, the system will suggest that employees take a break. The original idea was to anonymize the data, or only allow employees themselves rather than employers to access it, but this situation is unlikely to last for too long, especially since Emotiv CEO Tan Le predicted that brain activity tracking in the workplace will be "quite common" within the next five years.

Emotiv

EEG headbands have already been used for fatigue warning for truck drivers and miners, and they obviously have clear value in preventing work-related injuries. A more worrying case comes from BrainCo: it once provided Chinese primary school students with an EEG head-mounted device called "Focus" to monitor students' attention levels. The data was then uploaded to the company's servers, where teachers could view it, but students and their parents had no right to access it. (After *The Wall Street Journal* released a related video report, the program was quickly shut down.)

Dubai Police have been using a technology called iCognative for several years, which scans suspects' brains via EEG to capture fleeting weak signals when they recognize an item or a piece of information. Allegedly, in one case, the police showed a murder suspect an image of the murder weapon, and his brainwaves immediately showed uncontrolled spikes. Faced with this data, he pleaded guilty immediately. His mental activities alone exposed him. In other words, he was caught by the "thought police."

In the consumer market, EEG devices are mainly targeted at the fields of health and physical and mental management. For example, the producer of the iBand head-mounted device claims that it can induce lucid dreams; Flow provides home-based anxiety or depression treatment, sending electrical pulses to the brain regions responsible for regulating mood, sleep, and motivation. Another device called BrainBit EEG promises to help users "use their brains in ways never before possible," its headband can "monitor brain activity, collect data, and convert and use this data without the need for professional personnel to interpret it".

Its advertised application scenarios extend from meditation and sleep to education — "extend school children's attention span", business — "analyze and understand employees' brain state signals", and even online dating — "listen to your brain and swipe left or right based on instinctive reactions". It even suggests that users can "automatically post their emotions to social media homepages, or add emotion indicators to posts". (I wonder if it can capture an emotion called "existential despair"?) Another proposed use for BrainBit is "neuromarketing": revealing "consumers' subconscious responses by 'collecting neural insights' to improve the effectiveness of market research".

BrainBit

It is not hard to see where things are heading: advertisers and enterprises will sneak into our subconscious to find more accurate ways to sell products. In 2022, Emotiv partnered with L'Oréal, the world's largest cosmetics company, to develop in-store EEG technology, integrating it into personalized fragrance consultation services, using neural activity to identify consumers' perfume preferences. In Emotiv's words: "We connect emotions and scents with technology, thus simplifying the decision-making process."

Tech giants such as Apple, Meta, and Snap are all developing their own neurotechnology products. Meta and Snap's goal is to use neural activity to turn thoughts into operations on a computer, "allowing you to press a virtual button just by concentrating your attention". Apple has patented a device that is speculated to integrate EEG sensors into future versions of AirPods. This already ubiquitous device will effectively allow the world's most valuable company to instantly access our brain activity.

Currently, these wearable EEG devices cannot yet "decode" thoughts — at least not to the extent demonstrated by the neuroscience research teams in California and Texas — but technological progress is accelerating, and this only seems to be a matter of time. A company called Kernel has already built a device also called Flow, which combines EEG sensors with infrared light — essentially shining lasers into the brain — and can collect neural activity data of quality close to fMRI. The entire system is housed in a sturdy, compact head-mounted device that looks like a BMX helmet, which is perhaps the closest product to Gallant's universal "thinking cap" to date. It is expected that within the next 10 years, a consumer version of this technology may be launched.

Yuste told me that his goal in devoting himself to neuroscience has always been to find the brain's own natural "malware" and eventually find a cure. Early in his career, he worked in a psychiatric hospital in Madrid, treating patients with brain diseases, many of whom suffered from paranoid schizophrenia, and some were so dangerous that he had to conduct interviews accompanied by bodyguards. One of the highly intelligent patients deduced where Yuste lived based on details such as his accent and threatened to come to his house and kill his father. This experience left a deep impression on Yuste. Something in the patient's mind was driving them to be hostile to themselves and society, but doctors did not know what that was, because ultimately, they did not understand the brain.

When Yuste successfully implanted false images directly into the brains of mice, it was both a breakthrough and a wake-up call. It provided a possible path to understand where and how hallucinations occur in the brain, and even how to eliminate them.

"I thought at the time, my God, now we can really help schizophrenia patients. We can enter the brain, reprogram their cerebral cortex, and