Apple's "brain-controlled" iPad makes its first public demonstration: He, a quadriplegic, re-embraces the world with just a thought.

He lay in bed, barely able to move his body. His limbs had long lost control, and even the simplest action of tapping the screen was beyond his reach. But when his eyes locked onto the home screen of the iPad - a few seconds later, the screen lit up, an icon was selected, and he successfully "opened" the device with just a thought.

Mark Jackson is one of the world's first amyotrophic lateral sclerosis (ALS) patients who can control Apple devices with "thoughts". What makes all this possible is the Stentrode developed by the brain-computer interface company Synchron - a tiny metal stent implanted in the blood vessels of his brain to capture neural signals.

Complementary to it is a brand - new human - computer interaction protocol launched by Apple: BCI HID (Brain - Computer Interface Human Interface Device). This is the first time Apple has incorporated "brain signals" into the native input methods of its operating system, alongside touch, keyboard, and voice.

In short, the brain is becoming the next native "input method" on Apple devices.

Brainwaves + Apple System: The Strongest "Cyber Fusion"

The Stentrode used by Jackson is a brain - computer interface device as thin as a hair and shaped like a stent. It is implanted into a vein near the motor cortex of the brain through a blood vessel. The electrode array on the device captures neural signals, and then algorithms identify the user's intentions to ultimately control digital devices.

More importantly, it has achieved native integration with the Apple ecosystem for the first time. The key to this integration lies in the brand - new protocol launched by Apple in May this year - BCI HID (Brain - Computer Interface Human Interface Device), the brain - computer interface human - computer interaction standard.

It is like a "common language" between the brain and iOS, iPadOS, and visionOS, officially making brainwaves a legitimate input method on par with touch, keyboard, and voice.

By accessing the Switch Control accessibility feature of iOS, Stentrode users can now use electroencephalogram signals to replace button presses, clicks, or swipes.

Mark Jackson is one of the first patients to receive a Stentrode implant. He suffers from ALS (amyotrophic lateral sclerosis), cannot stand, and cannot leave his residence in the suburbs of Pittsburgh. However, this technology has brought him new freedom of movement.

In August 2023, he underwent surgery. According to a previous report in The Wall Street Journal, after the Stentrode was implanted, Jackson began training to control the Vision Pro with his thoughts.

He "saw" himself standing on a cliff in the Alps and "felt" his legs trembling - even though his body in reality could no longer stand.

Later, he gradually learned more complex operations: starting applications, sending messages, and opening emails through brain control. "In the remaining time of my life, I hope to promote the progress of technology and enhance people's understanding." Jackson said.

His words also express the core mission of the Synchron team - to make this technology benefit more people.

Peter, the senior director of neuroscience and algorithms at Synchron, said: "Our vision is to make brain - computer interfaces as popular as keyboards and mice." He explained that the difficulty of BCI lies not only in the technology itself but also in the lack of a standardized "interaction language".

So, they collaborated with Apple to develop the BCI HID protocol based on the HID standard. "It's like the common language between a computer and a keyboard. Now, we've also given the brain its own input protocol."

BCI HID not only transmits the user's neural intentions but also supports the device to provide visual feedback to the user. When Mark wants to select a button, a colored highlight box appears on the screen. The darker the color, the stronger the neural signal, and the more certain the system is that he wants to click that button. Mark can "fill" this color block through brain control to make an accurate selection.

"For users of implanted BCIs, this visual feedback is extremely important. They can see in real - time whether their neural signals are'strong enough' and can focus more easily." Kurt Haggstrom, the chief commercial officer of Synchron, explained. The entire system is connected via Bluetooth and does not require additional devices or the assistance of caregivers. As long as Mark "wants", the device can start.

Different from traditional assistive devices, BCI HID is a closed - loop interaction system. It not only identifies the user's intentions but also provides real - time context information to improve decoding accuracy and response speed.

It can also directly bind mental actions to system shortcut commands: imagining clicking a finger is equivalent to returning to the home screen, imagining making a fist is equivalent to opening messages, and imagining waving is equivalent to starting a video call. This not only enhances the freedom of control but also allows system interaction to truly enter a "zero - intervention" state.

In addition, BCI HID has extremely high privacy - brain signals are "exclusive" to the user, cannot be controlled by others, and cannot be "read" by other devices. In the future, Synchron will promote BCI HID to become a cross - platform and cross - manufacturer neural interaction standard, allowing all BCI devices to seamlessly access the digital world.

Apple's involvement is regarded as the "final push" in the whole thing. "Apple can recognize user needs and respond, which reflects their high - level attention to the user's barrier - free experience." Kurt Haggstrom commented.

The Non - Craniotomy Brain - Computer Interface May Defeat Musk

When it comes to brain - computer interfaces, most people first think of Musk's Neuralink. Whether it was Neuralink's previous live broadcasts or tweets on X, they have all attracted a lot of attention to brain - computer interfaces.

In contrast, Synchron is relatively unknown outside the industry.

However, there has been an intersection between the two companies. Three years ago, on a weekend, just as Synchron was implanting a brain - computer interface device in a patient in the United States for the first time, Musk called Tom Oxley, the founder and CEO of Synchron.

Oxley later recalled that during the call, Musk believed that the brain - computer interface solution should involve removing most of the skull and replacing it with an embedded titanium alloy shell. But he himself firmly believed that the goal could be achieved without touching the skull.

Tom Oxley

Moreover, Musk also offered to provide help if Oxley faced a shortage of funds in pursuing this goal, especially in the area of brain - computer interfaces. However, perhaps due to differences in concepts, this "partnership" ultimately came to nothing.

In fact, over the past two decades, researchers have been testing brain chip implants on humans. However, almost all of these devices require cutting open the skull and inserting electrodes into the brain, with wires hanging from the head.

To put it simply, it means making a hole in the top of the head and then inserting a device the size of an Apple Watch. Not to mention the risks during the surgery, even if the surgery is successful, the human brain will have a rejection reaction to the device, which is one of the technical difficulties of invasive brain - computer interfaces.

The Stentrode does not have this pain point.

Its surgical method is similar to implanting a cardiac stent. The product is implanted into the motor cortex of the brain (the area that expresses human motor intentions) through the jugular vein. The brain's rejection of the Stentrode is to push it into the brain tissue, so the Stentrode will be covered by tissue and fixed in that area within a few weeks.

Any brain signals detected by the Stentrode are sent through a wire that extends down the vein and connects to a receiver the size of an iPod Shuffle sewn to the patient's chest.

Similar to the battery in a pacemaker, the receiver's battery can last up to 10 years.

The receiver transmits instructions to the patient's computer or iPad via Bluetooth, enabling them to access text messages and control other applications. Once the Stentrode is installed, the patient will conduct calibration exercises, and Synchron's staff will guide them to think about moving different parts of their bodies.

The differences in implantation methods and concepts naturally lead to differences in technical performance.

For example, Neuralink's device, the N1, has more than 1000 electrodes and can capture more neural data; while the Stentrode has only 16 electrodes. The electrodes of the N1 are directly implanted into brain tissue, so the captured data is more abundant and can be converted into more sensitive mouse clicks and keyboard inputs.

In previous reports, Neuralink users could also move the cursor with their thoughts, and the speed was even faster than the mouse operations of some ordinary users.

Despite this, why did Apple ultimately choose to cooperate deeply with Synchron rather than Musk's Neuralink? Behind this, there is actually another answer from Apple regarding brain - computer interfaces: safety.

As mentioned above, the Neuralink N1 is a high - density, invasive implant. It's easy to understand that invasive surgeries have relatively high risks and may cause inflammation or tissue reactions. In contrast, the Synchron Stentrode surgery has low risks and a short recovery time, making it especially suitable for patients who are not suitable for craniotomy.

Of course, the price of the Stentrode is that since the electrodes do not directly contact neurons, the signal quality and resolution are lower, and the data bandwidth is lower, only suitable for basic - level neural signal decoding.

One Thought, One Tweet

Technical parameters are just part of the grand story. What really catches people's attention about Synchron is what it has already achieved.

In March 2024, a Neuralink patient posted a tweet on the X platform. However, going back three years, 62 - year - old ALS patient Phillip O'Keefe had already "typed" the first sentence on the X platform using the Synchron brain - computer interface:

Hello world!

Note that this is the first tweet "sent" by brainwaves in human history. There was no keyboard, no voice, and not even eye - tracking. It was all "thought" out. Although the tweet is not long, for him personally, it may be more valuable than a 100,000 - word novel.

Synchron's story doesn't end here.

When the whole world was occupied by ChatGPT, many people were thinking about how to use it to write papers, code, and love letters, while Synchron was thinking about how to use AI to improve brain - computer interface technology.

Mark is also one of the first users to experience the integration of AI and brain - computer interfaces. Although he has lost most of his limb and language abilities due to ALS, he can still play Apple Solitaire, watch Apple TV, and even "look up at the stars" on the Vision Pro with his brainwaves.

Specifically, Synchron allows large - language models such as ChatGPT to obtain relevant context in text, audio, and visual forms, predict what the user may want to express, and provide them with an operation menu to choose from.

Moreover, after adding GPT - 4o, the Synchron brain - computer interface has undergone four significant changes:

1. Assisted communication: GPT generates pre - set response options, so users don't need to input word by word.

2. Intelligent prediction: GPT predicts possible needs based on context, significantly reducing the number of operation steps.

3. Multimodal input: GPT - 4o receives text, audio, and video inputs and provides information in multiple ways.

4. Adaptive learning: The system gradually learns user preferences to achieve efficient personalized customization.