Never in my long life did I imagine that the brain of a deceased person can control a robotic manipulator to play the piano.

Brain slices from deceased people can now play the piano!

Recently, a French scientific research team directly connected post-mortem human brain tissue to a robotic hand and trained it to imitate humans playing the piano.

They extracted still-viable brain tissue from post-mortem human brains, placed it in a petri dish, and connected it to electrodes, a microphone, and a bionic robotic hand.

After training, when a human presses a piano key, the brain tissue can "hear" the note and then control the robotic hand to play the same sound.

There is even video evidence to prove this.

To be fair, the original topic of this research is actually a very serious AI problem:

Can we use the natural learning ability and ultra-high energy efficiency of biological neurons to create a new hybrid AI system?

But when the keywords "post-mortem human brain", "brain slice", "prosthetic limb" and "hybrid AI" are put together, it is hard not to associate them with cyberpunk.

After the research was released, it quickly sparked widespread discussion.

Some content creators have specifically analyzed the experimental video, arguing that while this type of work is inherently accompanied by ethical controversies and a certain "eerie feeling", the intersection of neuroscience and AI may also promote the development of cognitive science, neurological disease research, and even medical research.

Many netizens have even followed this experiment to imagine future super individuals, mechanical prosthetic bodies, and even philosophical problems such as what consciousness ultimately depends on.

However, let's not rush to discuss the "brain in a vat" thought experiment for now.

Because the really interesting part of this research may not be that a dead brain suddenly starts playing the piano.

It is that for the first time, researchers placed a real adult human brain tissue into a closed-loop system that can perceive, act, and re-learn.

So how on earth did they achieve this?

The human brain is inherently a more efficient computing system

To better understand what this research is specifically about, let's first go through the necessary background context.

As mentioned above, compared with the human brain, artificial neural networks running on AI chips are still, in a sense, a very "inefficient" learning system.

The most intuitive indicator is energy consumption. The power consumption of an adult human brain is only about 20 watts, while the world's most powerful supercomputers have a power consumption of tens of megawatts.

It can be said that the human brain is far more cost-effective than AI when producing the same number of tokens.

Secondly, in terms of learning efficiency, the human brain has the ability to learn from small samples and generalize quickly, which AI does not possess.

For example, in order to beat Lee Sedol, AlphaGo needed to be trained on more than 150,000 Go games.

Calculated by practicing 8 hours a day, this is equivalent to a person practicing continuously for more than 100 years, which far exceeds the actual training volume of any human Go player.

A language model like GPT-3 requires an amount of training data equivalent to a human reading 5 e-books a day for 250 consecutive years.

Therefore, although today's AI is very powerful, many of its capabilities essentially still rely on massive amounts of data and massive amounts of computing power to "brute force" results.

In contrast, humans often only need very limited experience to learn new movements, concepts and associations.

So over the past few years, an increasingly popular research direction has emerged:

Since artificial neural networks are designed to imitate the human brain, why not directly connect real neurons into the computing system?

This is the so-called hybrid AI system.

Previously, researchers mainly tried two-dimensional cultured neurons and brain organoids, but the former is difficult to form stable neural plasticity, while the latter has relatively immature neural circuits and large individual differences.

So this French team chose a different material: post-mortem adult brain explants, also known as OPAB.

The biggest difference of this material is that it is not a brain model "grown" from stem cells, but real tissue from an adult human brain, so it retains mature cell types, tissue structures and partial neural connections.

The team has previously proved that these brain slices can maintain neural activity in vitro after being supplied with oxygen and glucose.

The only real remaining question is: Can it continue to learn after leaving the human body?

Therefore, the researchers decided to directly equip it with a "hand" and an "ear".

Equip the brain slice with a "hand" and an "ear"

Overall, this "brain slice playing piano" system is not complicated.

In the experimental setup, the researchers first placed the OPAB on a microelectrode array, then randomly selected three electrodes as "motor electrodes", which were respectively connected to the three fingers of the robotic hand.

The three fingers correspond to three notes on the keyboard respectively: LA, TI and DO.

Once a certain motor electrode is activated, the corresponding finger bends and presses the key. At this point, the brain slice is connected to a "hand".

The next step is to equip it with an "ear". After the robotic hand presses the key, a nearby microphone collects the sound.

The sound decoder identifies whether the note is LA, TI or DO, and then stimulates the three corresponding "sensory electrodes" on the brain slice.

Thus, a complete closed loop is formed: the brain slice generates motion signals → the robotic hand presses the key → the microphone captures the sound → the sound is converted back into electrical stimulation → transmitted back to the brain slice.

From the perspective of the brain slice, this is very similar to a baby getting to know its own body for the first time.

When a baby is just born, it does not know what will happen after a certain muscle contracts. It will constantly wave its hands and kick its legs randomly, and then observe the results of these actions through vision, touch and hearing.

This process has a vivid name in robotics: Motor Babbling.

What the researchers need to do is to replicate this process in a petri dish.

Bidirectional Neural Plasticity

Specifically, during the training phase, the system randomly stimulates one of the motor electrodes.

After the stimulation, one finger of the robotic hand bends immediately and presses a key. Then the microphone captures this note and converts it into stimulation, which is sent back to the corresponding sensory electrode.

For example, when electrode A is stimulated, the index finger bends and presses the LA key. The microphone picks up the LA note, and the sensory electrode corresponding to LA is stimulated again.

One action is followed by one sensory result, repeated 300 times a day for three consecutive days, so that the brain slice forms muscle memory.

It is worth mentioning that throughout the whole process, there is no feedback of "you are right" or "you are wrong", nor is there any reward function.

The researchers just let the two events happen consecutively in time repeatedly, and leave the rest to the neurons themselves.

The reason why this is possible relies on the most basic learning mechanism of the brain —

Neural Plasticity.

Simply put, the connections between neurons that are often activated together will gradually become stronger. So after repeated training, the brain slice gradually associates:

"This finger" with "this sound".

But more critically, this connection is not unidirectional, which means that the brain slice can not only learn: "When I move this finger, I will hear this sound."

Conversely, it can also learn that "when I hear this sound, I will activate this finger."

Thus, the whole process changes from a simple association between action and sensation to active imitation by the learning system.

To verify this hypothesis, in the formal test, human experimenters began to play LA, TI and DO randomly.

This time, the motor electrodes are not stimulated first. Instead, the microphone directly transmits the notes played by humans to the brain slice.

If the association mentioned above has indeed been learned, after the brain slice "hears" LA, it should reversely activate the motor region that has previously established an association with LA, and the robotic hand will press LA again, completing an imitation.

As it turns out... this really happened.

The study used a total of 30 OPAB from 3 donors, 16 of which entered the main test phase.

On the first day of training, the average imitation accuracy of this system was only 31.22%. Since there are only three notes in total, the accuracy rate of completely random guessing is 33.3%.

In other words, the brain slice learned nothing at the beginning.

But after three days of training, the average accuracy rate on the fourth day has risen to 58.5%. Among the 16 samples, 3 OPAB could correctly imitate all three notes with 100% accuracy.

Another 7 could stably learn two of the notes. In the end, 62.5% of the OPAB successfully learned at least two notes.

It is also necessary to emphasize one detail here. This 62.5% is not the best result obtained by researchers after carefully selecting electrodes.

On the contrary, in order to prove that this association is not formed by taking advantage of the pre-existing connections inside the brain slice, but generated through acquired training, the researchers deliberately selected motor electrodes and sensory electrodes randomly.

In addition, in order to rule out the influence of software and hardware modules, the research team also conducted several groups of controlled experiments.

The results show that after switching to untrained electrodes, the accuracy rate drops back to the random level; after blocking synaptic transmission with drugs, or infecting the brain tissue with Tahyna virus which disrupts neural activity, the formed imitation ability will also decrease significantly.

That is to say, what carries this set of sensory-motor associations is the altered neural connections in the brain tissue itself, not external programs.

Moreover, this "memory" can last for a fairly long time.

Among the 5 OPAB tracked for a long time, 4 still retained the learning results 17 days after the end of the training, and 2 of them could still correctly imitate all three notes with 100% accuracy.

Of course, this is still far from the "human brain computer" concept.

The whole experiment only has three sets of finger-note associations, the diameter of the brain slice is about 5mm, and sound recognition and robotic hand control still rely on traditional AI modules.

So more accurately, this does not mean that a post-mortem brain tissue suddenly "learned to play the piano", but that a still-viable adult brain tissue formed new non-pre-existing associations in an artificially built sensory-motor closed loop, and used this set of associations to complete imitation.

Truly, learning has no end.

Reference Link [1]https://www.researchsquare.com/article/rs-9638576/v1

This article is from the WeChat official account "QbitAI", written by authors focusing on cutting-edge technologies, and published with authorization from 36Kr.