Birds' brain activity translated into song

The findings could allow scientists to design high-quality vocal prosthetics for people who can no longer speak.

Published: June 22, 2021 at 8:29 am

Here’s something worth tweeting about: researchers have managed to synthesise birds' brain activity into song.

The scientists, which come from the University of California, San Diego, say that this research could help create a means of communication for people who are no longer able to speak.

Current state-of-the-art communication prosthetics are implantable devices that allow the user to generate text at a speed of about 20 words per minute. “Now imagine a vocal prosthesis that enables you to communicate naturally with speech, saying out loud what you're thinking nearly as you're thinking it,” said senior author Timothy Gentner, a professor of psychology and neurobiology at UC San Diego. “That is our ultimate goal, and it is the next frontier in functional recovery.”

While human speech and birdsong might not seem immediately comparable to each other, the researchers say that there are a lot of similarities, as both types of vocalisation are learned and complex behaviours.

zebrafinch
The zebra finches' brain activity was recorded as they sang, in order to create computer generated versions of their songs © Ondina Arneodo

The team implanted electrodes into the brains of four male zebra finches and read their brain activity while they sang. Specifically, they monitored the electrical activity in regions of the brain that control the muscles responsible for singing. They then fed this information into a machine-learning algorithm, with the idea that they’d be able to create computer-generated versions of the zebra finches’ songs using just their neural activity. This was not easy.

“There are just too many neural patterns and too many sound patterns to ever find a single solution for how to directly map one signal onto the other,” said Gentner.

To overcome this hurdle, the team designed some mathematical equations that modelled physical changes that happen in the birds’ vocal organ (the syrinx) when they sing. They then trained the algorithms to map the neural activity onto these equations.

Read more about birds:

"If you need to model every little nuance, every little detail of the underlying sound, then the mapping problem becomes a lot more challenging," said co-author Vikash Gilja, a professor of electrical and computer engineering at UC San Diego. "By having this simple representation of the songbirds’ complex vocal behaviour, our system can learn mappings that are more robust and more generalisable to a wider range of conditions and behaviours.”

Next, the researchers want to create a way of reconstructing neural activity to birdsong in real time. According to Gentner, this would help them to develop a successful vocal prosthesis that could operate on a rapid timescale, and also make adjustments for any errors.

“We are leveraging 40 years of research in birds to build a speech prosthesis for humans – a device that would not simply convert a person's brain signals into a rudimentary set of whole words but give them the ability to make any sound, and so any word, they can imagine, freeing them to communicate whatever they wish,” he said.

The research is published in the journal Current Biology.

Reader Q&A: How do baby birds breathe inside their eggs?

Asked by: Eleanor Tew, York

It’s all down to some nifty engineering inside the eggshell. Early on in a chick’s development, it grows a hollow, sac-like structure from its gut, known as an ‘allantois’. This pouch fuses with a second membrane (‘chorion’) surrounding the chick and its yolk, which together form the ‘chorioallantoicmembrane’.

With one end attached to the chick, and one end close to the eggshell’s inner surface, this membrane effectively acts like lung tissue, connecting the chick’s circulatory system to the outside world. Oxygen diffuses through microscopic pores in the shell to the blood vessels in thechorioallantoic membrane, and then on to the chick’s bloodstream. Carbon dioxide, the gaseous waste product of respiration, passes in the opposite direction.

Read more: