Forget touchscreens –mind-reading machines are on the way. Researchers at Columbia University have built a system that can turn your thoughts into speech.
The machine monitors your brain activity, then takes what you’re thinking and with the help of speech synthesisers and artificial intelligence, turns it into clear, understandable dialogue.
Besides its potential as means of interfacing with and controlling technology, the system could also be used to help people who can’t speak to communicate.
Previous research has shown that when we speak, listen, or even just imagine speaking or listening, distinct patterns of activity are produced by our brains. In order to translate these patterns into speech the researchers, from Columbia University’s Mortimer B Zuckerman Mind Brain Behavior Institute, used a vocoder – the same technology that’s used by the Amazon Echo and Apple’s Siri to synthesize speech and respond to your voice commands.
To teach the vocoder to interpret brain activity, the researchers worked with a group of epilepsy patients who were already undergoing brain surgery. The patients’ brain activity was recorded as they listened to someone recite the numbers zero through to nine. The signals their brain activity generated were then run through the vocoder, which turned the signals into speech.
The researchers then used neural networks, a type of artificial intelligence that mimics the workings of the human brain, to analyse and clean up the sound produced by the vocoder. What they were left with was a robotic-sounding voice that recited the numbers the patients were hearing. About three-quarters of the time the numbers were correct and understandable, which lead author Dr Nima Mesgarani described as being “well above and beyond any previous attempts”.
The research team’s ultimate goal is to train the vocoder to produce speech based on the brain signals emitted when a person imagines speaking. That way, in the future they might be able to produce an implant that translates the wearer’s thoughts into words. Such technology could be life-changing for people living with conditions like amyotrophic lateral sclerosis (ALS) or recovering from a stroke, who have lost their ability to speak.
“In this scenario, if the wearer thinks ‘I need a glass of water,’ our system could take the brain signals generated by that thought and turn them into synthesised, verbal speech,” said Dr Mesgarani. “This would be a game changer. It would give anyone who has lost their ability to speak, whether through injury or disease, the renewed chance to connect to the world around them.”
Follow Science Focus onTwitter,Facebook, Instagramand Flipboard