Developing Hearing Aids That ‘See’

A team of Scottish researchers is developing a hearing aid system that can read lips as well as amplify sound. Their goal is to develop a system that can work effectively in busy or noisy environments by creating a 360-degree approach that is more consistent with normal hearing.

The device, developed by a multidisciplinary team at the University of Stirling, incorporates a tiny camera that can lip read and process information in real time. According to a Stirling press release, the device can fuse the auditory and visual information, or it can switch back and forth.

“Listeners naturally combine information from both their eyes and ears: We use our eyes to help us hear. Our hearing aid will act in much the same way,” said the researchers in an abstract.

Amir Hussain, a computer scientist from Stirling who is leading the research team, said the first-of-its-kind technology could represent the next generation of hearing aids.

“Existing commercial hearing aids are capable of working on an audio-only basis, but the next-generation audio-visual model we want to develop will intelligently track the target speaker’s face for visual cues, like lip reading. These will further enhance the audio sounds that are picked up and amplified by conventional hearing aids,” Hussain said.

According to the National Institute on Deafness and Other Communication Disorders, only 30 percent of the 38 million Americans who suffer from significant hearing loss use a hearing aid. A study conducted in Finland in 2013 reported that the most-cited reason for noncompliance with hearing aids was “disturbing background noise.”

Hussain believes that an audio-visual combination hearing aid could greatly reduce the effort required for wearers to process what they hear in a busy environment.

“The 360-degree approach of our software design is expected to open up more everyday environments to device users, enabling them to confidently communicate in noisier settings,” said Hussain. “This will be important to help ensure uptake of this transformational technology.

A recent $652,000 grant from the Engineering and Physical Sciences Research Council (EPSRC) will allow Hussain’s team to continue its research, and he said the next step will be to refine the device’s time delays to be less than 10 ms. Hussain also is working with John Barker from the University of Sheffield to develop biologically-inspired ways to separate speech sources.

The researchers claim their technology has many potential applications beyond hearing aids. For instance, the technology could advance cochlear implant signaling processing, or it could be used to develop systems that allow workers using noise reduction headgear—such as those in factories or in the military—to continue communicating.

William Whitmer, of the Medical Research Council’s Institute of Hearing Research, said, “We are excited about the potential ability for this new technology—that takes advantage of the similar information presented to the eyes and ears in noisy conversation—to aid listening in those difficult situations, a consistent issue for those affected by hearing loss.”