Wednesday 11 December 2024
Select a region
Scientists have created a computer program that can lip-read better than humans

Scientists have created a computer program that can lip-read better than humans

1 months ago

Scientists have created a computer program that can lip-read better than humans

1 months ago


Scientists have managed to build a computer program that can read people’s lips with more accuracy than trained experts.

Researchers at the University of Oxford built the software, called LipNet, which can read lips correctly 93.4% of the time.

In the research paper, the team claim their program is the first to perform sentence-level sequence prediction. All previous work done on lip-reading has been focused on performing just single word classification instead.

The goal is to automate lip-reading and the team’s list of potential uses includes “improved hearing aids, silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification, and silent-movie processing.”

a woman's mouth is lip read (Screengrab/YouTube)
(Screengrab/YouTube)

The computer was trained by playing around 30,000 three-second videos of men and women speaking short sentences as the machine extracted a mouth-centred crop and learned to match the different movements of the lips to the words being spoken.

The program was then tested against three hearing-impaired people who can lip-read and they scored an average of 47.7% accuracy – far less than the ability of the machine.

The software learned to lip-read using a set format of grammar known as GRID – which goes “command + colour + preposition + letter + digit + adverb”. Therefore more research needs to be done into everyday speech that doesn’t naturally follow this format before it can be released to the world.


« Return to Tech

You have landed on the Bailiwick Express website, however it appears you are based in . Would you like to stay on the site, or visit the site?