Beards and mustaches confuse visual speech recognition systems

Beards and mustaches can significantly confuse visual speech recognition systems. Consequently, they are more successful with female than male speakers.

A man hard to read. Photo of Alan Moore by Mirka. CC-BY-SA 3.0

Another problem is that some people are less expressive with their lips. Some even hardly move their lips at all and these so-called “visual-speechless persons” are almost impossible to interpret.

Most significant problem is however ambiguity between mouth shapes and sounds. During speech, the mouth forms between 10 and 14 different shapes, known as visemes. By contrast, speech contains around 50 individual sounds known as phonemes. So a single viseme can represent several different phonemes.

Source: The Challenges and Threats of Automated Lip Reading