US 4,975,960 A · Electronic facial tracking and detection system and method and apparatus for automated speech recognition

Abstract

The apparatus includes circuitry for obtaining a video image of an individual's face, circuitry for electronically locating and tracking a first feature, such as the nostrils, of the facial image for use as reference coordinates and circuitry responsive to the reference coordinates for locating and tracking a second facial feature, such as the mouth, of the facial image with respect to the first feature. By tracking the location of the nostrils, the apparatus can follow the movement of the mouth, and thereby automatically recognize speech. In a preferred embodiment, the video image is grayscale encoded and the raster lines are smoothed to eliminate noise. The transitions between gray levels of the smoothed image are encoded and the resulting transition code is used to form a contour map of the image from which region parameters are computed which can be compared against stored speech templates to recognize speech. In the preferred embodiment, acoustic speech recognition is combined with visual speech recognition to improve accuracy.

Claims94

Dependent claims

An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising:

means for detecting sounds, converting said sounds to electrical signals, analyzing said signals to detect for words, and then producing an electrical acoustic output signal representing at least one spoken word;
means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker;
means, responsive to each said image signal, for tracking said speaker's mouth by tracking said speaker's nostrils;
means for analyzing portions of said image signals, said portions defined by said means for tracking, to produce a video output signal representing at least one visual manifestation of at least one spoken word;
means for receiving and correlating said acoustic output signal and said video output signal to produce said output.

2 more elements

An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising:

an acoustic speech recognizer in operative combination with a microphone, for detecting sounds, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech;
means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker;
means for analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils;
means, activatable in response to one of said location signals, for defining a portion of one of said image signals for mouth analysis, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils;
means for analyzing portions of said video image signals, responsive to said means for defining, to produce a video output signal, said video output signal representing a visual manifestation of speech;
means for receiving and correlating said acoustic output signal and said video output signal to produce said output.

3 more elements

A method for producing output indicating words spoken by a human speaker, said method comprising steps of:

detecting sounds by means of an acoustic speech recognizer in operative combination with a microphone, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech;
scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker;
analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils;
defining a portion of one of said image signals for mouth analysis in response to one of said location signals, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils;
analyzing portions of said video image signals, responsive to said step of defining, t produce a video output signal, said video output signal representing a visual manifestation of speech;
receiving and correlating said acoustic output signal and said video output signal to produce said output.

3 more elements

Dependencies are inferred from the claim text.

Electronic facial tracking and detection system and method and apparatus for automated speech recognition

Life of this patent

Electronic facial tracking and detection system and method and apparatus for automated speech recognition

Life of this patent

Abstract

Claims94