US 5,473,726 AGranted

Audio and amplitude modulated photo data collection for speech recognition

AssigneeThe United States of America as represented by the Secretary of the Air Force

InventorPatrick T. Marshall

Life of this patent

Estimating the term…

Loading patent…

Abstract

A speech recognition data collection arrangement in which audio sounds spoken by the user are supplemented with lip and mouth movement information obtained from a combination of a light emitting diode and photodiode disposed immediately before the user's face is disclosed. The lip and mouth movement photo information is obtained in the form of an AC coupled amplitude modulated carrier waveform which results from energization of the light emitting diode from a pulsating or alternating current component inclusive source of electrical energy. The lip and mouth movement amplitude modulated carrier waveform is processed with filtering and envelope detection and communicated to a programmed digital computer for the purpose of dual-channel audio and photo based accomplishment of human speech recognition.

Claims12

Dependent claims

Dual-channel, optical energy and sound energy, human speech recognition data collection apparatus comprising the combination of:

a user's head-carried headset apparatus including a boom end received speech actuated microphone transducer member located close to lip and mouth cavity facial regions of said user;
light emitting diode means disposed adjacent said microphone transducer member at said boom end for illuminating said lip and mouth cavity regions of said user with visible spectrum optical energy;
pulse modulated electrical energy source means, of fixed predetermined operating frequency, connected with said light emitting diode means, for pulsed electrical energizing of said light emitting diode means and for pulsed optical energy generation therein;
electrical photodiode transducer means disposed adjacent said microphone transducer member for generating a user's lip movement responsive amplitude modulated electrical signal having a carrier component of said predetermined frequency and having signal correlation with a voice generated electrical signal from said microphone transducer member;
dual-channel electrical conduction means for communicating electrical signals from said microphone transducer member and said photodiode electrical transducer means to a spoken sound and lip movement correlation speech recognition electrical signal processor.

2 more elements

The method of collecting coordinated vocal and lip movement signals for dual-channel human speech recognition correlated processing comprising the steps of:

disposing an audio sound to electrical signal transducing microphone adjacent a lip and mouth cavity facial area of a user subject, said microphone generating electrical signals representative of user speech sounds;
illuminating a predetermined mouth and lip facial portion of said user with visible spectrum light energy originating in a light emitting diode electrical energy to optical energy transducer disposed adjacent said microphone;
energizing said light emitting diode with pulsating electrical energy of predetermined pulsation frequency, said illuminating visible spectrum light energy being also pulsated at said pulsation frequency;
collecting user lip-reflected illumination energy in a photodiode optical signal to electrical signal transducer, said collected energy generating a lip movement modulated electrical signal of said predetermined pulsation frequency, carrier frequency and lip movement responsive carrier amplitude modulation at output terminals of said photodiode transducer;
locating said photodiode optical signal to electrical signal transducer adjacent said microphone and said light emitting diode and proximate a predetermined portion of said user's lips and mouth cavity;
and
communicating said microphone electrical signals and said photodiode electrical signals via separate communication paths to a correlated dual-channel human speech recognition processor means for speech recognition processing.

4 more elements

Dependencies are inferred from the claim text.