Audiovisual speech recognition using visual speech information

A system comprising a sub-vocal input sensor comprising at least one of:

• (i) a laser microphone positioned to measure eardrum vibrations,
• (ii) an internal microphone oriented toward an ear canal of the user, or
• (iii) a bone-conducting or cartilage-conducting transducer disposed at a head or ear location;
• a viseme recognition module including a mouth-region imager and processing circuitry configured to extract lip-shape features, lip-motion features, or both, from the image data and to classify viseme states;
• an ambient noise detector comprising at least one external microphone and processing circuitry configured to estimate an ambient noise level or spectrum and to supply a noise metric to a fusion processor;
• and the fusion processor comprising processing circuitry configured to increase weighting of viseme recognition relative to acoustic input in response to the detected ambient noise level, wherein the fusion processor generates a semantic intent command for execution in an augmented-reality or virtual-reality environment.