System and method for disambiguating a source of sound based on detected lip movement
AssigneeDmai, Inc.
InventorsNishant Shukla, Ashwin Dharne
Life of this patent
Estimating the term…
Abstract
The present teaching relates to method, system, medium, and implementations for detecting a source of speech sound in a dialogue. A visual signal acquired from a dialogue scene is first received, where the visual signal captures a person present in the dialogue scene. A human lip associated with the person is detected from the visual signal and tracked to detect whether lip movement is observed. If lip movement is detected, a first candidate source of sound is generated corresponding to an area in the dialogue scene where the lip movement occurred.
Claims21
A method implemented on at least one machine including at least one processor, memory, and communication platform capable of connecting to a network for detecting a source of speech sound in a dialogue, the method comprising:
- receiving sensor data acquired by a plurality of sensors from a dialogue scene, wherein the sensor data capture information about the dialogue scene having one or more people present including a person currently engaged in a human machine dialogue;
- estimating at least one candidate source of sound based on the sensor data, wherein each of the at least one candidate source of sound is characterized by one or more parameters;
- obtaining user information related to the person, wherein the user information characterizes a current state of the person, including a location of the person most recently tracked;
- and
- identifying one or more candidate sources of speech sound from the at least one candidate source of sound via filtering based on the location of the person and the one or more parameters associated with each of the at least one candidate source of sound.
Machine readable medium having information recorded thereon for detecting a source of speech sound in a dialogue, wherein the information, when read by the machine, causes the machine to perform the following steps:
- receiving sensor data acquired by a plurality of sensors from a dialogue scene, wherein the sensor data capture information about the dialogue scene having one or more people present including a person currently engaged in a human machine dialogue;
- estimating at least one candidate source of sound based on the sensor data, wherein each of the at least one candidate source of sound is characterized by one or more parameters;
- obtaining user information related to the person, wherein the user information characterizes a current state of the person, including a location of the person most recently tracked;
- and
- identifying one or more candidate sources of speech sound from the at least one candidate source of sound via filtering based on the location of the person and the one or more parameters associated with each of the at least one candidate source of sound.
A system for detecting a source of speech sound in a dialogue, comprising:
- a plurality of sound source estimators implemented using a processor and configured for
- receiving sensor data acquired by a plurality of sensors from a dialogue scene, wherein the sensor data capture information about the dialogue scene having one or more people present including a person currently engaged in a human machine dialogue,
- estimating at least one candidate source of sound based on the sensor data, wherein each of the at least one candidate source of sound is characterized by one or more parameters, and
- obtaining user information related to the person, wherein the user information characterizes a current state of the person, including a location of the person most recently tracked;
- and
- a sound source disambiguation unit implemented using a processor and configured for identifying one or more candidate sources of speech sound from the at least one candidate source of sound via filtering based on the location of the person and the one or more parameters associated with each of the at least one candidate source of sound.
Dependencies are inferred from the claim text.