US 11,082,460 B2Granted

Audio source enhancement facilitated using video data

AssigneeSynaptics Incorporated

InventorsFrancesco Nesta, Boyan Bonev, Utkarsh Gaur

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for audio signal enhancement facilitated using video data are provided. In one example, a method includes receiving a multi-channel audio signal including audio inputs detected by a plurality of audio input devices. The method further includes receiving an image captured by a video input device. The method further includes determining a first signal based at least in part on the image. The first signal is indicative of a likelihood associated with a target audio source. The method further includes determining a second signal based at least in part on the multi-channel audio signal and the first signal. The second signal is indicative of a likelihood associated with an audio component attributed to the target audio source. The method further includes processing the multi-channel audio signal based at least in part on the second signal to generate an output audio signal.

Claims20

Dependent claims

A method comprising:

authenticating a user for a voice application;
receiving a multi-channel audio signal comprising audio inputs detected by a plurality of audio input devices;
receiving an image captured by a video input device;
determining, based at least in part on the image and a predefined facial identifier associated with the authenticated user, a face detection signal indicative of a location and/or a size in the image associated with the authenticated user;
determining, based at least in part on the face detection signal, a lip motion detection signal indicative of lip motion associated with the authenticated user;
determining an audio-video supervision signal based at least in part on the multi-channel audio signal and the lip motion detection signal, wherein the audio-video supervision signal is indicative of a likelihood associated with an audio component attributed to the authenticated user;
and
processing the multi-channel audio signal based at least in part on the audio-video supervision signal to generate an output audio signal.

5 more elements

A system comprising:

a video subsystem configured to receive an image captured by a video input device, the video subsystem comprising:
an identification component configured to determine a face detection signal based at least in part on the image and a predefined facial identifier associated with a user authenticated for an audio application, wherein the face detection signal is indicative of a location and/or a size in the image associated with the authenticated user;
and
a lip-motion detection component configured to determine a lip motion detection signal based at least in part on the image, wherein the lip motion detection signal is indicative of lip motion associated with the authenticated user;
and
an audio subsystem configured to receive a multi-channel audio signal comprising audio inputs detected by a plurality of audio input devices, the audio subsystem comprising:
a logic component configured to determine an audio-video supervision signal based at least in part on the multi-channel audio signal and the lip motion detection signal, wherein the audio-video supervision signal is indicative of a likelihood associated with an audio component attributed to the authenticated user;
and
an audio processing component configured to process the multi-channel audio signal based at least in part on the audio-video supervision signal to generate an output audio signal.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/455,668Filed 27 Jun 2019
Published: US 2020/0412772 A131 Dec 2020
Granted: 3 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification34 codes

H04N 7/147
Between two video terminals, e.g. videophone: communication arrangements, e.g. identifying the communication as a video-communication,…
G06F 3/012
Arrangements for interaction with the human body, e.g. for user immersion in virtual reality: head tracking input arrangements
G06F 3/013
Arrangements for interaction with the human body, e.g. for user immersion in virtual reality: eye tracking input arrangements
G06F 3/165
Sound input; Sound output: management of the audio stream, e.g. setting of volume, audio stream path