Method of detecting a speech impediment and providing visual feedback as an avatar to a user
AssigneeMicrosoft Technology Licensing, LLC
InventorMrinal Kumar Sharma
Life of this patent
Estimating the term…
Abstract
A system and method and for providing speech assistance during a virtual meeting includes receiving a request over a communication network to provide speech assistance during a virtual meeting between a plurality of participants and analyzing speech data of the virtual meeting, via a speech impediment detection engine, to detect a speech impediment for one of the plurality of participants. Upon detecting the speech impediment, an avatar is automatically generated for the participant experiencing speech impediment and the avatar is synchronized with the participant's speech in real-time during the communication session to provide real-time visual feedback to the participant. The avatar is then provided for display to the participant.
Claims19
A data processing system comprising:
- a processor;
- and
- a memory in communication with the processor, the memory comprising executable instructions that, when executed by the processor, cause the data processing system to perform functions of:
- accessing speech data from a communication session between a plurality of participants, the communication session being conducted via a communications application over a network;
- extracting a plurality of audio features from the speech data;
- providing the plurality of audio features as an input to a speech impediment detection machine-learning (ML) model to detect a speech impediment and a type of the speech impediment for one of the plurality of participants;
- upon detecting the speech impediment, determining from the type of the speech impediment whether an avatar is needed;
- when the avatar is determined to be needed, utilizing a lip synchronization engine to generate the avatar that provides a visual representation of the one of the plurality of participants;
- providing the avatar for display on a user interface screen of the communications application of the one of the plurality of participants, upon detecting the speech impediment;
- and
- synchronizing the avatar with a speech of the one of the plurality of participants in real-time during the communication session to provide real-time visual feedback of the speech to the one of the plurality of participants.
A method for providing speech assistance during a virtual meeting comprising:
- receiving a request over a communication network to provide speech assistance during a virtual meeting between a plurality of participants;
- extracting a plurality of audio features from speech data of the virtual meeting;
- providing the plurality of audio features as an input to a speech impediment detection machine-learning (ML) model to detect a speech impediment and a type of the speech impediment for one of the plurality of participants;
- upon detecting the speech impediment, determining from the type of the speech impediment whether an avatar is needed;
- when the avatar is determined to be needed, generating the avatar for the one of the plurality of participants;
- synchronizing the avatar with a speech of the one of the plurality of participants in real-time during the virtual meeting to provide real-time visual feedback of the speech to the one of the plurality of participants;
- and
- providing the avatar for display on a user interface screen of a communications application to the one of the plurality of participants.
A non-transitory computer readable medium on which are stored instructions that, when executed, cause a programmable device to perform functions of:
- accessing speech data from a communication session between a plurality of participants, the communication session being conducted via a communications application over a network;
- extracting a plurality of audio features from the speech data of a virtual meeting;
- providing the plurality of audio features as an input to a speech impediment detection machine-learning (ML) model to detect a speech impediment and a type of the speech impediment for one of the plurality of participants;
- upon detecting the speech impediment, determining from the type of the speech impediment whether an avatar is needed;
- when the avatar is determined to be needed, utilizing a lip synchronization engine to generate the avatar that provides a visual representation of the one of the participants;
- providing the avatar for display on a user interface screen of the communications application of the one of the plurality of participants, upon detecting the speech impediment;
- and
- synchronizing the avatar with a speech of the one of the plurality of participants in real-time during the communication session to provide real-time visual feedback of the speech to the one of the plurality of participants.
Dependencies are inferred from the claim text.