Generating a User Avatar for Video Communications
AssigneeRoku, Inc.
InventorsSunil Ramesh, Michael Cutter, Karina Levitian
Life of this patent
Estimating the term…
Abstract
In one aspect, an example method includes (i) receiving audio from a first client device engaged in a communication session with a second client device, the audio comprising one or more words spoken by a user of the first client device; (ii) using the audio and a facial feature model to estimate facial movement that corresponds to the one or more words spoken by the user; (iii) generating a synthetic video depicting an avatar of the user moving according to the estimated facial movement; and (iv) in response to generating the synthetic video, causing the second client device to present the synthetic video synchronized with the audio.
Claims20
A computing system comprising a processor and a non-transitory computer-readable storage medium having stored thereon program instructions that upon execution by the processor cause the computing system to perform configured for performing a set of acts comprising:
- receiving audio from a first client device engaged in a communication session with a second client device, the audio comprising one or more words spoken by a user of the first client device;
- using the audio and a facial feature model to estimate facial movement that corresponds to the one or more words spoken by the user, wherein the facial feature model comprises a classifier configured to map each of a plurality of words to a respective one of a plurality of facial movements;
- receiving one or more pre-recorded videos of the user;
- using the one or more pre-recorded videos as training data to train the classifier;
- generating a synthetic video depicting an avatar of the user moving according to the estimated facial movement;
- and
- in response to generating the synthetic video, causing the second client device to present the synthetic video synchronized with the audio.
(canceled)
(canceled)
A method performed by a computing system, the method comprising:
- receiving audio from a first client device engaged in a communication session with a second client device, the audio comprising one or more words spoken by a user of the first client device;
- using the audio and a facial feature model to estimate facial movement that corresponds to the one or more words spoken by the user, wherein the facial feature model comprises a classifier configured to map each of a plurality of words to a respective one of a plurality of facial movements;
- receiving one or more pre-recorded videos of the user;
- using the one or more pre-recorded videos as training data to train the classifier;
- generating a synthetic video depicting an avatar of the user moving according to the estimated facial movement;
- and
- in response to generating the synthetic video, causing the second client device to present the synthetic video synchronized with the audio.
(canceled)
(canceled)
A non-transitory computer-readable medium having stored thereon program instructions that upon execution by a computing system, cause performance of a set of acts comprising:
- receiving audio from a first client device engaged in a communication session with a second client device, the audio comprising one or more words spoken by a user of the first client device;
- using the audio and a facial feature model to estimate facial movement that corresponds to the one or more words spoken by the user, wherein the facial feature model comprises a classifier configured to map each of a plurality of words to a respective one of a plurality of facial movements;
- receiving one or more pre-recorded videos of the user;
- using the one or more pre-recorded videos as training data to train the classifier;
- generating a synthetic video depicting an avatar of the user moving according to the estimated facial movement;
- and
- in response to generating the synthetic video, causing the second client device to present the synthetic video synchronized with the audio.
(canceled)
(canceled)
Dependencies are inferred from the claim text.