Back to results
US 12,744,032 B2Granted
Synthetic video model for instructing a user to improve speech
AssigneeCisco Technology, Inc.
InventorsDavid C. White, Jr., Jay K. Johnston, Magnus Mortensen
Life of this patent
Estimating the term…
Abstract
A method, computer system, and computer program product are provided for improving user speech. A data sample of a user speaking one or more words is received, wherein the data sample includes video data and audio data of the user speaking. The data sample is analyzed to determine a correct articulation of a mouth when speaking the one or more words. A synthetic video of the user performing the correct articulation is generated. The synthetic video of the user is presented to the user. A live video of the user is presented to the user while the synthetic video is presented.
Claims20
A computer-implemented method comprising:
- receiving a data sample of a user speaking one or more words, wherein the data sample includes video data and audio data of the user speaking;
- analyzing the data sample to determine a correct articulation of a mouth when speaking the one or more words;
- analyzing the data sample to detect a tone, a volume, or a cadence of speech of the user;
- determining one or more feedback instructions based on one or more conditional rules related to the tone, the volume, or the cadence;
- determining at least one additional feedback instruction to correct a mouth articulation of the user, wherein the at least one additional feedback instruction is generated by a machine learning model that is trained based on one or more example feedback instructions and corresponding one or more samples of incorrect articulations;
- generating a synthetic video of the user performing the correct articulation;
- presenting the synthetic video of the user to the user;
- and
- presenting the one or more feedback instructions, the at least one additional feedback instruction, and a live video of the user to the user while the synthetic video is presented, wherein the mouth articulation of the user in the live video is arranged to superimpose the correct articulation in the synthetic video.
6 more elements
A computer system comprising:
- one or more computer processors;
- one or more computer readable storage media;
- and
- program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions to:
- receive a data sample of a user speaking one or more words, wherein the data sample includes video data and audio data of the user speaking;
- analyze the data sample to determine a correct articulation of a mouth when speaking the one or more words;
- analyze the data sample to detect a tone, a volume, or a cadence of speech of the user;
- determine one or more feedback instructions based on one or more conditional rules related to the tone, the volume, or the cadence;
- determine at least one additional feedback instruction to correct a mouth articulation of the user, wherein the at least one additional feedback instruction is generated by a machine learning model that is trained based on one or more example feedback instructions and corresponding one or more samples of incorrect articulations;
- generate a synthetic video of the user performing the correct articulation;
- present the synthetic video of the user to the user;
- and
- present the one or more feedback instructions, the at least one additional feedback instruction, and a live video of the user to the user while the synthetic video is presented, wherein the mouth articulation of the user in the live video is arranged to superimpose the correct articulation in the synthetic video.
10 more elements
A non-transitory computer program product comprising one or more computer readable storage media collectively having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
- receive a data sample of a user speaking one or more words, wherein the data sample includes video data and audio data of the user speaking;
- analyze the data sample to determine a correct articulation of a mouth when speaking the one or more words;
- analyze the data sample to detect a tone, a volume, or a cadence of speech of the user;
- determine one or more feedback instructions based on one or more conditional rules related to the tone, the volume, or the cadence;
- determine at least one additional feedback instruction to correct a mouth articulation of the user, wherein the at least one additional feedback instruction is generated by a machine learning model that is trained based on one or more example feedback instructions and corresponding one or more samples of incorrect articulations;
- generate a synthetic video of the user performing the correct articulation;
- present the synthetic video of the user to the user;
- and
- present the one or more feedback instructions, the at least one additional feedback instruction, and a live video of the user to the user while the synthetic video is presented, wherein the mouth articulation of the user in the live video is arranged to superimpose the correct articulation in the synthetic video.
6 more elements
Dependencies are inferred from the claim text.