System and method for talking avatar
AssigneeLexia Learning Systems LLC
InventorCarl Adrian Woffenden
Life of this patent
Estimating the term…
Abstract
Aspects of this disclosure provide techniques for generating a viseme and corresponding intensity pair. In some embodiments, the method includes generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription. The method may include generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset. The method may further include generating, by the server or a client end application, a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file, where the viseme is scheduled to align with a corresponding phoneme.
Claims20
A method for generating a viseme and corresponding intensity pair, comprising:
- generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
- generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset;
- and
- generating, by the server or a client end application, a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file,
- wherein the viseme is scheduled to align with a corresponding phoneme, and scheduled to coincide when a user hears a sound to allow two or more mouth shapes or facial expressions to be in an expected position using at least one visual offset, the at least one visual offset being used to compensate for longer blending between the two or more mouth shapes, and wherein the scheduling is based at least on one of:
- a size of a decoder audio buffer for the compressed audio file,
- a size of a processing buffer, or
- a latency between transferring the decoder audio buffer to the client end application and the sound being heard.
A method for generating a viseme event and corresponding intensity pair, comprising:
- generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
- generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset;
- and
- inserting, by the server, a viseme generator based at least on one of a processing buffer or the compressed audio file,
- wherein the viseme is scheduled to align with a corresponding phoneme, and scheduled to coincide when a user hears a sound to allow two or more mouth shapes or facial expressions to be in an expected position using at least one visual offset, the at least one visual offset being used to compensate for longer blending between the two or more mouth shapes, and wherein the scheduling is based at least on one of:
- a size of a decoder audio buffer for the compressed audio file,
- a size of the processing buffer, or
- a latency between transferring the decoder audio buffer to a client end application and the sound being heard.
A system for generating a viseme event and corresponding intensity pair, comprising:
- a processor;
- and
- a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:
- generate a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
- generate a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset;
- and
- generate a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file,
- wherein the viseme is scheduled to align with a corresponding phoneme, and scheduled to coincide when a user hears a sound to allow two or more mouth shapes or facial expressions to be in an expected position using at least one visual offset, the at least one visual offset being used to compensate for longer blending between the two or more mouth shapes, and wherein the scheduling is based at least on one of:
- a size of a decoder audio buffer for the compressed audio file,
- a size of a processing buffer, or
- a latency between transferring the decoder audio buffer to a client end application and the sound being heard.
Dependencies are inferred from the claim text.