# US 2021/0082452 A1 · System and Method for Talking Avatar

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0082452 A1
- Document ID: US20210082452A1
- Status: Published application
- Kind: A1
- Application number: 17/015,902
- Filed: 2020-09-09
- Published: 2021-03-18
- Earliest claimed: 2019-09-17
- Assignees: Rosetta Stone, Ltd.
- Applicants: Rosetta Stone, Ltd.
- Inventors: Carl Adrian Woffenden
- Web page: https://patentgrep.com/patent/US20210082452A1

## CPC

- G10L 21/10: Transforming into visible information
- G06T 13/00: Animation
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G09B 19/06: Foreign languages (with audible presentation of material to be studied G09B5/04)
- G10L 19/00: Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis (in musical instruments G10H)
- G10L 2021/105: {Synthesis of the lips movements from speech, e.g. for talking heads}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

Aspects of this disclosure provide techniques for generating a viseme and corresponding intensity pair. In some embodiments, the method includes generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription. The method may include generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset. The method may further include generating, by the server or a client end application, a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file, where the viseme is scheduled to align with a corresponding phoneme.

## Claims

### Claim 1

A method for generating a viseme and corresponding intensity pair, comprising:
generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset; and
generating, by the server or a client end application, a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file,
wherein the viseme is scheduled to align with a corresponding phoneme.

### Claim 2 (depends on 1)

The method of claim 1 further comprising storing the viseme and corresponding intensity pair in an intermediary file.

### Claim 3 (depends on 2)

The method of claim 2, wherein the viseme and corresponding intensity pair is stored in the intermediary file with specific timings for the viseme and corresponding intensity.

### Claim 4 (depends on 1)

The method of claim 1, wherein the frequency of decoding the compressed audio file is based on the size of the buffer.

### Claim 5 (depends on 1)

The method of claim 1 further comprising feeding, by the server, the PCM data to a user equipment.

### Claim 6 (depends on 1)

The method of claim 1 further comprising transmitting, by the server, the PCM data to a user equipment upon request.

### Claim 7 (depends on 1)

The method of claim 1 further comprising scheduling the phoneme to align with at least one of a corresponding mouth shape or facial expression of an animated character.

### Claim 8

A method for generating a viseme event and corresponding intensity pair, comprising:
generating, by a server, a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
generating, by the server, a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset; and
inserting, by the server, a viseme generator based at least on one of a processing buffer or the compressed audio file,
wherein the viseme is scheduled to align with a corresponding phoneme.

### Claim 9 (depends on 8)

The method of claim 8, wherein the scheduling of the viseme is based on the size of the processing buffer.

### Claim 10 (depends on 8)

The method of claim 8 further comprising scheduling the phoneme to align with at least one of a corresponding mouth shape or facial expression of an animated character.

### Claim 11 (depends on 10)

The method of claim 10, wherein the mouth shape or facial expression of the animated character is created based on the viseme.

### Claim 12 (depends on 8)

The method of claim 8, wherein the frequency of decoding the compressed audio file is based on the size of the processing buffer.

### Claim 13 (depends on 8)

The method of claim 8 further comprising feeding, by the server, the compressed audio file to a user equipment.

### Claim 14 (depends on 8)

The method of claim 8 further comprising transmitting, by the server, the compressed audio file to a user equipment upon request.

### Claim 15

A system for generating a viseme event and corresponding intensity pair, comprising:
a processor; and
a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:
generate a viseme and corresponding intensity pair based at least on one of a clean vocal track or corresponding transcription;
generate a compressed audio file based at least on one of the viseme, the corresponding intensity, music, or visual offset; and
generate a buffer of raw pulse-code modulated (PCM) data based on decoding at least a part of the compressed audio file,
wherein the viseme is scheduled to align with a corresponding phoneme.

### Claim 16 (depends on 15)

The system of claim 15, wherein the programming includes further instructions to store the viseme and corresponding intensity pair in an intermediary file.

### Claim 17 (depends on 16)

The system of claim 16, wherein the viseme and corresponding intensity pair is stored in the intermediary file with specific timings for the viseme and intensity.

### Claim 18 (depends on 15)

The system of claim 15, wherein the frequency of decoding the compressed audio file is based on the size of the buffer.

### Claim 19 (depends on 15)

The system of claim 15, wherein the programming includes further instructions to feed the PCM data to a user equipment.

### Claim 20 (depends on 15)

The system of claim 15, wherein the programming includes further instructions to transmit the PCM data to a user equipment upon request.
