# US 2025/0095651 A1 · Synthesizing facial expressions and speech based on facial micromovements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095651 A1
- Document ID: US20250095651A1
- Status: Published application
- Kind: A1
- Application number: 18/910,636
- Filed: 2024-10-09
- Published: 2025-03-20
- Earliest claimed: 2022-07-20
- Assignees: Q (Cue) Ltd.
- Applicants: Q (Cue) Ltd.
- Inventors: Yonatan WEXLER
- Web page: https://patentgrep.com/patent/US20250095651A1

## CPC

- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06Q 20/40145: {Biometric identity checks}
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 10/82: using neural networks
- G06V 40/166: {using acquisition arrangements}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/174: {Facial expression recognition}
- G06V 40/176: {Dynamic expression}
- G06V 40/45: {Detection of the body part being alive}
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 13/02: Methods for producing synthetic speech; Speech synthesisers
- G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08)
- G10L 15/08: Speech classification or search
- G10L 15/16: using artificial neural networks
- G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
- G10L 17/04: Training, enrolment or model building
- G10L 17/10: Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G10L 25/84: for discriminating voice from noise
- H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)}
- H04L 63/108: {when the policy decisions are valid for a limited amount of time}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems, methods, and non-transitory computer-readable media including instructions for detecting and utilizing facial skin micromovements are disclosed. In some non-limiting embodiments, the detection of the facial skin micromovements occurs using a speech detection system that may include a wearable housing, a light source (either a coherent light source or a non-coherent light source), a light detector, and at least one processor. One or more processors may be configured to analyze light reflections received from a facial region to determine the facial skin micromovements, and extract meaning from the determined facial skin micromovements. Examples of meaning that may be extracted from the determined facial skin micromovements may include words spoken by the individual (either silently spoken or vocally spoken), an identification of the individual, an emotional state of the individual, a heart rate of the individual, a respiration rate of the individual, or any other biometric, emotion, or speech-related indicator.

## Claims

### Claim at position 1

1-160. (canceled)

### Claim 181

A non-transitory computer readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for generating synthesized representations of facial expressions, the operations comprising:
controlling at least one light source in a manner enabling illumination of a portion of a face;
receiving output signals from a light detector, wherein the output signals correspond to reflections of light from the portion of the face;
applying speckle analysis on the output signals to determine speckle analysis-based facial skin micromovements;
using the determined speckle analysis-based facial skin micromovements to identify at least one word prevocalized or vocalized during a time period;
using the determined speckle analysis-based facial skin micromovements to identify at least one change in a facial expression during the time period; and
during the time period, outputting data for causing a virtual representation of the face to mimic the at least one change in the facial expression in conjunction with an audio presentation of the at least one word.

### Claim 182 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein controlling the at least one light source in a manner enabling illumination of the portion of the face includes projecting a light pattern on the portion of the face.

### Claim 183 (depends on 182)

The non-transitory computer readable medium of claim 182, wherein the light pattern includes a plurality of spots.

### Claim 184 (depends on 182)

The non-transitory computer readable medium of claim 182, wherein the portion of the face includes cheek skin.

### Claim 185 (depends on 182)

The non-transitory computer readable medium of claim 182, wherein the portion of the face excludes lips.

### Claim 186 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the output signals from the light detector emanate from a wearable device.

### Claim 187 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the output signals from the light detector emanate from a non-wearable device.

### Claim 188 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the determined speckle analysis-based facial skin micromovements are associated with recruitment of at least one of: a zygomaticus muscle, an orbicularis oris muscle, a genioglossus muscle, a risorius muscle, or a levator labii superioris alaeque nasi muscle.

### Claim 189 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the at least one change in the facial expression during the period of time includes speech-related facial expressions and non-speech-related facial expressions.

### Claim 190 (depends on 189)

The non-transitory computer readable medium of claim 189, wherein the virtual representation of the face is associated with an avatar of an individual from whom the output signals are derived, and wherein mimicking the at least one change in the facial expression includes causing visual changes to the avatar that reflect at least one of the speech-related facial expressions and the non-speech-related facial expressions.

### Claim 191 (depends on 190)

The non-transitory computer readable medium of claim 190, wherein the visual changes to the avatar involve changing a color of at least a portion of the avatar.

### Claim 192 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the audio presentation of the at least one word is based on a recording of an individual.

### Claim 193 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the audio presentation of the at least one word is based on a synthesized voice.

### Claim 194 (depends on 193)

The non-transitory computer readable medium of claim 193, wherein the synthesized voice corresponds with a voice of an individual from whom the output signals are derived.

### Claim 195 (depends on 193)

The non-transitory computer readable medium of claim 193, wherein the synthesized voice corresponds with template voice selected by an individual from whom the output signals are derived.

### Claim 196 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the operations further include determining an emotional state of an individual from whom the output signals are derived based at least in part on the facial skin micromovements and augmenting the virtual representation of the face to reflect the determined emotional state.

### Claim 197 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the operations further include receiving a selection of a desired emotional state, and augmenting the virtual representation of the face to reflect the selected emotional state.

### Claim 198 (depends on 181)

The non-transitory computer readable medium of claim 181, wherein the operations further include identifying a non-desirable facial expression, and wherein the outputted data for causing the virtual representation omits data for causing the non-desirable facial expression.

### Claim 199

A method for generating synthesized representations of facial expressions, the method comprising:
controlling at least one light source in a manner enabling illumination of a portion of a face;
receiving output signals from a light detector, wherein the output signals correspond to reflections of light from the portion of the face;
applying speckle analysis on the output signals to determine speckle analysis-based facial skin micromovements;
using the determined speckle analysis-based facial skin micromovements to identify at least one word prevocalized or vocalized during a time period;
using the determined speckle analysis-based facial skin micromovements to identify at least one change in a facial expression during the time period; and
during the time period, outputting data for causing a virtual representation of the face to mimic the at least one change in the facial expression in conjunction with an audio presentation of the at least one word.

### Claim 200

A system for generating synthesized representations of facial expressions, the system comprising:
at least one processor configured to:
control at least one light source in a manner enabling illumination of a portion of a face;
receive output signals from a light detector, wherein the output signals correspond to reflections of light from the portion of the face;
apply speckle analysis on the output signals to determine speckle analysis-based facial skin micromovements;
use the determined speckle analysis-based facial skin micromovements to identify at least one word prevocalized or vocalized during a time period;
use the determined speckle analysis-based facial skin micromovements to identify at least one change in a facial expression during the time period; and
during the time period, output data for causing a virtual representation of the face to mimic the at least one change in the facial expression in conjunction with an audio presentation of the at least one word.
with an audio presentation of the at least one word.

### Claim at position 22

201-560. (canceled)
