# US 11,908,478 B2 · Determining speech from facial skin movements using a housing supported by ear or associated with an earphone > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,478 B2 - Document ID: US11908478B2 - Status: Granted - Kind: B2 - Application number: 18/179,632 - Filed: 2023-03-07 - Published: 2023-07-20 - Granted: 2024-02-20 - Earliest claimed: 2021-08-04 - Assignees: Q (Cue) Ltd. - Applicants: Q (Cue) Ltd. - Inventors: Aviad Maizels; Avi Barliya; Yonatan Wexler - Web page: https://patentgrep.com/patent/US11908478B2 ## CPC - G10L 15/25: using position of the lips, movement of the lips or face analysis - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes} - A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 20/00: Machine learning - G06V 10/141: Control of illumination - G06V 10/147: Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) - G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model - G06V 10/82: using neural networks - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/174: {Facial expression recognition} - G06V 40/176: {Dynamic expression} - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) - G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management - G10L 15/16: using artificial neural networks - G10L 15/24: Speech recognition using non-acoustical features - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - A61B 5/004: {adapted for image acquisition of a particular organ or body part (A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062)} - A61B 5/1114: {Tracking parts of the body} - A61B 5/7267: {involving training the classification device} - A61B 5/7475: {User input or interface means, e.g. keyboard, pointing device, joystick} - A61B 2562/0233: Special features of optical sensors or probes classified in A61B5/00 - A61B 2576/02: specially adapted for a particular organ or body part - G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection} - G06N 3/02: Neural networks - G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings - G06V 10/245: {by locating a pattern; Special marks for positioning} - G06V 40/16: Human faces, e.g. facial parts, sketches or expressions - G10L 13/00: Speech synthesis; Text to speech systems - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for generating speech includes uploading a reference set of features that were extracted from sensed movements of one or more target regions of skin on faces of one or more reference human subjects in response to words articulated by the subjects and without contacting the one or more target regions. A test set of features is extracted a from the sensed movements of at least one of the target regions of skin on a face of a test subject in response to words articulated silently by the test subject and without contacting the one or more target regions. The extracted test set of features is compared to the reference set of features, and, based on the comparison, a speech output is generated, that includes the articulated words of the test subject. ## Claims ### Claim 1 A wearable sensing system for determining speech based on minute facial skin surface movements, the wearable sensing system comprising: a housing configured to be worn on a head of a user and to be supported by an ear of the user; at least one coherent light source associated with the housing and configured to direct light towards a facial region of the head; at least one sensor associated with the housing and configured to receive light source reflections from the facial region and to output associated reflection signals; at least one processor configured to: receive the reflection signals from the at least one sensor in a time interval; analyze the reflection signals to identify minute facial movements in the time interval; decipher the minute facial movements to determine associated speech; and generate output of the associated speech. ### Claim 2 (depends on 1) The wearable sensing system of claim 1, wherein the at least one coherent light source is configured to direct a plurality of beams of coherent light toward different locations on the facial region of the head thus creating an array of spots. ### Claim 3 (depends on 2) The wearable sensing system of claim 2, wherein the coherent light source is configured to cause an area of the array of spots to be at least 1 cm2. ### Claim 4 (depends on 2) The wearable sensing system of claim 2, wherein the coherent light source is configured to cause the area of the array of spots to be at least 4 cm2. ### Claim 5 (depends on 1) The wearable sensing system of claim 1, wherein the housing is supported by the ear of the user via association with a pair of spectacles. ### Claim 6 (depends on 5) The wearable sensing system of claim 5, wherein the at least one coherent light source is integrated with a frame of the pair of spectacles. ### Claim 7 (depends on 1) The wearable sensing system of claim 1, wherein the housing is configured such that when worn, the at least one sensor is held at a distance from a skin surface. ### Claim 8 (depends on 7) The wearable sensing system of claim 7, wherein the housing is configured such that the at least one sensor is held at least 5 mm from the skin surface. ### Claim 9 (depends on 1) The wearable sensing system of claim 1, wherein the output is audio, and wherein the system further comprises a speaker configured to output the speech in audio form. ### Claim 10 (depends on 9) The wearable sensing system of claim 9, wherein the audio output is a synthetization of the determined associated speech. ### Claim 11 (depends on 1) The wearable sensing system of claim 1, further comprising a microphone configured to sense sounds uttered by the user. ### Claim 12 (depends on 11) The wearable sensing system of claim 11, wherein the at least one processor is configured to calibrate spoken words detected by the microphone with the identified minute facial movements. ### Claim 13 (depends on 1) The wearable sensing system of claim 1, wherein the housing is configured, when worn, to assume an aiming direction of the at least one sensor for illuminating a portion of a cheek of the user. ### Claim 14 (depends on 1) The wearable sensing system of claim 1, wherein the generated output is textual output corresponding to the determined associated speech. ### Claim 15 (depends on 1) The wearable sensing system of claim 1, wherein the generated output is audible output of the determined associated speech. ### Claim 16 (depends on 1) The wearable sensing system of claim 1, wherein in deciphering the minute facial movements, the at least one processor is configured to determine activation of facial muscles associated with the minute facial movements. ### Claim 17 (depends on 1) The wearable sensing system of claim 1, wherein deciphering the minute facial movements is used to determine silent speech. ### Claim 18 (depends on 1) The wearable sensing system of claim 1, wherein deciphering the minute facial movements is used to determine vocalized speech. ### Claim 19 A wearable sensing system for determining speech based on minute facial skin surface movements, the wearable sensing system comprising: a housing configured to be worn on a head of a user, wherein the housing is associated with an earphone; at least one coherent light source associated with the housing and configured to direct light towards a facial region of the head; at least one sensor associated with the housing and configured to receive light source reflections from the facial region and to output associated reflection signals; at least one processor configured to: receive the reflection signals from the at least one sensor in a time interval; analyze the reflection signals to identify minute facial movements in the time interval; decipher the minute facial movements to determine associated speech; and generate output of the associated speech. ### Claim 20 (depends on 19) The wearable sensing system of claim 19, wherein the earphone has an arm extending therefrom and wherein the at least one coherent light source is located in the arm. ### Claim 21 A wearable sensing system for determining speech based on minute facial skin surface movements, the wearable sensing system comprising: a housing configured to be worn on a head of a user; at least one coherent light source associated with the housing and configured to direct light towards a facial region of the head; at least one sensor associated with the housing and configured to receive light source reflections from the facial region and to output associated reflection signals, wherein the housing is configured such that when worn, the at least one sensor is held at a distance of at least 2 cm from the skin surface; and at least one processor configured to: receive the reflection signals from the at least one sensor in a time interval; analyze the reflection signals to identify minute facial movements in the time interval; decipher the minute facial movements to determine associated speech; and generate output of the associated speech. ### Claim 22 A method for determining speech based on minute facial skin movements, the method comprising: controlling at least one coherent light source to generate a plurality of beams of coherent light for direction towards different locations on a facial region of a head of a user thus creating an array of spots of at least 1 cm2; receiving via at least one sensor reflections of the light from the facial region in a time interval and outputting associated reflection signals; analyzing the reflection signals to identify minute facial movements captured during the time interval; deciphering the minute facial movements to determine associated speech; and generating output of the associated speech.