# US 12,216,750 B2 · Earbud with facial micromovement detection capabilities > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,750 B2 - Document ID: US12216750B2 - Status: Granted - Kind: B2 - Application number: 18/512,925 - Filed: 2023-11-17 - Published: 2024-04-18 - Granted: 2025-02-04 - Earliest claimed: 2021-08-04 - Assignees: Q (Cue) Ltd. - Applicants: Q (Cue) Ltd. - Inventors: Yonatan Wexler - Web page: https://patentgrep.com/patent/US12216750B2 ## CPC - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - A61B 5/0064: {Body surface scanning} - A61B 5/117: Identification of persons (methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00) - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles} - G06F 9/541: {via adapters, e.g. between incompatible applications} - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06V 40/166: {using acquisition arrangements} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/172: {Classification, e.g. identification} - G06V 40/176: {Dynamic expression} - G06V 40/40: Spoof detection, e.g. liveness detection - G06V 40/70: Multimodal biometrics, e.g. combining information from different biometric modalities - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 15/08: Speech classification or search - G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/25: using position of the lips, movement of the lips or face analysis - H04R 1/028: {associated with devices performing functions other than acoustics, e.g. electric candles} - H04R 1/10: Earpieces; Attachments therefor {; Earphones; Monophonic headphones (H04R1/28 takes precedence; stereophonic headphones H04R5/033)} - G10L 2015/088: {Word spotting} - G10L 2015/223: {Execution procedure of a spoken command} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A multifunctional earpiece comprising an ear-mountable housing, a speaker integrated with the ear-mountable housing for presenting sound, a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face, a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, and wherein the multifunctional earpiece is configured to simultaneously present the sound through the speaker, project the light toward the skin, and detect the received reflections indicative of the prevocalized words. ## Claims ### Claim 1 A multifunctional earpiece, comprising: an ear-mountable housing; a speaker integrated with the ear-mountable housing for presenting sound; a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face; a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and wherein the multifunctional earpiece is configured to simultaneously present the sound through the speaker, project the light toward the skin, and use image processing on the produced at least one image to determine the prevocalized words. ### Claim 2 (depends on 1) The multifunctional earpiece of claim 1, wherein at least a portion of the ear-mountable housing is configured to be placed in an ear canal. ### Claim 3 (depends on 1) The multifunctional earpiece of claim 1, wherein at least a portion of the ear-mountable housing is configured to be placed over or behind an ear. ### Claim 4 (depends on 1) The multifunctional earpiece of claim 1, further comprising at least one processor configured to output via the speaker an audible simulation of the prevocalized words determined from at least one image. ### Claim 5 (depends on 4) The multifunctional earpiece of claim 4, wherein the audible simulation of the prevocalized words includes a synthetization of a voice of an individual other than the wearer. ### Claim 6 (depends on 4) The multifunctional earpiece of claim 4, wherein the audible simulation of the prevocalized words includes a synthetization of the prevocalized words in a first language other than a second language of the prevocalized words. ### Claim 7 (depends on 1) The multifunctional earpiece of claim 1, further comprising a microphone integrated with the ear-mountable housing for receiving audio indicative of a wearer's speech. ### Claim 8 (depends on 1) The multifunctional earpiece of claim 1, wherein the light source is configured to project a pattern of coherent light toward the skin of the wearer's face, the pattern including a plurality of spots. ### Claim 9 (depends on 1) The multifunctional earpiece of claim 1, wherein the light detector is configured to output associated reflection signals indicative of muscle fiber recruitments. ### Claim 10 (depends on 9) The multifunctional earpiece of claim 9, wherein the recruited muscle fibers include at least one of zygomaticus muscle fibers, orbicularis oris muscle fibers, risorius muscle fibers, or levator labii superioris alaeque nasi muscle fibers. ### Claim 11 (depends on 1) The multifunctional earpiece of claim 1, further comprising at least one processor configured to analyze the light reflections to determine the facial skin micromovements. ### Claim 12 (depends on 11) The multifunctional earpiece of claim 11, wherein the analysis includes speckle analysis. ### Claim 13 (depends on 11) The multifunctional earpiece of claim 11, further comprising a microphone integrated with the ear-mountable housing for receiving audio indicative of a wearer's speech, and wherein the at least one processor is configured to use the audio received via the microphone and the reflections received via the light detector to correlate facial skin micromovements with spoken words and to train a neural network to determine subsequent prevocalized words from subsequent facial skin micromovements. ### Claim 14 (depends on 11) The multifunctional earpiece of claim 11, wherein the at least one processor is configured to identify a trigger in the determined facial skin micromovements for activating the microphone. ### Claim 15 (depends on 11) The multifunctional earpiece of claim 11, further comprising a pairing interface for pairing with a communications device, and wherein the at least one processor is configured to transmit an audible simulation of the prevocalized words to the communications device. ### Claim 16 (depends on 11) The multifunctional earpiece of claim 11, further comprising a pairing interface for pairing with a communications device, and wherein the at least one processor is configured to transmit a textual presentation of the prevocalized words to the communications device. ### Claim 17 (depends on 1) The multifunctional earpiece of claim 1, wherein the light source is configured to project coherent light toward the skin of the wearer's face. ### Claim 18 (depends on 1) The multifunctional earpiece of claim 1, wherein the light source is configured to project a noncoherent light toward the skin of the wearer's face. ### Claim 19 A method for operating a multifunctional earpiece, the method comprising: operating a speaker integrated with an ear-mountable housing associated with the multifunctional earpiece for presenting sound; operating a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face; operating a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and simultaneously presenting the sound through the speaker, projecting the light toward the skin, and using image processing on the produced at least one image to determine the prevocalized words. ### Claim 20 A non-transitory computer readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for operating a multifunctional earpiece, the operations comprising: operating a speaker integrated with an ear-mountable housing associated with the multifunctional earpiece for presenting sound; operating a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face; operating a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and simultaneously presenting the sound through the speaker, projecting the light toward the skin, and using image processing on the produced at least one image to determine the prevocalized words.