# US 12,014,741 B2 · Electronic device and controlling method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,014,741 B2 - Document ID: US12014741B2 - Status: Granted - Kind: B2 - Application number: 17/503,849 - Filed: 2021-10-18 - Published: 2022-08-25 - Granted: 2024-06-18 - Earliest claimed: 2021-02-23 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Jubum Han; Yeaseul Song; Jaewon Lee - Web page: https://patentgrep.com/patent/US12014741B2 ## CPC - G06N 3/08: Learning methods - G10L 17/18: Artificial neural networks; Connectionist approaches - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G10L 15/08: Speech classification or search - G10L 17/04: Training, enrolment or model building - G10L 17/14: Use of phonemic categorisation or speech recognition prior to speaker recognition or verification - G10L 17/24: the user being prompted to utter a password or a predefined phrase - G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) - H04R 1/1016: {Earpieces of the intra-aural type} - H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} - G10L 2015/088: {Word spotting} - H04R 1/1091: {Details not provided for in groups H04R1/1008 - H04R1/1083} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract An electronic device including a microphone; an output interface; and a processor configured to: based on a word registered on the electronic device by a user, detect a voice of a talker who utters the registered word from an audio signal received through the microphone; based on voice information of a registered talker who is registered on the electronic device, determine whether the detected voice is a voice of the registered talker; and based on the detected voice being determined as the voice of the registered talker, control the output interface to output a voice notification, wherein the registered word refers to the user. ## Claims ### Claim 1 An electronic device comprising: a microphone; an output interface; and a processor configured to: obtain a plurality of audio signals through the microphone; and cluster the plurality of audio signals as a plurality of groups, based on feature vectors of the plurality of audio signals, wherein the processor is further configured to: based on a word registered on the electronic device by a user, detect a voice of a talker who utters the registered word from an audio signal received through the microphone; based on voice information of a registered talker who is registered on the electronic device, determine whether the detected voice is a voice of the registered talker; based on determining that the detected voice is the voice of the registered talker, control the output interface to output a voice notification; and based on determining that the detected voice is not the voice of the registered talker, receive a command for registering voice information of the talker through the microphone; and based on the command, register information about the talker and a group corresponding to the voice of the talker from among the plurality of groups, and wherein the registered word refers to the user. ### Claim 2 (depends on 1) The device according to claim 1, wherein the voice notification indicates that the registered talker is calling the user. ### Claim 3 (depends on 1) The device according to claim 1, wherein the processor is further configured to: input a voice for the registered word to a first neural network model and obtain a first output value from the first neural network model; input a voice obtained using the microphone to the first neural network model and obtain a second output value from the first neural network model; and detect the voice of the talker who utters the registered word from the audio signal received through the microphone based on the first output value and the second output value. ### Claim 4 (depends on 3) The device according to claim 3, wherein the processor is further configured to obtain the voice for the registered word based on at least one of a text and a voice of the user that is input to the electronic device to register the word. ### Claim 5 (depends on 1) The device according to claim 1, wherein the processor is further configured to: input the voice for the registered word and the voice information of the registered talker to a second neural network model and obtain a converted voice that is converted from the voice for the registered word based on a voice style of the registered talker; and determine whether the detected voice is the voice of the registered talker based on the converted voice. ### Claim 6 (depends on 5) The device according to claim 5, wherein the processor is further configured to: input the converted voice to a third neural network model and obtain a third output value from the third neural network model; input the detected voice to the third neural network model and obtain a fourth output value from the third neural network model; and determine whether the detected voice is the voice of the registered talker based on the third output value and the fourth output value. ### Claim 7 (depends on 5) The device according to claim 5, wherein the voice information of the registered talker comprises a voice uttered by the registered talker. ### Claim 8 (depends on 1) The electronic device according to claim 1, wherein a new registered talker is registered to a group selected by the user from among the plurality of groups displayed in a user interface. ### Claim 9 A method for controlling an electronic device comprising a microphone, the method comprising: obtaining a plurality of audio signals through the microphone; clustering the plurality of audio signals as a plurality of groups, based on identifying feature vectors of the plurality of audio signals; based on a word registered on the electronic device by a user, detecting a voice of a talker who utters the registered word from an audio signal received through the microphone; based on voice information of a registered talker who is registered on the electronic device, determining by the electronic device whether the detected voice is a voice of the registered talker; and based on determining that the detected voice is the voice of the registered talker, outputting a voice notification; and based on determining that the detected voice is not the voice of the registered talker, receiving a command for registering voice information of the talker through the microphone, wherein the command comprises an input of information about a group including the voice of the talker from among the plurality of groups, and an input of information about the talker, and wherein the registered word refers to the user. ### Claim 10 (depends on 9) The method according to claim 9, wherein the voice notification indicates that the registered talker is calling the user. ### Claim 11 (depends on 9) The method according to claim 9, further comprising: inputting voice for the registered word to a first neural network model and obtaining a first output value from the first neural network model; inputting a voice obtained using the microphone to the first neural network model and obtaining a second output value from the first neural network model; and identifying the voice of the talker who utters the registered word from the audio signal received through the microphone based on the first output value and the second output value. ### Claim 12 (depends on 11) The method according to claim 11, further comprising: obtaining the voice for the registered word based on at least one of a text and a voice of the user that is input to the electronic device to register the word. ### Claim 13 (depends on 9) The method according to claim 9, further comprising: inputting the voice for the registered word and the voice information of the registered talker to a second neural network model and obtaining a converted voice that is converted from the voice for the registered word based on a voice style of the registered talker; and determining whether the detected voice is the voice of the registered talker based on the converted voice. ### Claim 14 (depends on 13) The method according to claim 13, further comprising: inputting the converted voice to a third neural network model and obtaining a third output value from the third neural network model; inputting the detected voice to the third neural network model and obtaining a fourth output value from the third neural network model; and determining whether the detected voice is the voice of the registered talker based on the third output value and the fourth output value. ### Claim 15 (depends on 13) The method according to claim 13, wherein the voice information of the registered talker comprises a voice uttered by the registered talker.