# US 12,118,996 B2 · Method for processing voice signals of multiple speakers, and electronic device according thereto > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,996 B2 - Document ID: US12118996B2 - Status: Granted - Kind: B2 - Application number: 17/975,074 - Filed: 2022-10-27 - Published: 2023-02-09 - Granted: 2024-10-15 - Earliest claimed: 2017-12-19 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Young Ho Han; Nam Hoon Kim; Jae Young Roh; Chi Youn Park; Kyung Min Lee; Keun Seok Cho; Jong Youb Ryu - Web page: https://patentgrep.com/patent/US12118996B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 17/00: Speaker identification or verification techniques - G06F 3/16: Sound input; Sound output (speech processing G10L) - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/10: using distance or distortion measures between unknown speech and reference templates - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed is an electronic device. The electronic device includes a processor configured to execute one or more instructions stored in a memory to: control a receiver to receive a speech signal; determine whether the received speech signal includes speech signals of a plurality of different speakers; when the received speech signal includes the speech signals of the plurality of different speakers, detect feature information from a speech signal of each speaker; determine relations between pieces of speech content of the plurality of different speakers, based on the detected feature information; determine a response method based on the determined relations between the pieces of speech content; and control the electronic device such that an operation of the electronic device is performed according to the determined response method. ## Claims ### Claim 1 An electronic device comprising: a receiver receiving a speech signal; memory storing one or more computer programs; and one or more processors communicatively coupled to the receiver and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: control the receiver to receive the speech signal, determine whether the speech signal comprises speech signals of a plurality of different speakers, in response to determining that the speech signal comprises the speech signals of the plurality of different speakers, detect feature information from a speech signal of each speaker, based on the feature information, determine relationships between speech content of the plurality of different speakers, based on the determined relationships between the speech content of the plurality of different speakers, determine that speech content of a first speaker among the plurality of different speakers and speech content of a second speaker among the plurality of different speakers are generated in a same speech domain and that conflicts occur between the speech content of the first speaker and the speech content of the second speaker, and based on the determining that conflicts occur and the determined relationships between the speech content of the plurality of different speakers, control the electronic device and at least one other electronic device to perform an operation corresponding to each speech content of the plurality of different speakers. ### Claim 2 (depends on 1) The electronic device of claim 1, wherein the feature information comprises at least one of speaker information, a speaker position, a voice volume, a speech time point, a speech domain, speech content, an inter-speaker relation, or a relation between a previous speaker and a current speaker. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: in response to the determined relationships between the speech content of the plurality of different speakers indicating different operations to be performed by the electronic device, determine priorities of the speech content of the plurality of different speakers. ### Claim 4 (depends on 1) The electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: in response to the determined relationships between the speech content of the plurality of different speakers indicating a same operation to be performed by the electronic device, combine the speech content of the plurality of different speakers, and based on the combined speech content, determine a response method such that an operation is performed by the electronic device. ### Claim 5 (depends on 3) The electronic device of claim 3, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: determine the priorities of the speech content of the plurality of different speakers by using information about the plurality of different speakers and the speech content of the plurality of different speakers including the determined relationships between the speech content of the plurality of different speakers. ### Claim 6 (depends on 1) The electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, is cause the electronic device to: classify the speech content into at least one of speech content requiring a prompt response, speech content to be piled up in a queue, and speech content not requiring a response, and based on the classified speech content, determine a response method for the speech content of the plurality of different speakers. ### Claim 7 (depends on 3) The electronic device of claim 3, further comprising: a display; wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: in response to determining the priorities of the speech content of the plurality of different speakers, provide, through the display, response method information with respect to the speech content of the plurality of different speakers. ### Claim 8 (depends on 3) The electronic device of claim 3, further comprising: a communicator; wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: in response to determining the priorities of the speech content of the plurality of different speakers, provide, through the communicator to each device operated by the plurality of different speakers, response method information with respect to the speech content of the plurality of different speakers. ### Claim 9 A method of an electronic device processing a speech signal, the method comprising: receiving, by one or more processors of the electronic device, the speech signal through a receiver of the electronic device; determining, by the one or more processors, whether the speech signal comprises speech signals of a plurality of different speakers; in response to determining that the speech signal comprises the speech signals of the plurality of different speakers, detecting, by the one or more processors, feature information from a speech signal of each speaker; based on the feature information, determining, by the one or more processors, relationships between speech content of the plurality of different speakers; based on the determined relationships between the speech content of the plurality of different speakers, determining, by the one or more processors, that speech content of a first speaker among the plurality of different speakers and speech content of a second speaker among the plurality of different speakers are generated in a same speech domain and that conflicts occur between the speech content of the first speaker and the speech content of the second speaker; and based on the determining that conflicts occur and the determined relationships between the speech content of the plurality of different speakers, controlling, by the one or more processors, the electronic device and at least one other electronic device to perform an operation corresponding to each speech content of the plurality of different speakers. ### Claim 10 (depends on 9) The method of claim 9, wherein the feature information comprises at least one of speaker information, a speaker position, a voice volume, a speech time point, a speech domain, speech content, an inter-speaker relationship, or a relationship between a previous speaker and a current speaker. ### Claim 11 (depends on 9) The method of claim 9, further comprising: in response to the determined relationships between the speech content of the plurality of different speakers indicating different operations to be performed by the electronic device, determining, by the one or more processors, priorities of the speech content of the plurality of different speakers. ### Claim 12 (depends on 9) The method of claim 9, further comprising: in response to the determined relationships between the speech content of the plurality of different speakers indicating a same operation to be performed by the electronic device, combining, by the one or more processors, the speech content of the plurality of different speakers; and based on the combined speech content, determining, by the one or more processors, a response method such that an operation is performed by the electronic device. ### Claim 13 (depends on 11) The method of claim 11, wherein the determining of the priorities of the speech content of the plurality of different speakers is performed by using information about the plurality of different speakers and the speech content of the plurality of different speakers including the determined relationships between the speech content of the plurality of different speakers. ### Claim 14 (depends on 9) The method of claim 9, further comprising: classifying, by the one or more processors, the speech content into at least one of speech content requiring a prompt response, speech content to be piled up in a queue, and speech content requiring no response; and based on the classified speech content, determining, by the one or more processors, a response method for the speech content of the plurality of different speakers. ### Claim 15 (depends on 11) The method of claim 11, further comprising: in response to determining the priorities of speech content of the plurality of different speakers, providing, by the one or more processors through a display included in the electronic device, response method information with respect to the speech content of the plurality of different speakers. ### Claim 16 (depends on 11) The method of claim 11, further comprising: in response to determining the priorities of the speech content of the plurality of different speakers, providing, by the one or more processors through a communicator of the electronic device to each device operated by the plurality of different speakers, response method information with respect to the speech content of the plurality of different speakers. ### Claim 17 A non-transitory recording medium having recorded thereon a program comprising instructions which, when executed by one or more processors of an electronic device, cause the electronic device to perform operations, the operations comprising: receiving, by the one or more processors, a speech signal through a receiver of the electronic device; determining, by the one or more processors, whether the speech signal comprises speech signals of a plurality of different speakers; in response to determining that the speech signal comprises the speech signals of the plurality of different speakers, detecting, by the one or more processors, feature information from a speech signal of each speaker; based on the feature information, determining, by the one or more processors, relationships between speech content of the plurality of different speakers; based on the determined relationships between the speech content of the plurality of different speakers, determining, by the one or more processors, that speech content of a first speaker among the plurality of different speakers and speech content of a second speaker among the plurality of different speakers are generated in a same speech domain and that conflicts occur between the speech content of the first speaker and the speech content of the second speaker; and based on the determining that conflicts occur and the determined relationships between the speech content of the plurality of different speakers, controlling, by the one or more processors, the electronic device and at least one other electronic device to perform an operation corresponding to each speech content of the plurality of different speakers. ### Claim 18 (depends on 17) The non-transitory recording medium of claim 17, wherein the feature information comprises at least one of speaker information, a speaker position, a voice volume, a speech time point, a speech domain, speech content, an inter-speaker relationship, or a relationship between a previous speaker and a current speaker. ### Claim 19 (depends on 17) The non-transitory recording medium of claim 17, the operations further comprising: in response to the determined relationships between the speech content of the plurality of different speakers indicating different operations to be performed by the electronic device, determining, by the one or more processors, priorities of the speech content of the plurality of different speakers. ### Claim 20 (depends on 17) The non-transitory recording medium of claim 17, the operations further comprising: in response to the determined relationships between the speech content of the plurality of different speakers indicating a same operation to be performed by the electronic device, combining, by the one or more processors, the speech content of the plurality of different speakers; and based on the combined speech content, determining, by the one or more processors, a response method such that an operation is performed by the electronic device.