# US 11,443,769 B2 · Enhancing audio using multiple recording devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,769 B2 - Document ID: US11443769B2 - Status: Granted - Kind: B2 - Application number: 17/194,827 - Filed: 2021-03-08 - Published: 2021-06-24 - Granted: 2022-09-13 - Earliest claimed: 2015-09-16 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Dimitri Kanevsky; Golan Pundak - Web page: https://patentgrep.com/patent/US11443769B2 ## CPC - G11B 20/10527: {Audio or video recording; Data buffering arrangements (G11B20/12 - G11B20/18 take precedence)} - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - G10L 17/00: Speaker identification or verification techniques - G10L 21/0364: for improving intelligibility - G10L 25/51: for comparison or discrimination - H04M 3/56: Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities (video conference systems H04N7/15) - H04M 3/568: {audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30)} - G10L 21/0208: Noise filtering - G10L 21/028: using properties of sound source - G10L 25/84: for discriminating voice from noise - G11B 2020/10546: {specifically adapted for audio data} - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for identifying that a first audio stream includes first, second, and third sources of audio. A computing system identifies that a second audio stream includes the first, second, and third sources of audio. The computing system determines that the first and second sources of audio are part of a first conversation. The computing system generates a third audio stream that combines the first source of audio from the first audio stream, the first source of audio from the second audio stream, the second source of audio from the first audio stream, and the second source of audio from the second audio stream, and diminishes the third source of audio from the first audio stream, and the third source of audio from the second audio stream. ## Claims ### Claim 1 A computer-implemented method for enhancing audio, the method comprising: receiving, using a hardware processor, an audio stream for playback on a media device; extracting, using the hardware processor, a first audio source, a second audio source, and a third audio source from the audio stream; determining, using the hardware processor, that a conversation between the first audio source and the second audio source occurs within a first portion of the audio stream and a second conversation between the first audio source and the third audio source occurs within a second portion of the audio stream at a second time point; and generating, using the hardware processor, an updated audio stream that enhances the first audio source and the second audio source extracted from the first portion of the audio stream and diminishes the third audio source extracted from the first portion of the audio stream and that enhances the first audio source and the third audio source extracted from the second portion of the audio stream and diminishes the second audio source extracted from the second portion of the audio stream. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising receiving a second audio stream for playback on the media device, wherein the conversation between the first audio and the second audio source is determined to occur within the audio stream and the second audio stream and wherein the update audio stream is generated by enhancing the first audio source from the audio stream, the first audio source from the second audio stream, the second audio source from the audio stream, and the second audio source from the second audio stream and diminishing the third audio source from the audio stream and the third audio source from the second audio stream. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein a first ratio of an amplitude of the first audio source in the audio stream to the second audio source in the audio stream is different than a second ratio of an amplitude of the first audio source in the second audio stream to the second audio source in the second audio stream. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein a third ratio of the first audio source in the updated audio stream to the second audio source in the updated audio stream is different than the first ratio and the second ratio. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving a second audio stream for playback on the media device; determining that a third conversation between the third audio source and a fourth audio source occurs within the second audio stream; and generating a second updated audio stream that enhances the third audio source and the fourth audio source extracted from the second audio stream and diminishes the first audio source and the second audio source from the second audio stream. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the first audio source, the second audio source, and the third audio source are extracted from the audio stream by decomposing the audio stream into a first audio portion in which the first audio source is speaking, a second audio portion in which the second audio source is speaking, and a third audio portion in which the third audio source is speaking. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising transmitting the updated audio stream to the media device while inhibiting the transmission of the updated audio stream to a second media device. ### Claim 8 A system for enhancing audio, the system comprising: a hardware processor that: receives an audio stream for playback on a media device; extracts a first audio source, a second audio source, and a third audio source from the audio stream; determines that a conversation between the first audio source and the second audio source occurs within a first portion of the audio stream and a second conversation between the first audio source and the third audio source occurs within a second portion of the audio stream at a second time point; and generates an updated audio stream that enhances the first audio source and the second audio source extracted from the first portion of the audio stream and diminishes the third audio source extracted from the first portion of the audio stream and that enhances the first audio source and the third audio source extracted from the second portion of the audio stream and diminishes the second audio source extracted from the second portion of the audio stream. ### Claim 9 (depends on 8) The system of claim 8, wherein the hardware processor also receives a second audio stream for playback on the media device, wherein the conversation between the first audio and the second audio source is determined to occur within the audio stream and the second audio stream and wherein the update audio stream is generated by enhancing the first audio source from the audio stream, the first audio source from the second audio stream, the second audio source from the audio stream, and the second audio source from the second audio stream and diminishing the third audio source from the audio stream and the third audio source from the second audio stream. ### Claim 10 (depends on 9) The system of claim 9, wherein a first ratio of an amplitude of the first audio source in the audio stream to the second audio source in the audio stream is different than a second ratio of an amplitude of the first audio source in the second audio stream to the second audio source in the second audio stream. ### Claim 11 (depends on 10) The system of claim 10, wherein a third ratio of the first audio source in the updated audio stream to the second audio source in the updated audio stream is different than the first ratio and the second ratio. ### Claim 12 (depends on 8) The system of claim 8, wherein the hardware processor also: receives a second audio stream for playback on the media device; determines that a third conversation between the third audio source and a fourth audio source occurs within the second audio stream; and generates a second updated audio stream that enhances the third audio source and the fourth audio source extracted from the second audio stream and diminishes the first audio source and the second audio source from the second audio stream. ### Claim 13 (depends on 8) The system of claim 8, wherein the first audio source, the second audio source, and the third audio source are extracted from the audio stream by decomposing the audio stream into a first audio portion in which the first audio source is speaking, a second audio portion in which the second audio source is speaking, and a third audio portion in which the third audio source is speaking. ### Claim 14 (depends on 8) The system of claim 8, wherein the hardware processor also transmits the updated audio stream to the media device while inhibiting the transmission of the updated audio stream to a second media device. ### Claim 15 A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform a method for enhancing audio, the method comprising: receiving an audio stream for playback on a media device; extracting a first audio source, a second audio source, and a third audio source from the audio stream; determining that a conversation between the first audio source and the second audio source occurs within a first portion of the audio stream and a second conversation between the first audio source and the third audio source occurs within a second portion of the audio stream at a second time point; and generating an updated audio stream that enhances the first audio source and the second audio source extracted from the first portion of the audio stream and diminishes the third audio source extracted from the first portion of the audio stream and that enhances the first audio source and the third audio source extracted from the second portion of the audio stream and diminishes the second audio source extracted from the second portion of the audio stream. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the method further comprises receiving a second audio stream for playback on the media device, wherein the conversation between the first audio and the second audio source is determined to occur within the audio stream and the second audio stream and wherein the update audio stream is generated by enhancing the first audio source from the audio stream, the first audio source from the second audio stream, the second audio source from the audio stream, and the second audio source from the second audio stream and diminishing the third audio source from the audio stream and the third audio source from the second audio stream. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein a first ratio of an amplitude of the first audio source in the audio stream to the second audio source in the audio stream is different than a second ratio of an amplitude of the first audio source in the second audio stream to the second audio source in the second audio stream. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein a third ratio of the first audio source in the updated audio stream to the second audio source in the updated audio stream is different than the first ratio and the second ratio. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the method further comprises: receiving a second audio stream for playback on the media device; determining that a third conversation between the third audio source and a fourth audio source occurs within the second audio stream; and generating a second updated audio stream that enhances the third audio source and the fourth audio source extracted from the second audio stream and diminishes the first audio source and the second audio source from the second audio stream. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the first audio source, the second audio source, and the third audio source are extracted from the audio stream by decomposing the first audio stream and the second audio stream into a first audio portion in which the first audio source is speaking, a second audio portion in which the second audio source is speaking, and a third audio portion in which the third audio source is speaking. ### Claim 21 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the method further comprises transmitting the updated audio stream to the media device while inhibiting the transmission of the updated audio stream to a second media device.