# US 12,114,023 B2 · Cloud byte stream alignment method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,114,023 B2 - Document ID: US12114023B2 - Status: Granted - Kind: B2 - Application number: 17/774,211 - Filed: 2020-12-23 - Published: 2022-11-03 - Granted: 2024-10-08 - Earliest claimed: 2019-12-26 - Assignees: Harman International Industries, Incorporated - Applicants: Harman International Industries, Incorporated - Inventors: Cristian Chis; Timothy Raymond VanGoethem; Jabir Kannathvalappil Moidunny - Web page: https://patentgrep.com/patent/US12114023B2 ## CPC - H04M 9/082: {using echo cancellers (echo cancellers per se H04B3/23)} - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04L 65/75: Media network packet handling - H04L 65/80: Responding to QoS - H04N 21/2387: Stream processing in response to a playback request from an end-user, e.g. for trick-play - H04N 21/2407: {Monitoring of transmitted content, e.g. distribution time, number of downloads (arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14)} - G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) - H04M 9/08: Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method for selecting an audio signal for alignment to compensate for the latency that is introduced by content being sent, such as from an end device, over a network to a cloud based or other computing environment located remote from the end unit. Audio that is processed in the cloud is also being sent back to the end device. Selection may be accomplished using a loop back method, a Time Stamp (TS) method or a Ping method. The Ping method allows incoming and outgoing audio signals to be selected and processed in the cloud. ## Claims ### Claim 1 A method for cloud-based Echo Noise Cancellation Reduction (ECNR) in audio signals originating in an audio system and played back at an end device, the method comprising the steps of: receiving, at a microphone of the audio system, a buffered, sequenced, and time-stamped uplink audio signal; transmitting the uplink audio signal over a network to a cloud based ECNR; receiving, at the ECNR, a downlink audio signal from a content cloud; at the cloud based ECNR, buffering, sequencing and time stamping the uplink and downlink audio signals for a second time with a reference time stamp; identifying from the buffer at the cloud based ECNR, a proper uplink audio signal; selecting the identified proper uplink audio signal; transmitting the proper uplink audio signal from the ECNR to the content cloud; and transmitting the downlink audio signal over the network to be played back at a speaker at the end device. ### Claim 2 (depends on 1) The method of claim 1 wherein the step of identifying and selecting a proper uplink audio signal further comprises the steps of: looping back the downlink audio signal with the uplink audio signal over the network to the ECNR; and selecting, from the buffered and sequenced audio signals going into and coming out of the ECNR, the audio signal that matches the looped back downlink audio signal to be the proper uplink audio signal. ### Claim 3 (depends on 1) The method of claim 1 wherein the step of selecting the proper uplink audio signal further comprises the steps of: combining the uplink audio signal with the time stamp for the downlink audio signal; and selecting, from the buffered and sequenced audio signals going into and coming out of the ECNR, the audio signal that matches the time stamp for the downlink audio signal to be the proper uplink audio signal. ### Claim 4 (depends on 1) The method of claim 1 wherein the step of selecting the proper uplink audio signal to be processed further comprises the steps of: looping a ping between the audio system and the cloud to measure a time delay; continuously aligning the ping with the uplink audio signal; and selecting, from the buffered and sequenced audio signals going into and coming out of the ECNR, the audio signal that matches the time delay of the ping as the proper uplink audio signal. ### Claim 5 (depends on 4) The method of claim 4 wherein the step of receiving at the ECNR a downlink audio signal further comprises the steps of: continuously aligning the ping with the downlink audio signal; processing the downlink audio signal at the ECNR; and transmitting a processed downlink audio signal to the audio system. ### Claim 6 A system for cancelling echo noise in audio signals, the system comprising: an audio system having a microphone and a loudspeaker; buffered, sequenced, and time-stamped uplink audio signals received at the microphone; a cloud-based processor; a communication link between the audio system and the cloud-based processor for transmitting the uplink audio signals between the audio system and the cloud-based processor; the cloud-based processor buffering, sequencing and timestamping the uplink audio signal for a second time with respect to a reference time stamp, the cloud-based processor identifying and selecting, from the buffered, sequenced, and timestamped uplink audio signals, a proper uplink audio signal, the cloud-based processor processes the proper uplink audio signal for echo noise cancellation reduction; and the proper uplink audio signal is transmitted to a content cloud. ### Claim 7 (depends on 6) The system of claim 6 further comprising a downlink audio signal generated at the content cloud that is looped back with the uplink signal, the proper uplink audio signal is selected by detecting an audio signal that matches the looped back downlink audio signal. ### Claim 8 (depends on 6) The system of claim 6 further comprising: a downlink audio signal; a timestamp for the downlink audio signal; and the proper uplink audio signal further comprises the audio signal that matches a combined uplink audio signal with the timestamp for the downlink audio signal. ### Claim 9 (depends on 6) The system of claim 6 further comprising: a downlink audio signal; a ping to measure a time delay, the ping is looped between the audio system and the cloud-based processor and continuously aligned with the uplink audio signals; and the proper uplink audio signal is identified to be the audio signal that matches the time delay of the ping. ### Claim 10 (depends on 9) The system of claim 9 wherein the ping is continuously aligned with the downlink audio signal and the proper uplink audio signal is identified to be the audio signal that matches the time delay of the ping.