# US 10,481,859 B2 · Audio synchronization and delay estimation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,481,859 B2 - Document ID: US10481859B2 - Status: Granted - Kind: B2 - Application number: 15/834,752 - Filed: 2017-12-07 - Published: 2019-06-13 - Granted: 2019-11-19 - Earliest claimed: 2017-12-07 - Assignees: Powerchord Group Limited - Applicants: Powerchord Group Limited - Inventors: Graham Tull; Troels Bastholm; Mads Berre Eriksen; Kurt Heick Rasmussen; Jens Christian Lindof - Web page: https://patentgrep.com/patent/US10481859B2 ## CPC - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - H04R 3/04: for correcting frequency response - H04R 3/12: for distributing signals to two or more loudspeakers {(specially adapted for hearing aids H04R25/407)} - H04R 5/02: Spatial or constructional arrangements of loudspeakers - H04R 27/00: Public address systems (circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; {monitoring or testing arrangements for public address systems H04R29/007}; amplifiers H03F) - H04S 7/301: {Automatic calibration of stereophonic sound system, e.g. with test microphone} - H04R 2227/003: Digital PA systems using, e.g. LAN or internet - H04R 2420/07: Applications of wireless loudspeakers or wireless microphones - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H04S: STEREOPHONIC SYSTEMS ## Abstract Systems and techniques are provided for audio synchronization and delay estimation. Audio metadata including a first discrete Fourier transform representation may be received. An audio signal may be pre-processed. A second discrete Fourier transform representation may be generated from the pre-processed audio signal. A correlation result in a discrete Fourier transform representation may be generated based on an element-wise multiplication of the first and second discrete Fourier transform representations. An inverse Fourier transform may be performed on the correlation result in a discrete Fourier transform representation to generate a correlated signal including samples that may have a position and a value. A relative delay value may be determined based on the position of a sample having a value with the greatest magnitude. Playback of a second audio signal may be adjusted based on a current delay value adjusted based on the relative delay value. ## Claims ### Claim 1 A method comprising: pre-processing an audio signal at a transmitter to generate a transmitter pre-processed audio signal comprising samples, each sample comprising a value and having a position in the transmitter pre-processed audio signal; reversing, by the transmitter, the positions of the samples of the transmitter pre-processed audio signal to generate a reversed audio signal; generating, by the transmitter, a transmitter discrete Fourier transform representation from the reversed audio signal; transmitting, by the transmitter, at least a section of the transmitter discrete Fourier transform representation to a receiver as audio metadata; receiving, by the receiver, the audio metadata comprising the at least a section of the transmitter discrete Fourier transform representation; pre-processing, by the receiver, a second audio signal; generating, by the receiver, a receiver discrete Fourier transform representation from the pre-processed second audio signal; generating, by the receiver, a correlation result in a discrete Fourier transform representation based on an element-wise multiplication of the at least a section of the transmitter discrete Fourier transform representation and the receiver discrete Fourier transform representation; performing, by the receiver, an inverse Fourier transform on the correlation result in a discrete Fourier transform representation to generate a correlated signal comprising one or more samples, each sample of the correlated signal having a position in the correlated signal and comprising a value; determining, by the receiver, a relative delay value based on the position in the correlated signal of a sample comprising a value with the greatest magnitude of the values of the samples of the correlated signal; and adjusting, by the receiver, playback of a third audio signal based on a current delay value adjusted based on the relative delay value by adding the relative delay value to a histogram, determining an adjustment to the current delay value based on the histogram, adjusting the current delay value based on the determined adjustment, and adjusting the playback of the third audio signal so that value of a total delay in playback of the third audio signal moves towards the current delay value if the total delay is not equal to the current delay value. ### Claim 2 (depends on 1) The method of claim 1, wherein pre-processing the audio signal at the transmitter comprises one or more of sampling the audio signal, and filtering the audio signal, down sampling the audio signal. ### Claim 3 (depends on 1) The method of claim 1, wherein the audio signal is an analog audio signal and wherein the pre-processed audio signal is a digital audio signal. ### Claim 4 (depends on 1) The method of claim 1, further comprising generating the second audio signal at the receiver based on sounds arriving at the location of a microphone of the receiver. ### Claim 5 (depends on 4) The method of claim 4, wherein the sounds arriving at the location of the microphone of the receiver are generated by one or more speakers based on source audio signals that are used to generate the audio signal that is pre-processed by the transmitter. ### Claim 6 (depends on 1) The method of claim 1, wherein the third audio signal is transmitted from the transmitter to the receiver, and wherein the third audio signal is generated based on source audio signals that are used to generate the audio signal that is pre-processed by the transmitter. ### Claim 7 (depends on 1) The method of claim 1, wherein the audio metadata further comprises a root mean square value determined by the transmitter based on the audio signal, and further comprising: weighting the relative delay value in the histogram based on one or both of the root mean square value and a recency of the relative delay value compared to one or more relative delay values of the histogram. ### Claim 8 (depends on 1) The method of claim 1, further comprising phase transform weighting the correlation result in a discrete Fourier transform representation before performing the inverse Fourier transform. ### Claim 9 (depends on 1) The method of claim 1, further comprising one or more of normalizing, by the transmitter, the transmitter discrete Fourier transform representation and normalizing, by the receiver, the receiver discrete Fourier transform representation. ### Claim 10 (depends on 1) The method of claim 1, further comprising windowing, by the transmitter, the pre-processed audio signal before reversing the positions of the samples of the pre-processed audio signal. ### Claim 11 (depends on 1) The method of claim 1, wherein pre-processing the audio signal is performed continuously by the transmitter, and wherein reversing the positions of the samples of the transmitter pre-processed, generating a transmitter discrete Fourier transform representation, and transmitting the discrete Fourier transform representation to a receiver as audio metadata are performed at specified intervals by the transmitter. ### Claim 12 (depends on 1) The method of claim 1, wherein pre-processing the second audio signal is performed continuously by the receiver, and wherein generating a receiver discrete Fourier transform representation, generating a correlation result in a discrete Fourier transform representation, performing an inverse Fourier transform on the correlation result in a discrete Fourier transform representation, and determining a relative delay value based on the position in the correlated signal of a sample comprising a value with the greatest magnitude of the values of the samples of the correlated signal, are performed at specified intervals by the receiver. ### Claim 13 A method comprising: receiving audio metadata comprising a discrete Fourier transform representation; pre-processing an audio signal; generating a second discrete Fourier transform representation from the pre-processed audio signal; generating a correlation result in a discrete Fourier transform representation based on an element-wise multiplication of the discrete Fourier transform representation and the second discrete Fourier transform representation; performing an inverse Fourier transform on the correlation result in a discrete Fourier transform representation to generate a correlated signal comprising one or more samples, each sample having a position in the correlated signal and comprising a value; determining a relative delay value based on the position in the correlated signal of a sample comprising a value with the greatest magnitude of the values of the samples of the correlated signal; and adjusting playback of a second audio signal based on a current delay value adjusted based on the relative delay value by adding the relative delay value to a histogram, determining an adjustment to the current delay value based on the histogram, adjusting the current delay value based on the determined adjustment, and adjusting the playback of the second audio signal so that value of a total delay in playback of the second audio signal moves towards the current delay value if the total delay is not equal to the current delay value. ### Claim 14 (depends on 13) The method of claim 13, wherein pre-processing the audio comprises one or more of sampling the audio signal, filtering the audio signal, and down sampling the audio signal. ### Claim 15 (depends on 13) The method of claim 13, wherein the audio signal is generated by a microphone. ### Claim 16 (depends on 13) The method of claim 13, wherein the second audio signal is received from a transmitter and comprises an audio signal based on source audio signals on which the discrete Fourier transform representation in the audio metadata is based. ### Claim 17 (depends on 13) The method of claim 13, wherein the audio metadata further comprises a root mean square value. ### Claim 18 A method comprising: pre-processing an audio signal to generate a pre-processed audio signal comprising samples, each sample comprising a value and having a position in the pre-processed audio signal; reversing the positions of the samples of the pre-processed audio signal to generate a reversed audio signal; generating a discrete Fourier transform representation from the reversed audio signal; determining a root mean square value based on the audio signal; and transmitting the discrete Fourier transform representation to one or more receivers as audio metadata and transmitting the root mean square value to the one or more receivers with the discrete Fourier transform representation as additional audio metadata. ### Claim 19 (depends on 18) The method of claim 18, further comprising transmitting a second audio signal to the one or more receivers, wherein the second audio signal is based on source audio signals on which the audio signal that is pre-processed is also based. ### Claim 20 (depends on 19) The method of claim 19, wherein transmission of the second audio signal is delayed based on a distance to a nearest speaker. ### Claim 21 A device comprising: a microphone that generates an audio signal; a wireless radio that receives a second audio signal and audio metadata comprising a discrete Fourier transform representation; and one or more hardware devices for processing audio signals that pre-process the audio signal generated by the microphone, generate a second discrete Fourier transform representation from the pre-processed audio signal, generate a correlation result in a discrete Fourier transform representation based on an element-wise multiplication of the discrete Fourier transform representation and the second discrete Fourier transform representation, perform an inverse Fourier transform on the correlation result in a discrete Fourier transform representation to generate a correlated signal comprising one or more samples, each sample having a position in the correlated signal and comprising a value, determine a relative delay value based on the position in the correlated signal of a sample comprising a value with the greatest magnitude of the values of the samples of the correlated signal, and adjust playback of a second audio signal based on a current delay value adjusted based on the relative delay value by adding the relative delay value to a histogram, determining an adjustment to the current delay value based on the histogram, adjusting the current delay value based on the determined adjustment, and adjusting the playback of the second audio signal so that value of a total delay in playback of the second audio signal moves towards the current delay value if the total delay is not equal to the current delay value. ### Claim 22 (depends on 21) The device of claim 21, wherein one of the one or more hardware devices for processing audio is an ADC that converts the audio signal generated by the microphone from an analog audio signal to a digital audio signal. ### Claim 23 (depends on 21) The device of claim 21, wherein one of the one or more hardware devices for processing audio is a processor with a built-in Fourier transform function. ### Claim 24 (depends on 21) The device of claim 21, wherein the one or more hardware devices for processing audio adjust playback of the second audio signal by resampling the second audio signal to add or drop samples. ### Claim 25 (depends on 21) The device of claim 21, further comprising a sound generating device that generates sound based on the second audio signal. ### Claim 26 A device comprising: a transmission device that transmits an audio signal and audio metadata; and a computing device that pre-processes a second audio signal to generate a pre-processed audio signal comprising samples, each sample comprising a value and having a position in the pre-processed audio signal, reverses the positions of the samples of the pre-processed audio signal to generate a reversed audio signal, generates a discrete Fourier transform representation from the reversed audio signal, determines a root mean square value based on the audio signal and causes the transmission device to transmit the discrete Fourier transform representation to one or more receivers as the audio metadata and transmit the root mean square value to the one or more receivers with the discrete Fourier transform representation as additional audio metadata. ### Claim 27 (depends on 26) The device of claim 26, wherein the transmission device is a wireless access point. ### Claim 28 (depends on 26) The device of claim 26, wherein the device further comprises one or more inputs for one or more audio signals, and wherein the one or more audio signals are analog or digital. ### Claim 29 (depends on 28) The device of claim 28, wherein the audio signal and the second audio signal are based on one or more audio signals received at the one or more inputs.