# US 12,009,006 B2 · Audio signal processing method, apparatus and device, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,009,006 B2 - Document ID: US12009006B2 - Status: Granted - Kind: B2 - Application number: 17/741,285 - Filed: 2022-05-10 - Published: 2022-08-25 - Granted: 2024-06-11 - Earliest claimed: 2020-07-17 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Rilin Chen; Kaiyu Jiang; Weiwei Li - Web page: https://patentgrep.com/patent/US12009006B2 ## CPC - G10L 21/0364: for improving intelligibility - G10L 21/003: Changing voice quality, e.g. pitch or formants - H04R 1/406: {microphones} - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - G10L 21/0208: Noise filtering - G10L 2021/02166: {Microphone arrays; Beamforming} - H04R 25/407: {Circuits for combining signals of a plurality of transducers} - H04R 2201/40: Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups - H04R 2430/20: Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract An electronic device obtains audio signals collected by different microphones in a microphone array. The device filters the audio signals using a first filter to obtain a first target beam. The first filter is configured to suppress an interference speech in the audio signals and enhance a target speech in the audio signals. The device filters the audio signals using a second filter to obtain a first interference beam. The second filter is configured to suppress the target speech and enhance the interference speech. The device a second interference beam of the first interference beam using a third filter. The device determines a difference between the first target beam and the second interference beam as a first audio processing output. The device adaptively updates at least one of the second filter and the third filter, and updates the first filter according to the updated second filter and/or third filter. ## Claims ### Claim 1 An audio signal processing method performed by an electronic device, the method comprising: obtaining audio signals collected by different microphones in a microphone array; filtering the audio signals using a first filter to obtain a first target beam, wherein the first filter is configured to suppress an interference speech in the audio signals and enhance a target speech in the audio signals; filtering the audio signals using a second filter to obtain a first interference beam, wherein the second filter is configured to suppress the target speech and enhance the interference speech; obtaining a second interference beam of the first interference beam using a third filter, wherein the third filter is configured to perform a weighted adjustment on the first interference beam; determining a difference between the first target beam and the second interference beam as a first audio processing output; and adaptively updating at least one of the second filter and the third filter; and updating the first filter according to the updated second filter and/or the third filter. ### Claim 2 (depends on 1) The method according to claim 1, wherein the first filter corresponds to a first weight matrix, the second filter corresponds to a second weight matrix, and the third filter corresponds to a third weight matrix; and updating the first filter according to the updated second filter and/or the third filter comprises: calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix, and updating the first filter according to the first weight matrix. ### Claim 3 (depends on 2) The method according to claim 2, wherein calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix comprises: determining, after the updating, a product of the second weight matrix and the third weight matrix as a target matrix; and determining a difference between an identity matrix and the target matrix as the first weight matrix. ### Claim 4 (depends on 1) The method according to claim 1, wherein adaptively updating at least one of the second filter and the third filter comprises at least one of: updating the second filter according to the first target beam, and updating the third filter according to the first audio processing output; updating the second filter and the third filter according to the first audio processing output; updating the second filter according to the first target beam; updating the second filter according to the first audio processing output; or updating the third filter according to the first audio processing output. ### Claim 5 (depends on 1) The method according to claim 1, wherein filtering, by the first filter, the audio signals to obtain the first target beam comprises: first filtering the audio signals using a pre-filter to obtain a target pre-beam, the pre-filter is a filter calculated using training data and is configured to suppress the interference speech and enhance the target speech; and second filtering the target pre-beam using the pre-filter to obtain the first target beam. ### Claim 6 (depends on 5) The method according to claim 5, further comprising: acquiring the training data collected by the microphone array in an application environment, the application environment is a spatial range where the microphone array is placed and used, and the training data comprising sample audio signals collected by different microphones in the microphone array; and obtaining the pre-filter by calculating the training data according to a linearly constrained minimum-variance (LCMV) criterion. ### Claim 7 (depends on 1) The method according to claim 1, wherein the microphone array comprises n target directions, wherein n is a positive integer greater than one, each of the target directions corresponding to a respective filter bank that is configured to process the audio signals by performing the steps of obtaining the audio signals, filtering the audio signals using the first filter, filtering the audio signals using the second filter, obtaining the second interference beam, determining, adaptively updating, and updating. ### Claim 8 (depends on 7) The method according to claim 7, further comprising: filtering, for the audio signals corresponding to the n target directions, the audio signals using the corresponding filter banks respectively to obtain n first audio processing outputs corresponding to the n target directions; and filtering an ith first audio processing output according to the n−1 first audio processing outputs except the ith first audio processing output to obtain an ith second audio processing output corresponding to an ith target direction, i being a positive integer greater than 0 and less than n; and repeating an operation to obtain second audio processing outputs corresponding to the n target directions respectively. ### Claim 9 (depends on 8) The method according to claim 8, wherein filtering the ith first audio processing output according to the n−1 first audio processing outputs except the ith first audio processing output to obtain the ith second audio processing output corresponding to the ith target direction comprises: determining the n−1 first audio processing outputs except the ith first audio processing output as an ith interference group; filtering, by an ith fourth filter corresponding to the ith target direction, the ith interference group to obtain an ith third interference beam, the fourth filter being configured to perform weighted adjustment on the interference group; determining a difference between the ith first audio processing output and the ith third interference beam as the ith second audio processing output; and updating the ith fourth filter adaptively according to the ith second audio processing output. ### Claim 10 (depends on 7) The method according to claim 7, wherein the respective filter bank is an ith filter bank comprising a pre-filter, obtained by training with training data collected by the microphone array in a ith target direction. ### Claim 11 An electronic device, comprising: one or more processors; and memory storing one or more programs, the one or more programs comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: obtaining audio signals collected by different microphones in a microphone array; filtering the audio signals using a first filter to obtain a first target beam, wherein the first filter is configured to suppress an interference speech in the audio signals and enhance a target speech in the audio signals; filtering the audio signals using a second filter to obtain a first interference beam, wherein the second filter is configured to suppress the target speech and enhance the interference speech; obtaining a second interference beam of the first interference beam using a third filter, wherein the third filter is configured to perform a weighted adjustment on the first interference beam; determining a difference between the first target beam and the second interference beam as a first audio processing output; adaptively updating at least one of the second filter and the third filter; and updating the first filter according to the updated second filter and/or the third filter. ### Claim 12 (depends on 11) The electronic device according to claim 11, wherein the first filter corresponds to a first weight matrix, the second filter corresponds to a second weight matrix, and the third filter corresponds to a third weight matrix; and updating the first filter according to the updated second filter and/or the third filter comprises: calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix, and updating the first filter according to the first weight matrix. ### Claim 13 (depends on 12) The electronic device according to claim 12, wherein calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix comprises: determining, after the updating, a product of the second weight matrix and the third weight matrix as a target matrix; and determining a difference between an identity matrix and the target matrix as the first weight matrix. ### Claim 14 (depends on 11) The electronic device according to claim 11, wherein adaptively updating the at least one of the second filter and the third filter adaptively comprises at least one of: updating the second filter according to the first target beam, and updating the third filter according to the first audio processing output; updating the second filter and the third filter according to the first audio processing output; updating the second filter according to the first target beam; updating the second filter according to the first audio processing output; or updating the third filter according to the first audio processing output. ### Claim 15 (depends on 11) The electronic device according to claim 11, wherein filtering, by the first filter, the audio signals to obtain the first target beam comprises: first filtering the audio signals using a pre-filter to obtain a target pre-beam, the pre-filter is a filter calculated using training data and is configured to suppress the interference speech and enhance the target speech; and second filtering the target pre-beam using the pre-filter to obtain the first target beam. ### Claim 16 (depends on 15) The electronic device according to claim 15, the operations further comprising: acquiring the training data collected by the microphone array in an application environment, the application environment being a spatial range where the microphone array is placed and used, and the training data comprising sample audio signals collected by different microphones in the microphone array; and obtaining the pre-filter by calculating the training data according to a linearly constrained minimum-variance (LCMV) criterion. ### Claim 17 (depends on 11) The electronic device according to claim 11, wherein the microphone array comprises n target directions, wherein n is a positive integer greater than one, each of the target directions corresponding to a respective filter bank that is configured to process the audio signals by performing the steps of obtaining the audio signals, filtering the audio signals using the first filter, filtering the audio signals using the second filter, obtaining the second interference beam, determining, adaptively updating, and updating. ### Claim 18 A non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by one or more processors of a computing device, cause the one or more processors to perform operations comprising: obtaining audio signals collected by different microphones in a microphone array; filtering the audio signals using a first filter to obtain a first target beam, wherein the first filter is configured to suppress an interference speech in the audio signals and enhance a target speech in the audio signals; filtering the audio signals using a second filter to obtain a first interference beam, wherein the second filter is configured to suppress the target speech and enhance the interference speech; obtaining a second interference beam of the first interference beam using a third filter, wherein the third filter is configured to perform a weighted adjustment on the first interference beam; determining a difference between the first target beam and the second interference beam as a first audio processing output; and adaptively updating at least one of the second filter and the third filter; and updating the first filter according to the updated second filter and/or the third filter. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium according to claim 18, wherein the first filter corresponds to a first weight matrix, the second filter corresponds to a second weight matrix, and the third filter corresponds to a third weight matrix; and updating the first filter according to the updated second filter and/or the third filter comprises: calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix, and updating the first filter according to the first weight matrix. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium according to claim 19, wherein the calculating, after the updating, the first weight matrix according to the second weight matrix and the third weight matrix comprises: determining, after the updating, a product of the second weight matrix and the third weight matrix as a target matrix; and determining a difference between an identity matrix and the target matrix as the first weight matrix.