# US 11,445,324 B2 · Audio rendering method and apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,324 B2 - Document ID: US11445324B2 - Status: Granted - Kind: B2 - Application number: 17/240,655 - Filed: 2021-04-26 - Published: 2021-08-12 - Granted: 2022-09-13 - Earliest claimed: 2018-10-26 - Assignees: Huawei Technologies Co., Ltd. - Applicants: Huawei Technologies Co., Ltd. - Inventors: Bin Wang; Zexin Liu; Risheng Xia - Web page: https://patentgrep.com/patent/US11445324B2 ## CPC - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)} - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/21: the extracted parameters being power information - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04S: STEREOPHONIC SYSTEMS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract This application provides an audio rendering method, including: obtaining a to-be-rendered BRIR signal, where an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtaining a direct sound signal based on the to-be-rendered BRIR signal; correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal, to obtain a frequency-domain signal corresponding to the target elevation angle; obtaining a time-domain signal based on the frequency-domain signal of the target elevation angle; and superposing the time-domain signal on a signal that is in the to-be-rendered BRIR signal and that is in a second time period after a first time period, to obtain a BRIR signal of the target elevation angle. ## Claims ### Claim 1 An audio rendering method, comprising: obtaining a to-be-rendered binaural room impulse response (BRIR) signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtaining a direct sound signal based on the to-be-rendered BRIR signal, wherein the direct sound signal corresponds to a first time period in a time period corresponding to the to-be-rendered BRIR signal; correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal, to obtain a frequency-domain signal corresponding to the target elevation angle; obtaining a time-domain signal based on the frequency-domain signal of the target elevation angle; and superposing the time-domain signal on a signal that is in the to-be-rendered BRIR signal and that is in a second time period after the first time period, to obtain a BRIR signal of the target elevation angle. ### Claim 2 (depends on 1) The method according to claim 1, wherein the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal comprises: determining a correction coefficient based on the target elevation angle and a correction function, wherein the correction function comprises a numerical relationship between coefficients of head related transfer function (HRTF) signals corresponding to different elevation angles; and correcting, based on the correction coefficient, the frequency-domain signal corresponding to the direct sound signal, to obtain the corrected frequency-domain signal. ### Claim 3 (depends on 1) The method according to claim 1, wherein the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal comprises: correcting, based on the target elevation angle, at least one piece of information about a peak point or a valley point in a spectral envelope corresponding to the direct sound signal, to obtain at least one piece of corrected information about the peak point or the valley point; determining a target filter based on the at least one piece of corrected information about the peak point or the valley point; and filtering the direct sound signal using the target filter, to obtain the corrected frequency-domain signal. ### Claim 4 (depends on 1) The method according to claim 1, wherein the obtaining a time-domain signal based on the corrected frequency-domain signal comprises: determining an energy adjustment coefficient based on the target elevation angle and an energy adjustment function, wherein the energy adjustment function comprises a numerical relationship between frequency band energy of head related transfer function (HRTF) signals corresponding to different elevation angles; adjusting the corrected frequency-domain signal based on the energy adjustment coefficient to obtain an adjusted frequency-domain signal; and performing frequency-time conversion on the adjusted frequency-domain signal to obtain the time-domain signal. ### Claim 5 (depends on 1) The method according to claim 1, wherein the obtaining a direct sound signal based on the to-be-rendered BRIR signal comprises: extracting a signal in the first time period from the to-be-rendered BRIR signal, and processing the signal in the first time period using a Hanning window, to obtain the direct sound signal. ### Claim 6 (depends on 5) The method according to claim 5, wherein the obtaining a time-domain signal based on the corrected frequency-domain signal comprises: superposing a spectrum of the corrected frequency-domain signal on a spectrum detail, wherein the spectrum detail is a difference between a spectrum of the signal in the first time period and a spectrum of the direct sound signal; and performing frequency-time conversion on a signal corresponding to a spectrum obtained through superposition, to obtain the time-domain signal. ### Claim 7 (depends on 5) The method according to claim 5, the obtaining a time-domain signal based on the corrected frequency-domain signal comprises: superposing a spectrum of the corrected frequency-domain signal on a spectrum detail, wherein the spectrum detail is a difference between a spectrum of the signal in the first time period and a spectrum of the direct sound signal; determining an energy adjustment coefficient based on the target elevation angle and an energy adjustment function, wherein the energy adjustment function comprises a numerical relationship between frequency band energy of the HRTF signals corresponding to different elevation angles; adjusting, based on the energy adjustment coefficient, a signal corresponding to a spectrum obtained through superposition, to obtain an adjusted frequency-domain signal; and performing frequency-time conversion on the adjusted frequency-domain signal to obtain the time-domain signal. ### Claim 8 An audio rendering method, comprising: obtaining a to-be-rendered binaural room impulse response (BRIR) signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; correcting, based on a target elevation angle, a frequency-domain signal corresponding to the to-be-rendered BRIR signal; and performing frequency-time conversion on a corrected frequency-domain signal to obtain a BRIR signal of the target elevation angle. ### Claim 9 (depends on 8) The method according to claim 8, wherein the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the to-be-rendered BRIR signal comprises: determining a correction coefficient based on the target elevation angle and a correction function, wherein the correction function comprises a numerical relationship between spectrums of head related transfer function (HRTF) signals corresponding to different elevation angles; and processing, using the correction coefficient, the frequency-domain signal corresponding to the to-be-rendered BRIR signal, to obtain the corrected frequency-domain signal. ### Claim 10 An audio rendering method, comprising: obtaining a to-be-rendered binaural room impulse response BRIR signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtaining a head related transfer function (HRTF) spectrum corresponding to a target elevation angle; and correcting the to-be-rendered BRIR signal based on the HRTF spectrum corresponding to the target elevation angle, to obtain a BRIR signal of the target elevation angle. ### Claim 11 An audio rendering apparatus, comprising: a memory storing instructions; and a processor, wherein execution of the instructions by the processor cause the apparatus to: obtain a to-be-rendered binaural room impulse response BRIR signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtain a direct sound signal based on the to-be-rendered BRIR signal, wherein the direct sound signal corresponds to a first time period in a time period corresponding to the to-be-rendered BRIR signal; correct, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal, to obtain a frequency-domain signal corresponding to the target elevation angle; obtain a time-domain signal based on the frequency-domain signal of the target elevation angle; and superpose the time-domain signal on a signal that is in the to-be-rendered BRIR signal and that is in a second time period after the first time period, to obtain a BRIR signal of the target elevation angle. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein execution of the instructions by the processor further cause the apparatus to: determine a correction coefficient based on the target elevation angle and a correction function, wherein the correction function comprises a numerical relationship between coefficients of head related transfer function (HRTF) signals corresponding to different elevation angles; and correct, based on the correction coefficient, the frequency-domain signal corresponding to the direct sound signal, to obtain the corrected frequency-domain signal. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein execution of the instructions by the processor further cause the apparatus to: correct, based on the target elevation angle, at least one piece of information about a peak point or a valley point in a spectral envelope corresponding to the direct sound signal, to obtain at least one piece of corrected information about the peak point or the valley point; determine a target filter based on the at least one piece of corrected information about the peak point or the valley point; and filter the direct sound signal using the target filter, to obtain the corrected frequency-domain signal. ### Claim 14 (depends on 11) The apparatus according to claim 11, wherein execution of the instructions by the processor further cause the apparatus to: determine an energy adjustment coefficient based on the target elevation angle and an energy adjustment function, wherein the energy adjustment function comprises a numerical relationship between frequency band energy of head related transfer function (HRTF) signals corresponding to different elevation angles; and adjust the corrected frequency-domain signal based on the energy adjustment coefficient to obtain an adjusted frequency-domain signal, and perform frequency-time conversion on the adjusted frequency-domain signal to obtain the time-domain signal. ### Claim 15 (depends on 11) The apparatus according to claim 11, wherein execution of the instructions by the processor further cause the apparatus to: extract a signal in the first time period from the to-be-rendered BRIR signal; and process the signal in the first time period using a Hanning window, to obtain the direct sound signal. ### Claim 16 (depends on 15) The apparatus according to claim 15, wherein execution of the instructions by the processor further cause the apparatus to: superpose a spectrum of the corrected frequency-domain signal on a spectrum detail, wherein the spectrum detail is a difference between a spectrum of the signal in the first time period and a spectrum of the direct sound signal; and perform frequency-time conversion on a signal corresponding to a spectrum obtained through superposition, to obtain the time-domain signal. ### Claim 17 An audio rendering apparatus, comprising: a memory storing instructions; and a processor, wherein execution of the instructions by the processor cause the apparatus to: obtain a to-be-rendered binaural room impulse response BRIR signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; correct, based on a target elevation angle, a frequency-domain signal corresponding to the to-be-rendered BRIR signal; and perform frequency-time conversion on a corrected frequency-domain signal to obtain a BRIR signal of the target elevation angle. ### Claim 18 (depends on 17) The apparatus according to claim 17, wherein execution of the instructions by the processor further cause the apparatus to: determine a correction coefficient based on the target elevation angle and a correction function, wherein the correction function comprises a numerical relationship between coefficients of head related transfer function (HRTF) signals corresponding to different elevation angles; and process, using the correction coefficient, the frequency-domain signal corresponding to the to-be-rendered BRIR signal, to obtain the corrected frequency-domain signal. ### Claim 19 An audio rendering apparatus, comprising: a memory storing instructions; and a processor, wherein execution of the instructions by the processor cause the apparatus to: obtain a to-be-rendered binaural room impulse response BRIR signal, wherein an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees, and obtain a head related transfer function (HRTF) spectrum corresponding to a target elevation angle; and correct the to-be-rendered BRIR signal based on the HRTF spectrum corresponding to the target elevation angle, to obtain a BRIR signal of the target elevation angle. ### Claim 20 (depends on 1) A non-transitory computer storage medium, comprising instructions, wherein when the instructions are run on a computer, the computer is enabled to perform the method according to claim 1.