# US 10,770,080 B2 · Audio decoder, audio encoder, method for providing at least four audio channel signals on the basis of an encoded representation, method for providing an encoded representation on the basis of at least four audio channel signals and computer program using a bandwidth extension > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,770,080 B2 - Document ID: US10770080B2 - Status: Granted - Kind: B2 - Application number: 16/209,008 - Filed: 2018-12-04 - Published: 2019-12-12 - Granted: 2020-09-08 - Earliest claimed: 2013-07-22 - Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Inventors: Sascha Dick; Christian Ertel; Christian Helmrich; Johannes Hilpert; Andreas Hoelzer; Achim Kuntz - Web page: https://patentgrep.com/patent/US10770080B2 ## CPC - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - G10L 19/0017: {Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error (G10L19/24 takes precedence)} - G10L 21/038: using band spreading techniques - H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved - H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence) - H04S 2420/03: Application of parametric coding in stereophonic audio systems - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04S: STEREOPHONIC SYSTEMS ## Abstract An audio decoder for providing at least four bandwidth-extended channel signals on the basis of an encoded representation provides first and second downmix signals on the basis of a jointly encoded representation of the first and second downmix signals using a multi-channel decoding and provides at least first and second audio channel signals on the basis of the first downmix signal using a multi-channel decoding, and provides at least third and fourth audio channel signals on the basis of the second downmix signal using a multi-channel decoding. It performs a multi-channel bandwidth extension on the basis of the first and third audio channel signals, to obtain first and third bandwidth-extended channel signals, and performs a multi-channel bandwidth extension on the basis of the second and fourth audio channel signals, to obtain second and fourth bandwidth extended channel signals. An audio encoder uses a related concept. ## Claims ### Claim 1 An audio decoder for providing at least four bandwidth-extended channel signals on the basis of an encoded representation, comprising: a multi-channel decoder configured to provide a first downmix signal and a second downmix signal on the basis of a jointly encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding; wherein the audio decoder is configured to provide at least a first audio channel signal and a second audio channel signal on the basis of the first downmix signal using a multi-channel decoding; wherein the audio decoder is configured to provide at least a third audio channel signal and a fourth audio channel signal on the basis of the second downmix signal using a multi-channel decoding; a first multi-channel bandwidth extension configured to perform a multi-channel bandwidth extension on the basis of the first audio channel signal and the third audio channel signal, to acquire a first bandwidth-extended channel signal and a third bandwidth-extended channel signal; and a second multi-channel bandwidth extension configured to perform a multi-channel bandwidth extension on the basis of the second audio channel signal and the fourth audio channel signal, to acquire a second bandwidth extended channel signal and a fourth bandwidth extended channel signal. ### Claim 2 (depends on 1) The audio decoder according to claim 1, wherein the first downmix signal and the second downmix signal are associated with different horizontal positions or azimuth positions of an audio scene. ### Claim 3 (depends on 1) The audio decoder according to claim 1, wherein the first downmix signal is associated with a left side of an audio scene, and wherein the second downmix signal is associated with a right side of the audio scene. ### Claim 4 (depends on 1) The audio decoder according to claim 1, wherein the first audio channel signal and the second audio channel signal are associated with vertically neighboring positions of an audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with vertically neighboring positions of the audio scene. ### Claim 5 (depends on 1) The audio decoder according to claim 1, wherein the first audio channel signal and the third audio channel signal are associated with a first common horizontal plane or a first common elevation of an audio scene but different horizontal positions or azimuth positions of the audio scene, wherein the second audio channel signal and the fourth audio channel signal are associated with a second common horizontal plane or a second common elevation of the audio scene but different horizontal positions or azimuth positions of the audio scene, wherein the first common horizontal plane or the first common elevation is different from the second common horizontal plane or the second common elevation. ### Claim 6 (depends on 5) The audio decoder according to claim 5, wherein the first audio channel signal and the second audio channel signal are associated with a first common vertical plane or a first common azimuth position of the audio scene but different vertical positions or elevations of the audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with a second common vertical plane or a second common azimuth position of the audio scene but different vertical positions or elevations of the audio scene, wherein the first common vertical plane or first azimuth position is different from the second common vertical plane or second azimuth position. ### Claim 7 (depends on 1) The audio decoder according to claim 1, wherein the first audio channel signal and the second audio channel signal are associated with a left side of an audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with a right side of the audio scene. ### Claim 8 (depends on 1) The audio decoder according to claim 1, wherein the first audio channel signal and the third audio channel signal are associated with a lower portion of an audio scene, and wherein the second audio channel signal and the fourth audio channel signal are associated with an upper portion of the audio scene. ### Claim 9 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to perform a horizontal splitting when providing the first downmix signal and the second downmix signal on the basis of the jointly encoded representation of the first downmix signal and the second downmix signal using the multi-channel decoding. ### Claim 10 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to perform a vertical splitting when providing at least the first audio channel signal and the second audio channel signal on the basis of the first downmix signal using the multi-channel decoding; and wherein the audio decoder is configured to perform a vertical splitting when providing at least the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal using the multi-channel decoding. ### Claim 11 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to perform a stereo bandwidth extension on the basis of the first audio channel signal and the third audio channel signal, to acquire the first bandwidth-extended channel signal and the third bandwidth-extended channel signal, wherein the first audio channel signal and the third audio channel signal represent a first left/right channel pair; and wherein the audio decoder is configured to perform a stereo bandwidth extension on the basis of the second audio channel signal and the fourth audio channel signal, to acquire the second bandwidth extended channel signal and the fourth bandwidth extended channel signal, wherein the second audio channel signal and the fourth audio channel signal represent a second left/right channel pair. ### Claim 12 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly encoded representation of the first downmix signal and the second downmix signal using a prediction-based multi-channel decoding. ### Claim 13 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly encoded representation of the first downmix signal and the second downmix signal using a residual-signal-assisted multi-channel decoding. ### Claim 14 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to provide at least the first audio channel signal and the second audio channel signal on the basis of the first downmix signal using a parameter-based multi-channel decoding; wherein the audio decoder is configured to provide at least the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal using a parameter-based multi-channel decoding. ### Claim 15 (depends on 14) The audio decoder according to claim 14, wherein the parameter-based multi-channel decoding is configured to evaluate one or more parameters describing a desired correlation between two channels and/or level differences between two channels in order to provide the two or more audio channel signals on the basis of a respective downmix signal. ### Claim 16 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to provide at least the first audio channel signal and the second audio channel signal on the basis of the first downmix signal using a residual-signal-assisted multi-channel decoding; and wherein the audio decoder is configured to provide at least the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal using a residual-signal-assisted multi-channel decoding. ### Claim 17 (depends on 1) The audio decoder according to claim 1, wherein the audio decoder is configured to provide a first residual signal, which is used to provide at least the first audio channel signal and the second audio channel signal, and a second residual signal, which is used to provide at least the third audio channel signal and the fourth audio channel signal, on the basis of a jointly encoded representation of the first residual signal and the second residual signal using a multi-channel decoding. ### Claim 18 (depends on 17) The audio decoder according to claim 17, wherein the first residual signal and the second residual signal are associated with different horizontal positions or azimuth positions of an audio scene. ### Claim 19 (depends on 17) The audio decoder according to claim 17, wherein the first residual signal is associated with a left side of an audio scene, and wherein the second residual signal is associated with a right side of the audio scene. ### Claim 20 An audio encoder for providing an encoded representation on the basis of at least four audio channel signals, comprising: a first bandwidth extension parameter extraction configured to acquire a first set of common bandwidth extension parameters on the basis of a first audio channel signal and a third audio channel signal; a second bandwidth extension parameter extraction configured to acquire a second set of common bandwidth extension parameters on the basis of a second audio channel signal and a fourth audio channel signal; a first multi-channel encoding configured to jointly encode at least the first audio channel signal and the second audio channel signal using a multi-channel encoding, to acquire a first downmix signal; a second multi-channel encoding is configured to jointly encode at least the third audio channel signal and the fourth audio channel signal using a multi-channel encoding, to acquire a second downmix signal; and wherein the audio encoder is configured to jointly encode the first downmix signal and the second downmix signal using a multi-channel encoding, to acquire an encoded representation of the first downmix signal and the second downmix signal. ### Claim 21 (depends on 20) The audio encoder according to claim 20, wherein the first downmix signal and the second downmix signal are associated with different horizontal positions or azimuth positions of an audio scene. ### Claim 22 (depends on 20) The audio encoder according to claim 20, wherein the first downmix signal is associated with a left side of an audio scene, and wherein the second downmix signal is associated with a right side of the audio scene. ### Claim 23 (depends on 20) The audio encoder according to claim 20, wherein the first audio channel signal and the second audio channel signal are associated with vertically neighboring positions of an audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with vertically neighboring positions of the audio scene. ### Claim 24 (depends on 20) The audio encoder according to claim 20, wherein the first audio channel signal and the third audio channel signal are associated with a first common horizontal plane or a first elevation of an audio scene but different horizontal positions or azimuth positions of the audio scene, wherein the second audio channel signal and the fourth audio channel signal are associated with a second common horizontal plane or a second elevation of the audio scene but different horizontal positions or azimuth positions of the audio scene, wherein the first common horizontal plane or the first elevation is different from the second common horizontal plane or the second elevation. ### Claim 25 (depends on 24) The audio encoder according to claim 24, wherein the first audio channel signal and the second audio channel signal are associated with a first common vertical plane or a first azimuth position of the audio scene but different vertical positions or elevations of the audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with a second common vertical plane or a second azimuth positions of the audio scene but different vertical positions or elevations of the audio scene, wherein the first common vertical plane or the first azimuth position is different from the second common vertical plane or the second azimuth position. ### Claim 26 (depends on 20) The audio encoder according to claim 20, wherein the first audio channel signal and the second audio channel signal are associated with a left side of an audio scene, and wherein the third audio channel signal and the fourth audio channel signal are associated with a right side of the audio scene. ### Claim 27 (depends on 20) The audio encoder according to claim 20, wherein the first audio channel signal and the third audio channel signal are associated with a lower portion of an audio scene, and wherein the second audio channel signal and the fourth audio channel signal are associated with an upper portion of the audio scene. ### Claim 28 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to perform a horizontal combining when providing the encoded representation of the downmix signals on the basis of the first downmix signal and the second downmix signal using the multi-channel encoding. ### Claim 29 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to perform a vertical combining when providing the first downmix signal on the basis of the first audio channel signal and the second audio channel signal using the multi-channel encoding; and wherein the audio encoder is configured to perform a vertical combining when providing the second downmix signal on the basis of the third audio channel signal and the fourth audio channel signal using the multi-channel encoding. ### Claim 30 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to provide the jointly encoded representation of the first downmix signal and the second downmix signal on the basis of the first downmix signal and the second downmix signal using a prediction-based multi-channel encoding. ### Claim 31 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to provide the the jointly encoded representation of the first downmix signal and the second downmix signal on the basis of the first downmix signal and the second downmix signal using a residual-signal-assisted multi-channel encoding. ### Claim 32 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to provide the first downmix signal on the basis of the first audio channel signal and the second audio channel signal using a parameter-based multi-channel encoding; and wherein the audio encoder is configured to provide the second downmix signal on the basis of the third audio channel signal and the fourth audio channel signal using a parameter-based multi-channel encoding. ### Claim 33 (depends on 32) The audio encoder according to claim 32, wherein the parameter-based multi-channel encoding is configured to provide one or more parameters describing a desired correlation between two channels and/or level differences between two channels. ### Claim 34 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to provide the first downmix signal on the basis of the first audio channel signal and the second audio channel signal using a residual-signal-assisted multi-channel encoding; and wherein the audio encoder is configured to provide the second downmix signal on the basis of the third audio channel signal and the fourth audio channel signal using a residual-signal-assisted multi-channel encoding. ### Claim 35 (depends on 20) The audio encoder according to claim 20, wherein the audio encoder is configured to provide a jointly encoded representation of a first residual signal, which is acquired when jointly encoding at least the first audio channel signal and the second audio channel signal, and of a second residual, which is acquired when jointly encoding at least the third audio channel signal and the fourth audio channel signal, using a multi-channel encoding. ### Claim 36 (depends on 35) The audio encoder according to claim 35, wherein the first residual signal and the second residual signal are associated with different horizontal positions or azimuth positions of an audio scene. ### Claim 37 (depends on 35) The audio encoder according to claim 35, wherein the first residual signal is associated with a left side of an audio scene, and wherein the second residual signal is associated with a right side of the audio scene. ### Claim 38 A method for providing at least four audio channel signals on the basis of an encoded representation, wherein the method comprises: providing a first downmix signal and a second downmix signal on the basis of a jointly encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding; providing at least a first audio channel signal and a second audio channel signal on the basis of the first downmix signal using a multi-channel decoding; providing at least a third audio channel signal and a fourth audio channel signal on the basis of the second downmix signal using a multi-channel decoding; performing a multi-channel bandwidth extension on the basis of the first audio channel signal and the third audio channel signal, to acquire a first bandwidth-extended channel signal and a third bandwidth-extended channel signal; and performing a multi-channel bandwidth extension on the basis of the second audio channel signal and the fourth audio channel signal, to acquire the second bandwidth extended channel signal and the fourth bandwidth extended channel signal. ### Claim 39 A method for providing an encoded representation on the basis of at least four audio channel signals, the method comprising: acquiring a first set of common bandwidth extension parameters on the basis of a first audio channel signal and a third audio channel signal; acquiring a second set of common bandwidth extension parameters on the basis of a second audio channel signal and a fourth audio channel signal; jointly encoding at least the first audio channel signal and the second audio channel signal using a multi-channel encoding, to acquire a first downmix signal; jointly encoding at least the third audio channel signal and the fourth audio channel signal using a multi-channel encoding, to acquire a second downmix signal; and jointly encoding the first downmix signal and the second downmix signal using a multi-channel encoding, to acquire an encoded representation of the first downmix signal and the second downmix signal. ### Claim 40 A non-transitory digital storage medium having a computer program stored thereon to perform the method for providing at least four audio channel signals on the basis of an encoded representation, wherein the method comprises: providing a first downmix signal and a second downmix signal on the basis of a jointly encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding; providing at least a first audio channel signal and a second audio channel signal on the basis of the first downmix signal using a multi-channel decoding; providing at least a third audio channel signal and a fourth audio channel signal on the basis of the second downmix signal using a multi-channel decoding; performing a multi-channel bandwidth extension on the basis of the first audio channel signal and the third audio channel signal, to acquire a first bandwidth-extended channel signal and a third bandwidth-extended channel signal; and performing a multi-channel bandwidth extension on the basis of the second audio channel signal and the fourth audio channel signal, to acquire the second bandwidth extended channel signal and the fourth bandwidth extended channel signal, when said computer program is run by a computer. ### Claim 41 A non-transitory digital storage medium having a computer program stored thereon to perform the method for providing an encoded representation on the basis of at least four audio channel signals, the method comprising: acquiring a first set of common bandwidth extension parameters on the basis of a first audio channel signal and a third audio channel signal; acquiring a second set of common bandwidth extension parameters on the basis of a second audio channel signal and a fourth audio channel signal; jointly encoding at least the first audio channel signal and the second audio channel signal using a multi-channel encoding, to acquire a first downmix signal; jointly encoding at least the third audio channel signal and the fourth audio channel signal using a multi-channel encoding, to acquire a second downmix signal; and jointly encoding the first downmix signal and the second downmix signal using a multi-channel encoding, to acquire an encoded representation of the first downmix signal and the second downmix signal, when said computer program is run by a computer.