# US 2019/0379990 A1 · Compatible multi-channel coding/decoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0379990 A1 - Document ID: US20190379990A1 - Status: Published application - Kind: A1 - Application number: 16/548,905 - Filed: 2019-08-23 - Published: 2019-12-12 - Earliest claimed: 2003-10-02 - Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Applicants: Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. - Inventors: JUERGEN HERRE; JOHANNES HILPERT; STEFAN GEYERSBERGER; ANDREAS HOELZER; CLAUS SPENGER - Web page: https://patentgrep.com/patent/US20190379990A1 ## CPC - H04S 3/02: of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - G10L 19/032: Quantisation or dequantisation of spectral components - H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence) - H04S 2420/03: Application of parametric coding in stereophonic audio systems - H04S: STEREOPHONIC SYSTEMS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract In processing a multi-channel audio signal having at least three original channels, first and second downmix channels derived from the original channels are provided. For a selected original channel of the original channels, channel side information are calculated such that a downmix channel or a combined downmix channel including the first and second downmix channels, when weighted using the channel side information, results in an approximation of the selected original channel. The channel side information and the first and second downmix channels form output data to be transmitted to a low-level decoder, which only decodes the first and second downmix channels, or to a high-level decoder, which provides a full multi-channel audio signal based on the downmix channels and the channel side information. Since the channel side information occupy few bits only and since the decoder does not use dematrixing, an efficient and high quality multi-channel extension for stereo players and enhanced multi-channel players is acquired. ## Claims ### Claim 1 Apparatus for processing a multi-channel audio signal, the multi-channel audio signal comprising at least three original channels, comprising: a provider for providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels; a calculator for calculating channel side information for a selected original channel of the original signals, the calculator being operative to calculate the channel side information such that a downmix channel or a combined downmix channel comprising the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel; and a generator for generating output data, the output data comprising the channel side information. ### Claim 2 (depends on 1) Apparatus in accordance with claim 1, in which the generator is operative to generate the output data such that the output data additionally comprise the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel. ### Claim 3 (depends on 1) Apparatus in accordance with claim 1, in which the calculator is operative to determine the channel side information as parametric data not comprising time domain samples or spectral values. ### Claim 4 (depends on 1) Apparatus in accordance with claim 1, in which the calculator is operative to perform joint stereo coding using a downmix channel as a carrier channel and using, as an input channel, the selected original channel, to generate joint stereo parameters as channel side information for the selected original channel. ### Claim 5 (depends on 3) Apparatus in accordance with claim 3, in which the calculator is operative to perform intensity stereo coding or binaural cue coding, such that the channel side information represent an energy distribution or binaural cue parameters for the selected original channel, wherein a downmix channel or a combined downmix channel is usable as a carrier channel. ### Claim 6 (depends on 1) Apparatus in accordance with claim 1, in which the multi-channel audio signal comprises a left channel, a left surround channel, a right channel and a right surround channel, in which the provider operative to provide the first downmix channel as a left downmix channel and to provide the second downmix channel as a right downmix channel, the left and the right downmix channels being formed such that a result, when played, is a stereo representation of the multi-channel audio signal, and in which the calculator is operative to calculate the channel side information for the left channel as the selected original channel using the left downmix channel, to calculate the channel side information for the right channel as the selected original channel using the right downmix channel, to calculate the channel side information for the left surround channel as the selected original channel using the left downmix channel, and to calculate the channel side information for the right surround channel as the selected original channel using the right downmix channel. ### Claim 7 (depends on 1) Apparatus in accordance with claim 1, in which the original channels comprise a center channel, which further comprises a combiner for combining the first downmix channel and the second downmix channel to acquire the combined downmix channel; and wherein the calculator for calculating the channel side information for the center channel as the selected original channel is operative to calculate the channel side information such that the combined downmix channel when weighted using the channel side information results in an approximation of the original center channel. ### Claim 8 (depends on 1) Apparatus in accordance with claim 1, in which the provider operative to derive the first downmix channel and the second downmix channel from the original channels using a first predetermined linear weighted combination for the first downmix channel and using a second predetermined linear weighted combination for the second downmix channel. ### Claim 9 (depends on 7) Apparatus in accordance with claim 7, in which the first predetermined linear weighted combination is defined as follows: Lc=t(L+aLs+bC); or in which the predetermined second linear weighted combination is defined as follows: Rc=t(R+aRs+bC), wherein Lc is the first downmix channel, wherein Rc is the second downmix channel, wherein t, a and b are weighting factors smaller than 1, wherein L is an original left channel, wherein C is an original center channel, wherein R is an original right channel, wherein Ls is an original left surround channel, and wherein Rs is an original right surround channel. ### Claim 10 (depends on 1) Apparatus in accordance with claim 1, in which the provider operative to receive externally supplied first and second downmix channels. ### Claim 11 (depends on 1) Apparatus in accordance with claim 1, in which the first downmix channel and the second downmix channel are composite channels being composite of the original channels in varying degrees, wherein the calculator is operative, to use, for calculating the channel side information, the downmix channel among both downmix channels, which is stronger influenced by the selected original channel when compared to the other downmix channel. ### Claim 12 (depends on 1) Apparatus in accordance with claim 1, in which the generator is operative to form the output data such that the output data are in compliance with an output data syntax to be used by a low level decoder for processing the first downmix channel or a signal derived from the first downmix channel or the second downmix channel or a signal derived from the second downmix channel to acquire a decoded stereo representation of the multi-channel audio signal. ### Claim 13 (depends on 12) Apparatus in accordance with claim 12, in which the output data syntax is structured such that same comprises a special data field to be ignored by a low level decoder, and in which the generator is operative to insert the channel side information into the special data field. ### Claim 14 (depends on 13) Apparatus in accordance with claim 13, in which the syntax is mp3 syntax and the special data field is an ancillary data field. ### Claim 15 (depends on 12) Apparatus in accordance with claim 12, in which the generator is operative to insert the channel side information into the output data such that the channel side information are only used by a high level decoder but are ignored by the low level decoder. ### Claim 16 (depends on 2) Apparatus in accordance with claim 2, which further comprises an encoder for encoding the first downmix channel to acquire the signal derived from the first downmix channel or for encoding the second downmix channel to acquire the signal derived from the second downmix channel. ### Claim 17 (depends on 16) Apparatus in accordance with claim 16, in which the encoder is a perceptual encoder which comprises a converter for converting a signal to be encoded into a spectral representation, a quantizer for quantizing the spectral representation using a psychoacoustic model and an entropy encoder for entropy encoding a quantized spectral representation to acquire an entropy encoded quantized spectral representation as the signal derived from the first downmix channel or the signal derived from the second downmix channel. ### Claim 18 (depends on 17) Apparatus in accordance with claim 17, in which the perceptual encoder is an encoder in accordance with MPEG-1/2 layer III (mp3) or MPEG-2/4 advanced audio coding (AAC). ### Claim 19 (depends on 1) Apparatus in accordance with claim 1, in which the calculator is operative to calculate downmix energy values for the downmix channel or the combined downmix channel, to calculate an original energy value for the selected original channel, and to calculate a gain factor as the channel side information, the gain factor being derived from the downmix energy value and the original energy value. ### Claim 20 (depends on 1) Apparatus in accordance with claim 1, in which the calculator is operative to calculate frequency dependent channel side information parameters such that for a plurality of frequency bands, a plurality of different channel side information parameters are acquired. ### Claim 21 Method of processing a multi-channel audio signal, the multi-channel audio signal comprising at least three original channels, comprising: providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels; calculating channel side information for a selected original channel of the original signals such that a downmix channel or a combined downmix channel comprising the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel; and generating output data, the output data comprising the channel side information. ### Claim 22 Apparatus for inverse processing of input data, the input data comprising channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multichannel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel comprising the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, the apparatus comprising: an input data reader for reading the input data to acquire the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information; and a channel reconstructor for reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to acquire the approximation of the selected original channel. ### Claim 23 (depends on 22) Apparatus in accordance with claim 22, further comprising a perceptual decoder for decoding the signal derived from the first downmix channel to acquire the decoded version of the first downmix channel and for decoding the signal derived from the second downmix channel to acquire a decoded version of the second downmix channel. ### Claim 24 (depends on 22) Apparatus in accordance with claim 22, further comprising a combiner for combining the first downmix channel and the second downmix channel to acquire the combined downmix channel. ### Claim 25 (depends on 22) Apparatus in accordance with claim 22, in which the original audio signal comprises a left channel, a left surround channel, a right channel, a right surround channel and center channel, wherein the first downmix channel and the second downmix channel are a left downmix channel and a right downmix channel, respectively, and wherein the input data comprise channel side information for at least three of the left channel, the left surround channel, the right channel, the right surround channel and the center channel, wherein the channel reconstructor is operative to reconstruct an approximation of the left channel using channel side information for the left channel and the left downmix channel, to reconstruct an approximation for the left surround channel using channel side information for the left surround channel and the left downmix channel, to reconstruct an approximation for the right channel using channel side information for the right channel and the right downmix channel, and to reconstruct an approximation for the right surround channel using channel side information for the right surround channel and the right downmix channel. ### Claim 26 (depends on 22) Apparatus in accordance with claim 22, in which the channel reconstructor is operative to reconstruct an approximation for the center channel using channel side information for the center channel and the combined downmix channel. ### Claim 27 Method of inverse processing of input data, the input data comprising channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multichannel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel comprising the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, the method comprising: reading the input data to acquire the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information; and reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to acquire the approximation of the selected original channel. ### Claim 28 A non-transitory digital storage medium having a computer program stored thereon to perform the method of processing a multi-channel audio signal, the multi-channel audio signal having at least three original channels, which method comprises: providing a first downmix channel and a second downmix channel, the first and the second downmix channels being derived from the original channels; calculating channel side information for a selected original channel of the original signals such that a downmix channel or a combined downmix channel comprising the first and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel; and generating output data, the output data comprising the channel side information, when said computer program is run by a computer. ### Claim 29 A non-transitory digital storage medium having a computer program stored thereon to perform the method for inverse processing of input data, the input data comprising channel side information, a first downmix channel or a signal derived from the first downmix channel and a second downmix channel or a signal derived from the second downmix channel, wherein the first downmix channel and the second downmix channel are derived from at least three original channels of a multi-channel audio signal, and wherein the channel side information are calculated such that a downmix channel or a combined downmix channel comprising the first downmix channel and the second downmix channel, when weighted using the channel side information, results in an approximation of the selected original channel, which method comprises: reading the input data to acquire the first downmix channel or a signal derived from the first downmix channel and the second downmix channel or a signal derived from the second downmix channel and the channel side information; and reconstructing the approximation of the selected original channel using the channel side information and the downmix channel or the combined downmix channel to acquire the approximation of the selected original channel, when said computer program is run by a computer.