Reconstruction of Audio Scenes from a Downmix

An audio encoding system configured to encode a time frame of an audio scene at least comprising N>1 audio objects as a bitstream,

• each audio object (Sn, n=1, . . . , N) being associated with positional metadata ({right arrow over (x)}n, n=1, . . . , N),
• the system comprising:
• a downmixer for receiving the audio objects and outputting, based thereon, a downmix signal comprising M downmix channels (Ym, m=1, . . . , M), wherein M>1, each downmix channel is a linear combination of one or more of the N audio objects, and each downmix channel is associated with a positional locator ({right arrow over (z)}m, m=1, . . . , M);
• a downmix encoder for encoding the downmix signal and including this in the bitstream;
• an upmix coefficient analyzer for receiving the spatial metadata of an audio object and the spatial locators of the downmix channels and computing, based thereon, correlation coefficients (dn=(dn,1, . . . , dn,M)) indicative of the spatial relatedness of the audio object and each downmix channel;
• and
• a metadata encoder for encoding the positional metadata and the object gains and including these in the bitstream,
• wherein the upmix coefficient analyzer is further configured, for a frequency band of an audio object, to receive the downmix signal (Y) and the correlation coefficients (dn) relating to the audio object and to determine, based thereon, an object gain (gn) in such manner that an inner product of the correlation coefficients and the downmix signal resealed by the object gain (gn×dnTY) approximates the audio object in that frequency band of the time frame.

A method for reconstructing a time frame of an audio scene with at least a plurality of audio objects from a bitstream, the method comprising:

• extracting from the bitstream, for each of N audio objects, an object gain (gn,n=1, . . . , N) and positional metadata ({right arrow over (x)}n, n=1, . . . , N) associated with each audio object, wherein N>1, wherein the object gain and positional metadata are encoded in the bitstream;
• extracting a downmix signal (Y) from the bitstream, the downmix signal comprising M downmix channels (Ym, m=1, . . . , M), wherein M>1 and each downmix channel is associated with a positional locator ({right arrow over (z)}m, m=1, . . . , M);
• for each audio object:
• computing, on the basis of the positional metadata of the audio object and the spatial locators of the downmix channels, correlation coefficients (dn=(dn,1, . . . , dn,M)) indicative of the spatial relatedness of the audio object and each downmix channel;
• and
• reconstructing the audio object as an inner product of the correlation coefficients and the downmix signal resealed by the object gain (Ŝn=gn×dnTY).

An audio decoding system configured to reconstruct a time frame of an audio scene at least comprising a plurality of audio objects based on a bitstream, the system comprising:

• a metadata decoder for receiving the bitstream and extracting from this, for each of N audio objects, an object gain (gn, n=1, . . . , N) and positional metadata ({right arrow over (x)}n, n=1, . . . , N) associated with each audio object, wherein N>1, wherein the object gain and positional metadata are encoded in the bitstream;
• a downmix decoder for receiving the bitstream and extracting from this a downmix signal (Y) comprising M downmix channels (Ym, m=1, . . . , M), wherein M>1;
• an upmix coefficient decoder storing, for each downmix channel, an associated positional locator ({right arrow over (z)}m, m=1, . . . , M) and being configured to compute correlation coefficients (dn=(dn,1, . . . , dn,M)) indicative of the spatial relatedness of the audio object and each downmix channel, on the basis of the positional locators of the downmix channels and the positional metadata of an audio object;
• and
• an upmixer for reconstructing an audio object on the basis of the correlation coefficients and the object gains, wherein the audio object is reconstructed as an inner product of the correlation coefficients and the downmix signal resealed by the object gain (Ŝn=gn×dnTY).

A method for reconstructing a time/frequency tile of an audio scene with at least one audio object (Sn, n=NB+1, . . . , N), which is associated with positional metadata ({right arrow over (x)}n, n=NB+1, . . . , N), and at least one bed channel (Sn, n=1, . . . , NB), the method comprising:

• receiving a bitstream;
• from the bitstream, extracting a downmix signal (Y) comprising M downmix channels, each of which comprises a linear combination of one or more of the audio object(s) and the bed channel(s) (Ym=Σn=1Ndm,nSn, m=1, . . . , M) in accordance with downmix coefficients (dm,n, m=1, . . . , M, n=1, . . . , N),
• wherein each of the NB≦M bed channels is associated with a corresponding downmix channel;
• from the bitstream, further extracting the positional metadata of the audio objects or the downmix coefficients;
• and
• reconstructing a bed channel as the corresponding downmix channel after suppressing the content representing at least one audio object from the corresponding downmix channel, wherein the suppression is made either on the basis of a positional locator ({right arrow over (z)}m, m=1, . . . , M), with which the corresponding downmix channel is associated, and the extracted positional metadata of the audio objects, or on the basis of the downmix coefficients;
• wherein the bed channel is reconstructed by suppressing content representing so many audio objects that the signal energy of the remaining content representing audio objects is below a predefined threshold.

The method of, further comprising:

• computing, on the basis of the positional metadata and the positional locator of the corresponding downmix channel, the downmix coefficients applied to the audio objects or obtaining the downmix coefficients extracted from the bitstream;
• optionally reconstructing the audio objects based on at least the downmix coefficients;
• estimating an energy (E[Sn2], n=NB+1, . . . , N) of at least one audio object based on the reconstructed audio objects or based on the downmix coefficients and the downmix signal;
• and
• for a bed channel (Sn for some n=1, . . . , NB):
• estimating the energy (E[Yn2]) of the corresponding downmix channel;
• and
• reconstructing the bed channel as a resealed version of the corresponding downmix channel (Ŝn=hnYn), wherein the scaling factor (hn) is based on the estimated energy of said at least one of the audio objects, the energy of the corresponding downmix channel and the downmix coefficients (dn,NB+1, dn,NB+2, . . . , dn,N) controlling contributions from the audio objects to the corresponding downmix channel.