Apparatus and method for rendering audio objects

An apparatus for generating loudspeaker signals for a plurality of loudspeakers so that an application of the loudspeaker signals at the plurality of loudspeakers renders at least one audio object at an intended virtual position, wherein the plurality of loudspeakers are distributed onto one or more horizontal layers, the apparatus comprising a computer or an electronic circuit implementing

• an interface configured to receive an audio input signal which represents the at least one audio object,
• a first loudspeaker signal set determiner, configured to determine, depending on the intended virtual position, first panning gains for a first set of loudspeakers of the plurality of loudspeakers, and use the first panning gains to derive first partial loudspeaker signals from the at least one audio input signal, which are associated with a rendering of the at least one audio object at a first virtual position upon application of the first partial loudspeaker signals onto the first set of loudspeakers,
• a second loudspeaker signal set determiner, configured to, by spectral shaping and by panning gains, derive second partial loudspeaker signals from the at least one audio input signal, the second partial loudspeaker signals being associated with a rendering of the at least one audio object at a second virtual position upon application of the second partial loudspeaker signals onto a second set of loudspeakers of the plurality of loudspeakers, wherein the panning gains are selected so that the second virtual position is above or below the one or more horizontal layers and corresponds to a horizontal position which coincides with a listener position along a vertical projection, and
• a vertical panning gain determiner configured to, depending on the intended virtual position, determine further panning gains for the first and second partial loudspeaker signals so as to pan between the first and second virtual positions, and
• a composer configured to compose the loudspeaker signals from the first and second partial loudspeaker signals using the further panning gains,
• a) wherein the second loudspeaker signal set determiner is configured so that the second virtual position is vertically above the one or more horizontal layers and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, or
• wherein the second loudspeaker signal set determiner is configured so that the second virtual position is vertically below the one or more horizontal layers and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• b) wherein the second loudspeaker signal set determiner is configured so that the second virtual position is vertically above the one or more horizontal layers and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, or
• wherein the second loudspeaker signal set determiner is configured so that the second virtual position is vertically below the one or more horizontal layers and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz with an intermediate reduction of the dampening within a spectral subrange within the spectral range, located between 5 and 10 kHz, and amplified between 500 Hz and 1 kHz or
• c) wherein the second loudspeaker signal set determiner is configured to,
• if the intended virtual position is vertically above the one or more horizontal layers, position the second virtual position to be vertically above the one or more horizontal layers, perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges Between 1000 and 10 kHz, and
• if the intended virtual position is vertically below the one or more horizontal layers, position the second virtual position to be vertically below the one or more horizontal layers, perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• d) wherein the composer is configured to be responsive to a change of the intended virtual position from an in-layer position, which is vertically within or in-between the one or more layers, to a position vertically offset from the one or more horizontal layers, by
• controlling the further panning gains so as to fade from composing the loudspeaker signals purely from the first partial loudspeaker signals to composing the loudspeaker signals from the first and second partial loudspeaker signals so that the further panning gains pan from the first virtual position towards the second virtual position.

A system comprising:

• a plurality of loudspeakers;
• a first apparatus for generating loudspeaker signals for the plurality of loudspeakers so that an application of the loudspeaker signals at the plurality of loudspeakers renders at least one audio object at an intended virtual position, the first apparatus comprising a computer or a electronic circuit implementing
• a first apparatus interface configured to receive an audio input signal which represents the at least one audio object,
• a first panning gain determiner, configured to determine, depending on the intended virtual position, first panning gains for a first set of loudspeakers of the plurality of loudspeakers, which are arranged within a first horizontal layer, the first panning gains defining a derivation of first partial loudspeaker signals from the audio input signal, which are associated with a rendering of the at least one audio object at a first virtual position upon application of the first partial loudspeaker signals onto the first set of loudspeakers,
• a first apparatus vertical panning gain determiner, configured to determine, depending on the intended virtual position, further panning gains for a panning between the first partial loudspeaker signals and one or more second partial loudspeaker signals which is to be applied to a second set of loudspeakers of the plurality of loudspeakers, which is arranged within a second horizontal layer, which is vertically offset relative to the first layer set, and is associated with a rendering of the at least one audio object at a second virtual position so as to pan between the first virtual position and the second virtual position, wherein the first apparatus is configured to compose the loudspeaker signals from the audio input signal using the first panning gains and the further panning gains,
• wherein the first apparatus is adaptive to different setups of the plurality of loudspeakers and configured to associate the plurality of loudspeakers to a plurality of horizontal layers so that one of the loudspeakers may be associated with different ones of the horizontal layers, and to select the first horizontal layer and the second horizontal layer out of the plurality of horizontal layers so that the intended virtual position is between the first horizontal layer and the second horizontal layer;
• and
• a second apparatus for generating loudspeaker signals for the plurality of loudspeakers so that an application of the loudspeaker signals at the plurality of loudspeakers renders the at least one audio object at the intended virtual position, wherein the plurality of loudspeakers are distributed onto one or more horizontal layers, the second apparatus comprising a second computer or a second electronic circuit implementing
• a second apparatus interface configured to receive the audio input signal which represents the at least one audio object,
• a first loudspeaker signal set determiner, configured to determine, depending on the intended virtual position, first panning gains for the first set of loudspeakers of the plurality of loudspeakers, and use the first panning gains to derive the first partial loudspeaker signals from the audio input signal, which are associated with the rendering of the at least one audio object at the first virtual position upon application of the first partial loudspeaker signals onto the first set of loudspeakers,
• a second loudspeaker signal set determiner, configured to, by spectral shaping and by panning gains, derive second partial loudspeaker signals from the audio input signal, the second partial loudspeaker signals being associated with the rendering of the at least one audio object at the second virtual position upon application of the second partial loudspeaker signals onto the second set of loudspeakers of the plurality of loudspeakers, wherein the panning gains are selected so that the second virtual position is above or below the one or more horizontal layers and corresponds to a horizontal position which coincides with a listener position along a vertical projection, and
• a second apparatus vertical panning gain determiner configured to, depending on the intended virtual position, determine further panning gains for the first and second partial loudspeaker signals so as to pan between the first and second virtual positions, and
• a composer configured to compose the loudspeaker signals from the first and second partial loudspeaker signals using the further panning gains.

Method for generating loudspeaker signals for a plurality of loudspeakers so that an application of the loudspeaker signals at the plurality of loudspeakers renders at least one audio object at an intended virtual position, the method comprising

• receiving an audio input signal which represents the at least one audio object,
• determining, depending on the intended virtual position, first panning gains for a first set of loudspeakers of the plurality of loudspeakers, which are arranged within a first layer set of one or more first horizontal layers, the first panning gains defining a derivation of first partial loudspeaker signals from the at least one audio input signal, which are associated with a rendering of the at least one audio object at a first virtual position upon application of the first partial loudspeaker signals onto the first set of loudspeakers,
• determining, depending on the intended virtual position, further panning gains for a panning between the first partial loudspeaker signals and one or more second partial loudspeaker signals which is to be applied to a second set of one or more loudspeakers, and is associated with a rendering of the at least one audio object at a second position so as to pan between the first virtual position and the second position,
• composing the loudspeaker signals from the audio input signal using the first panning gains and the further panning gains, and
• adapting to different setups of the plurality of loudspeakers by associating the plurality of loudspeakers to a plurality of horizontal layers so that one of the loudspeakers may be associated with different ones of the horizontal layers, and selecting the first horizontal layer and the second horizontal layer out of the plurality of horizontal layers depending on the intended virtual position so that, if the intended virtual position is between the horizontal layers, the first horizontal layer and the second horizontal layer are vertically offset to each other with the intended virtual position being between the first horizontal layer and the second horizontal layer or, if the intended virtual position is vertically offset to all horizontal layers towards above or below the horizontal layers, the first horizontal layer and the second horizontal layer are an outmost layer of the horizontal layers nearest to the intended virtual position,
• wherein the second set of one or more loudspeakers comprises more than one loudspeaker, the one or more second partial loudspeaker signals comprise more than one second partial loudspeaker signals and the method further comprises
• determining, depending on the intended virtual position, second panning gains for the second set of loudspeakers, the second panning gains defining a derivation of the second partial loudspeaker signals from the at least one audio input signal, and
• wherein the loudspeaker signals are composed from the audio input signal using the first and second panning gains and the further panning gains, wherein
• a) the second partial loudspeaker signals are derived by spectral shaping which mimics properties of a Head Related Transfer Function, HRTF, along a perception direction from the second position so that the second position is a virtual position vertically offset from the second horizontal layer and a vertical projection of the second virtual position onto the second horizontal layer coincides with a listener position, or
• b) the second partial loudspeaker signals are derived by spectral shaping
• so that the second position is vertically above the second layer set and a vertical projection of the second position onto the second horizontal layer coincides with a listener position and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio input signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, or
• so that the second position is vertically below the second layer set and a vertical projection of the second virtual position onto the second horizontal layer coincides with a listener position and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• c) the second partial loudspeaker signals are derived by spectral shaping
• so that the second position is vertically above the second layer set and a vertical projection of the second position onto the second horizontal layer coincides with a listener position and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio input signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, or
• so that the second position is vertically below the second layer set and a vertical projection of the second position onto the second horizontal layer coincides with a listener position and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz with an intermediate reduction of the dampening within a spectral subrange within the spectral range, located between 5 and 10 KHz, and amplified between 500 Hz and 1 kHz, or
• d) the method comprises
• if the intended virtual position is vertically above the second layer set, positioning the second position to be vertically above the second layer set and so that a vertical projection of the second position onto the second horizontal layer coincides with a listener position, and deriving the second partial loudspeaker signals by, spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges Between 1000 and 10 kHz, and
• if the intended virtual position is vertically below the second layer set, positioning the second virtual position to be vertically below the second layer set and so that a vertical projection of the second position onto the second horizontal layer coincides with a listener position, and deriving the second partial loudspeaker signals by spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• e) the second partial loudspeaker signals are derived involving spectral shaping as an option and the plurality of loudspeakers forms a setup in which the loudspeakers are associated with horizontal layers, and the apparatus is configured to be responsive to a change of the intended virtual position so as to
• if the intended virtual position is between two horizontal layers,
• select the first layer set to be a first of the two horizontal layers and the second layer set to be a second of the two horizontal layers, and the first set out of loudspeakers associated with the first horizontal layer and the second set out of loudspeakers associated with the second horizontal layer, wherein the spectral shaping is switched off, so that the first virtual position is within the first horizontal layer and the second virtual position is within the second horizontal layer and the first virtual position and the second position coincide in a vertical projection, and
• if the intended virtual position is vertically offset to all horizontal layers towards above or below the horizontal layers,
• select the first layer set and the second layer set to be an outmost layer of the horizontal layers nearest to the intended virtual position, and the first set and the second set out of loudspeakers associated with the outmost layer, wherein the spectral shaping is used, so that the second position is a virtual position vertically offset relative to the outmost layer towards a direction at which the intended virtual position lies and a vertical projection of the second position onto the second horizontal layer coincides with a listener position, or
• f) the second partial loudspeaker signals are derived involving spectral shaping as an option and the plurality of loudspeakers forms a setup in which the loudspeakers are associated with one or more horizontal layers, and the apparatus is configured to be responsive to a number of the one or more horizontal layers and a change of the intended virtual position so as to
• if the number of one or more horizontal layers is larger than one,
• if the intended virtual position is between two horizontal layers,
• select the first layer set to be a first of the two horizontal layers and the second layer set to be a second of the two horizontal layers, and the first set out of loudspeakers associated with the first horizontal layer and the second set out of loudspeakers associated with the second horizontal layer, wherein the spectral shaping is switched off, so that the first virtual position is within the first horizontal layer and the second virtual position is within the second horizontal layer and the first virtual position and the second position coincide in a vertical projection, and
• if the intended virtual position is vertically offset to all horizontal layers towards above or below the horizontal layers,
• select the first layer set and the second layer set to be an outmost layer of the horizontal layers nearest to the intended virtual position, and the first set and the second set out of loudspeakers associated with the outmost layer, wherein the spectral shaping is used, so that the second position is a virtual position vertically offset relative to the outmost layer towards a direction at which the intended virtual position lies and a vertical projection of the second position onto the second horizontal layer coincides with a listener position, and
• if the number of one or more horizontal layers is one,
• if the intended virtual position is within the one horizontal layer,
• the loudspeaker signals are composed purely from the first partial loudspeaker signals, and
• if the intended virtual position is vertically offset to the one horizontal layer, the first layer set and the second layer set are selected to be the one horizontal layer, and the first set and the second set out of loudspeakers associated with the one horizontal layer, wherein the spectral shaping is used, so that the second position is a virtual position vertically offset relative to the one horizontal layer towards a direction at which the intended virtual position lies and a vertical projection of the second position onto the second horizontal layer coincides with a listener position.

Method for generating loudspeaker signals for a plurality of loudspeakers so that an application of the loudspeaker signals at the plurality of loudspeakers renders at least one audio object at an intended virtual position, wherein the plurality of loudspeakers are distributed onto one or more horizontal layers, the method comprising

• receiving an audio input signal which represents the at least one audio object,
• determining, depending on the intended virtual position, first panning gains for a first set of loudspeakers of the plurality of loudspeakers, and use the first panning gains to derive first partial loudspeaker signals from the at least one audio input signal, which are associated with a rendering of the at least one audio object at a first virtual position upon application of the first partial loudspeaker signals onto the first set of loudspeakers,
• by spectral shaping, deriving second partial loudspeaker signals from the at least one audio input signal, the second partial loudspeaker signals being associated with a rendering of the at least one audio object at a second virtual position upon application of the second partial loudspeaker signals onto a second set of loudspeakers, the second virtual position being above or below the one or more horizontal layers, and
• depending on the intended virtual position, determining further panning gains for the first and second partial loudspeaker signals so as to pan between the first and second virtual positions, and
• composing the loudspeaker signals from the first and second partial loudspeaker signals using the further panning gains,
• a) wherein the second virtual position is vertically above the one or more horizontal layers and the spectral shaping is performed so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 KHz, or
• wherein the second virtual position is vertically below the one or more horizontal layers and to perform the spectral shaping so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• b) wherein the second virtual position is vertically above the one or more horizontal layers and the spectral shaping is performed so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, or
• wherein the second virtual position is vertically below the one or more horizontal layers and the spectral shaping is performed so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz with an intermediate reduction of the dampening within a spectral subrange within the spectral range, located between 5 and 10 kHz, and amplified between 500 Hz and 1 kHz or
• c) wherein
• if the intended virtual position is vertically above the one or more horizontal layers, the second virtual position is positioned to be vertically above the one or more horizontal layers, and the spectral shaping is performed so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a notch spectral range between 200 and 1000 Hz and amplified within one or more in peak spectral ranges between 1000 and 10 kHz, and
• if the intended virtual position is vertically below the one or more horizontal layers, the second virtual position is positioned to be vertically below the one or more horizontal layers, the spectral shaping is performed so that the second partial loudspeaker signals are, relative to the at least one audio signal, dampened in a spectral range above 1000 Hz, or
• d) wherein, responsive to a change of the intended virtual position from an in-layer position, which is vertically within or in-between the one or more layers, to a position vertically offset from the one or more horizontal layers,
• the further panning gains are controlled so as to fade from composing the loudspeaker signals purely from the first partial loudspeaker signals to composing the loudspeaker signals from the first and second partial loudspeaker signals so that the further panning gains pan from the first virtual position towards the second virtual position.