Audio processor, audio processing system, audio decoder, method for providing a processed audio signal representation and computer program using a time scale modification

An audio processing system for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processing system is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processing system is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processing system is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals, or on the basis of signals derived from the one or more time scale modified intermediate audio signals;
• wherein functionalities of the audio processing system are distributed to two or more devices.

A non-transitory digital storage medium having a computer program stored thereon to perform the method for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the method comprises performing a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the method comprises performing a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the method comprises performing a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals,
• when said computer program is run by a computer.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing, that is performed by the audio processor, on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein audio processor is configured to perform the first processing for time segments of a first length;
• and
• wherein audio processor is configured to perform the second processing for time segments of a second length which is shorter than the first length, or
• wherein the audio processor is configured to perform the second processing with temporal increments of a second length, which is smaller than the first length, wherein one or more intermediate audio signals of the second length are input into the second processing for each execution of the second processing;
• wherein a granularity of the time scale modification is finer than the second length, such that the time scale modification may result in frame lengths of the one or more time scale modified intermediate audio signals which deviate from integer multiples of the second length.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio signal processor is configured to provide the one or more time-scale-modified intermediate audio signals such that a length of a respective frame of the one or more time-scale-modified intermediate audio signals is different from a multiple of a block size used in the second processing;
• and/or
• wherein the audio processor is configured to perform the second processing with step-wise temporal increments,
• wherein the audio signal processor is configured to provide the one or more time-scale-modified intermediate audio signals such that a length of a respective frame of the one or more time-scale-modified intermediate audio signals is different from a multiple of the temporal increments.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio signal processor is configured to form a block of samples for the second processing on the basis of one or more time domain samples associated with a previous frame of the one or more time-scale-modified intermediate audio signals and on the basis of one or more time domain samples associated with a current frame of the one or more time-scale-modified intermediate audio signals, and
• wherein the audio signal processor is configured to form one or more blocks of samples for the second processing on the basis of a plurality time domain samples associated with a current frame of the one or more time-scale-modified intermediate audio signals, until there are not enough time domain samples associated with a current frame of the one or more time-scale-modified intermediate audio signals left to form a block of samples for the second processing,
• such that blocks of samples for the second processing having uniform length are formed even if the time scale modification changes a frame length to be different from an integer multiple of the length of the blocks of samples used for the second processing, and
• wherein the audio signal processor is configured to buffer one or more residual time domain samples, i.e. excessive time domain samples which are not sufficient for a formation of a block of samples for the second processing, associated with a current frame of the one or more time-scale-modified intermediate audio signals for a formation of a block of samples for the second processing which includes one or more time domain samples associated with a subsequent frame of the one or more time-scale-modified intermediate audio signals.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to use one or more buffered residual time domain samples, i.e. excessive time domain samples which are not sufficient for a formation of a block of samples for the second processing, associated with a previous frame and a maximum possible number of time domain samples associated with the current frame of the one or more time-scale-modified intermediate audio signals for a formation of blocks of samples for the second processing associated with the current frame,
• such that blocks of samples for the second processing having uniform length are formed even if the time scale modification changes a frame length to be different from an integer multiple of the length of the blocks of samples used for the second processing.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to selectively switch to a special time scale modification mode for frames classified as silent;
• and
• wherein the audio processor is configured to perform the time scale modification in such a manner that no buffering of residual frames is required, when operating in the special time scale modification mode;
• wherein a number of time domain samples of a current time-scale modified frame plus a number of buffered time domain samples of a previous time scale modified frame is an integer multiple of a number of time domain samples of a block of time domain samples processed by the second processing, or
• wherein a number of time domain samples of a current time-scale modified frame plus a number of buffered time domain samples of a previous time scale modified frame is an integer multiple of a number of time domain samples input into the second processing to make a temporal increment
• when operating in the special time scale modification mode.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to time-interpolate between one or more processing parameters associated with a previous frame of the one or more intermediate audio signals and one or more processing parameters associated with a current frame of the one or more intermediate audio signals, in order to acquire processing parameters associated with different time instances of a current frame of the one or more time-scale modified intermediate audio signals using an interpolation function which determines contributions of the one or more processing parameters associated with the previous frame and contributions of one or more processing parameters associated with the current frame in a linear combination of processing parameters;
• wherein for a temporally shortened audio frame, the interpolation function comprises a larger slope for a portion of the time scale modified current frame which has been acquired using an overlap-add-combination than for a portion of the time scale modified current frame which is unmodified;
• and/or
• wherein for a temporally lengthened audio frame, the interpolation function comprises a smaller slope for a portion of the time scale modified current frame which has been acquired using an overlap-add-combination than for a portion of the time scale modified current frame which is unmodified.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to time-interpolate between one or more processing parameters associated with a previous frame of the one or more intermediate audio signals and one or more processing parameters associated with a current frame of the one or more intermediate audio signals, in order to acquire processing parameters associated with different time instances of a current frame of the one or more time-scale modified intermediate audio signals using an interpolation function which determines contributions of the one or more processing parameters associated with the previous frame and contributions of one or more processing parameters associated with the current frame in a linear combination of processing parameters;
• wherein the audio processor is configured to acquire interpolated processing parameters Pinter[n] associated with time instances having time index n according to Pinter[n]=ga,f[n]P+(1−ga,f)Pprev,
• wherein P is a processing parameter or a set of processing parameters associated with a current frame of the one or more intermediate audio signals,
• wherein Pprev is a processing parameter or a set of processing parameters associated with a previous frame of the one or more intermediate audio signals,
• wherein Pinter is a processing parameter or a set of processing parameters associated with a time instance n in a current frame of the one or more time scale modified intermediate audio signals, and
• wherein ga,f is an interpolation function;
• wherein the interpolation function ga,f is defined as
• g
• a
• ,
• f
• [
• n
• ]
• =
• max
• ⁡
• (
• 0
• ,
• g
• a
• [
• n
• ]
• )
• ,
• and
• wherein
• g
• a
• [
• n
• ]
• =
• {
• n
• +
• 1
• ⁢
• (
• L
• seg
• -
• L
• rem
• )
• 2
• ⁢
• L
• seg
• ,
• L
• Seg
• ≤
• n
• <
• L
• f
• (
• n
• +
• 1
• )
• ⁢
• 2
• ⁢
• L
• Seg
• +
• 1
• -
• L
• rem
• 2
• ⁢
• L
• seg
• (
• L
• Seg
• -
• 1
• )
• ,
• 0
• ≤
• n
• <
• L
• seg
• wherein n is a time index,
• wherein
• Lrem
• =
• Lf
• -
• Lseg
• ,
• wherein Lf is a length of a frame to be processed;
• wherein Lseg is the length of a portion in which an overlap-and-add is applied by the time scale modification.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to perform, as the second processing, a processing of values of the one or more time-scale-modified intermediate audio signals according to
• y
• [
• n
• ]
• =
• (
• g
• a
• ,
• f
• [
• n
• ]
• ⁢
• M
• +
• (
• 1
• -
• g
• a
• ,
• f
• [
• n
• ]
• )
• ⁢
• M
• p
• ⁢
• r
• ⁢
• e
• ⁢
• v
• )
• ⁢
• x
• [
• n
• ]
• ,
• 0
• ≤
• n
• <
• L
• f
• or according to
• y
• k
• [
• n
• ]
• =
• (
• g
• a
• ,
• f
• [
• n
• ]
• ⁢
• M
• k
• +
• (
• 1
• -
• g
• a
• ,
• f
• [
• n
• ]
• )
• ⁢
• M
• k
• ,
• p
• ⁢
• r
• ⁢
• e
• ⁢
• v
• )
• ⁢
• x
• k
• [
• n
• ]
• ,
• 0
• ≤
• n
• <
• L
• f
• ,
• wherein x[n] and xk[n] are of values or sets of value of the one or more time-scale-modified intermediate audio signals,
• wherein M is a processing parameter or a set of processing parameters associated with a current frame of the one or more intermediate audio signals;
• wherein Mprev is a processing parameter or a set of processing parameters associated with a previous frame of the one or more intermediate audio signals,
• wherein n is a time index,
• wherein Lf is a length of a frame to be processed, and
• wherein ga,f is an interpolation function.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• Wherein the audio processor is configured to determine a mapping vector defining an association between time slots of the one or more time scale modified intermediate audio signals and subframe indices of subframes of the one or more original intermediate audio signals in dependence on the time scale modification such that the mapping describes which processing parameters are allocated with which time slot of the time scale modified intermediate audio signals.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• Wherein the audio processor is configured to determine a mapping vector defining an association between time slots of the one or more time scale modified intermediate audio signals and time slots of the one or more original intermediate audio signals in dependence on the time scale modification,
• such that the mapping vector describes which processing parameters are allocated with which time slot of the time scale modified intermediate audio signals.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to modify a subframe grid of subframes of the one or more time scale modified intermediate audio signals in dependence on a time scale modification of a current frame,
• wherein lengths of subframes of the one or more time scale modified intermediate audio signals are left unchanged, and
• wherein the second processing is configured to perform a subframe-based operation.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• Wherein the audio processor is configured to modify a transient position information in dependence on the time scale modification.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to determine a mapping vector defining an association between time slots of the one or more time scale modified intermediate audio signals and subframe indices of subframes of the one or more original intermediate audio signals in dependence on the time scale modification such that the mapping describes which processing parameters are allocated with which time slot of the time scale modified intermediate audio signals;
• wherein the audio processor is configured to determine the mapping vector according to
• m
• s
• ⁢
• f
• [
• n
• ]
• =
• modulo
• ⁢
• (
• n
• offset
• +
• max
• ⁡
• (
• 0
• ,
• ⌊
• (
• 2
• ⁢
• L
• s
• ⁢
• e
• ⁢
• g
• -
• 1
• )
• ⁢
• g
• a
• ,
• m
• ⁢
• a
• ⁢
• p
• [
• n
• ]
• ⌉
• L
• s
• ⁢
• f
• )
• ,
• L
• m
• ⁢
• d
• )
• or according to
• m
• s
• ⁢
• f
• [
• n
• ]
• =
• max
• ⁡
• (
• 0
• ,
• ⌊
• (
• 2
• ⁢
• L
• s
• ⁢
• e
• ⁢
• g
• -
• 1
• )
• ⁢
• g
• a
• ,
• m
• ⁢
• a
• ⁢
• p
• [
• n
• ]
• ⌉
• L
• s
• ⁢
• f
• )
• Wherein msf[n] is an element of the mapping vector;
• Wherein n is a time slot index;
• Wherein noffset is an offset value;
• Wherein Lseg is a half of a length of a frame of the one or more original intermediate audio signals in time slots;
• Wherein 2Lseg is a length of a frame of the one or more original intermediate audio signals in time slots;
• Wherein Lsf is a length of a subframe in time slots;
• Wherein Lmd is a length of a metadata buffer;
• Wherein noffset is;
• and
• Wherein ga,map[n] is an interpolation function;
• Wherein └ ┐ denotes a rounding to a nearest integer;
• Wherein modulo is a modulo operation and
• Wherein a division is an integer division.

An audio processor for providing a processed audio signal representation on the basis of an input audio signal representation,

• wherein the audio processor is configured to perform a plurality of processing steps, in order to provide the processed audio signal representation on the basis of the input audio signal representation;
• wherein the audio processor is configured to perform a time scale modification on one or more intermediate audio signals, which are provided by a first processing on the basis of the input audio representation, in order to acquire one or more time-scale-modified intermediate audio signals, and
• wherein the audio processor is configured to perform a second processing, which follows the first processing and which provides the processed audio signal representation, on the basis of the one or more time-scale-modified intermediate audio signals;
• wherein the audio processor is configured to determine a mapping vector defining an association between time slots of the one or more time scale modified intermediate audio signals and time slots of the one or more original intermediate audio signals in dependence on the time scale modification,
• such that the mapping vector describes which processing parameters are allocated with which time slot of the time scale modified intermediate audio signals,
• wherein the audio processor is configured to determine the mapping vector according to
• m
• t
• ⁢
• s
• [
• n
• ]
• =
• max
• (
• 0
• ,
• ⌊
• (
• 2
• ⁢
• L
• s
• ⁢
• e
• ⁢
• g
• -
• 1
• )
• ⁢
• g
• a
• ,
• map
• [
• n
• ]
• ⌉
• )
• Wherein n is a time slot index,
• Wherein 2Lseg is a length of a frame of the one or more original intermediate audio signals in time slots;
• and
• Wherein ga,map[n] is an interpolation function.