# US 2021/0233543 A1 · Audio encoder, audio decoder, methods and computer program using jointly encoded residual signals

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0233543 A1
- Document ID: US20210233543A1
- Status: Published application
- Kind: A1
- Application number: 16/990,566
- Filed: 2020-08-11
- Published: 2021-07-29
- Earliest claimed: 2013-07-22
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Sascha Dick; Christian Ertel; Christian Helmrich; Johannes Hilpert; Andreas Hoelzer; Achim Kuntz
- Web page: https://patentgrep.com/patent/US20210233543A1

## CPC

- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 19/0017: {Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error (G10L19/24 takes precedence)}
- G10L 21/038: using band spreading techniques
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- H04S 2420/03: Application of parametric coding in stereophonic audio systems
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04S: STEREOPHONIC SYSTEMS

## Abstract

An audio decoder for providing at least four audio channel signals on the basis of an encoded representation is configured to provide a first residual signal and a second residual signal on the basis of a jointly encoded representation of the first residual signal and of the second residual signal using a multi-channel decoding. The audio decoder is configured to provide a first audio channel signal and a second audio channel signal on the basis of a first downmix signal and the first residual signal using a residual-signal-assisted multi-channel decoding. The audio decoder is configured to provide a third audio channel signal and a fourth audio channel signal on the basis of a second downmix signal and the second residual signal using a residual-signal-assisted multi-channel decoding. An audio encoder is based on corresponding considerations.

## Claims

### Claim 1

An audio decoder for providing at least four audio channel signals on the basis of an encoded representation,
wherein the audio decoder is configured to provide a first residual signal and a second residual signal on the basis of a jointly encoded representation of the first residual signal and of the second residual signal using a multi-channel decoding;
wherein the audio decoder is configured to provide a first audio channel signal and a second audio channel signal on the basis of a first downmix signal and the first residual signal using a residual-signal-assisted multi-channel decoding; and
wherein the audio decoder is configured to provide a third audio channel signal and a fourth audio channel signal on the basis of a second downmix signal and the second residual signal using a residual-signal-assisted multi-channel decoding.

### Claim 2 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly-encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding.

### Claim 3 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first residual signal and the second residual signal on the basis of the jointly encoded representation of the first residual signal and of the second residual signal using a prediction-based multi-channel decoding.

### Claim 4 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first residual signal and the second residual signal on the basis of the jointly encoded representation of the first residual signal and of the second residual signal using a residual-signal-assisted multi-channel decoding.

### Claim 5 (depends on 3)

The audio decoder according to claim 3, wherein the prediction-based multi-channel decoding is configured to evaluate a prediction parameter describing a contribution of a signal component, which is derived using a signal component of a previous frame, to the provision of the residual signals of the current frame.

### Claim 6 (depends on 3)

The audio decoder according to claim 3, wherein the prediction-based multi-channel decoding is configured to obtain the first residual signal and the second residual signal on the basis of a downmix signal of the first residual signal and of the second residual signal and on the basis of a common residual signal of the first residual signal and the second residual signal.

### Claim 7 (depends on 6)

The audio decoder according to claim 6, wherein the prediction-based multi-channel decoding is configured to apply the common residual signal with a first sign, to obtain the first residual signal, and to apply the common residual signal with a second sign, which is opposite to the first sign, to obtain the second residual signal.

### Claim 8 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first residual signal and the second residual signal on the basis of the jointly encoded representation of the first residual signal and of the second residual signal using a multi-channel decoding which is operative in a MDCT domain.

### Claim 9 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first residual signal and the second residual signal on the basis of the jointly encoded representation of the first residual signal and of the second residual signal using a USAC Complex Stereo Prediction.

### Claim 10 (depends on 1)

The audio decoder according to claim 1,
wherein the audio decoder is configured to provide the first audio channel signal and the second audio channel signal on the basis of the first downmix signal and the first residual signal using a parameter-based residual-signal-assisted multi-channel decoding; and
wherein the audio decoder is configured to provide the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal and the second residual signal using a parameter-based residual-signal-assisted multi-channel decoding.

### Claim 11 (depends on 10)

The audio decoder according to claim 10, wherein the parameter-based residual-signal-assisted multi-channel decoding is configured to evaluate one or more parameters describing a desired correlation between two channels and/or level differences between two channels in order to provide the two or more audio channel signals on the basis of a respective one of the downmix signals and a corresponding one of the residual signals.

### Claim 12 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configure to provide the first audio channel signal and the second audio channel signal on the basis of the first downmix signal and the first residual signal using a residual-signal-assisted multi-channel decoding which is operative in a QMF domain; and
wherein the audio decoder is configured to provide the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal and the second residual signal using a residual-signal-assisted multi-channel decoding which is operative in the QMF domain.

### Claim 13 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first audio channel signal and the second audio channel signal on the basis of the first downmix signal and the first residual signal using a MPEG Surround 2-1-2 decoding or a Unified Stereo Decoding; and
wherein the audio decoder is configured to provide the third audio channel signal and the fourth audio channel signal on the basis of the second downmix signal and the second residual signal using a MPEG Surround 2-1-2 decoding or a Unified Stereo Decoding.

### Claim 14 (depends on 1)

The audio decoder according to claim 1, wherein the first residual signal and the second residual signal are associated with different horizontal positions of an audio scene or with different azimuth positions of the audio scene.

### Claim 15 (depends on 1)

The audio decoder according to claim 1, wherein the first audio channel signal and the second audio channel signal are associated with vertically neighboring positions of an audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with vertically neighboring positions of the audio scene.

### Claim 16 (depends on 1)

The audio decoder according to claim 1, wherein the first audio channel signal and the second audio channel signal are associated with a first horizontal position or azimuth position of an audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with a second horizontal position or azimuth position of the audio scene, which is different from the first horizontal position or the first azimuth position.

### Claim 17 (depends on 1)

The audio decoder according to claim 1, wherein the first residual signal is associated with a left side of an audio scene, and wherein the second residual signal is associated with a right side of an audio scene.

### Claim 18 (depends on 17)

The audio decoder according to claim 17,
wherein the first audio channel signal and the second audio channel signal are associated with the left side of the audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with the right side of the audio scene.

### Claim 19 (depends on 18)

The audio decoder according to claim 18, wherein the first audio channel signal is associated with a lower left position of the audio scene,
wherein the second audio channel signal is associated with an upper left position of the audio scene,
wherein the third audio channel signal is associated with a lower right position of the audio scene, and
wherein the fourth audio channel signal is associated with an upper right position of the audio scene.

### Claim 20 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly-encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding, wherein the first downmix signal is associated with a left side of an audio scene and the second downmix signal is associated with a right side of the audio scene.

### Claim 21 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly encoded representation of the first downmix signal and of the second downmix signal using a prediction-based multi-channel decoding.

### Claim 22 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly encoded representation of the first downmix signal and of the second downmix signal using a residual-signal-assisted prediction-based multi-channel decoding.

### Claim 23 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to perform a first multi-channel bandwidth extension on the basis of the first audio channel signal and the third audio channel signal, and
wherein the audio decoder is configured to perform a second multi-channel bandwidth extension on the basis of the second audio channel signal and the fourth audio channel signal.

### Claim 24 (depends on 23)

The audio decoder according to claim 23, wherein the audio decoder is configured to perform the first multi-channel bandwidth extension in order to obtain two or more bandwidth-extended audio channel signals associated with a first common horizontal plane or a first common elevation of an audio scene on the basis of the first audio channel signal and the third audio channel signal and one or more bandwidth extension parameters, and
wherein the audio decoder is configured to perform the second multi-channel bandwidth extension in order to obtain two or more bandwidth-extended audio channel signals associated with a second common horizontal plane or a second common elevation of the audio scene on the basis of the second audio channel signal and the fourth audio channel signal and one or more bandwidth extension parameters.

### Claim 25 (depends on 1)

The audio decoder according to claim 1, wherein the jointly encoded representation of the first residual signal and of the second residual signal comprises a channel pair element comprising a downmix signal of the first and second residual signal and a common residual signal of the first and second residual signal.

### Claim 26 (depends on 1)

The audio decoder according to claim 1, wherein the audio decoder is configured to provide the first downmix signal and the second downmix signal on the basis of a jointly-encoded representation of the first downmix signal and the second downmix signal using a multi-channel decoding,
wherein the jointly encoded representation of the first downmix signal and of the second downmix signal comprises a channel pair element comprising a downmix signal of the first and second downmix signal and a common residual signal of the first and second downmix signal.

### Claim 27

An audio encoder for providing an encoded representation on the basis of at least four audio channel signals,
wherein the audio encoder is configured to jointly encode at least a first audio channel signal and a second audio channel signal using a residual-signal-assisted multi-channel encoding, to obtain a first downmix signal and a first residual signal; and
wherein the audio encoder is configured to jointly encode at least a third audio channel signal and a fourth audio channel signal using a residual-signal-assisted multi-channel encoding, to obtain a second downmix signal and a second residual signal; and
wherein the audio encoder is configured to jointly encode the first residual signal and the second residual signal using a multi-channel encoding, to obtain a jointly encoded representation of the residual signals.

### Claim 28 (depends on 27)

The audio encoder according to claim 27, wherein the audio encoder is configured to jointly encode the first downmix signal and the second downmix signal using a multi-channel encoding, to obtain a jointly encoded representation of the downmix signals.

### Claim 29 (depends on 28)

The audio encoder according to claim 28, wherein the audio encoder is configured to jointly encode the first residual signal and the second residual signal using a prediction-based multi-channel encoding, and
wherein the audio encoder is configured to jointly encode the first downmix signal and the second downmix signal using a prediction-based multi-channel encoding.

### Claim 30 (depends on 27)

The audio encoder according to claim 27, wherein the audio encoder is configured to jointly encode at least the first audio channel signal and the second audio channel signal using a parameter-based residual-signal-assisted multi-channel encoding, and
wherein the audio encoder is configured to jointly encode at least the third audio channel signal and the fourth audio channel signal using a parameter-based residual-signal-assisted multi-channel encoding.

### Claim 31 (depends on 27)

The audio encoder according to claim 27, wherein the first audio channel signal and the second audio channel signal are associated with vertically neighboring positions of an audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with vertically neighboring positions of the audio scene.

### Claim 32 (depends on 27)

The audio encoder according to claim 27, wherein the first audio channel signal and the second audio channel signal are associated with a first horizontal position or azimuth position of an audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with a second horizontal position or azimuth position of the audio scene, which is different from the first horizontal position or azimuth position.

### Claim 33 (depends on 27)

The audio encoder according to claim 27, wherein the first residual signal is associated with a left side of an audio scene, and wherein the second residual signal is associated with a right side of the audio scene.

### Claim 34 (depends on 33)

The audio encoder according to claim 33,
wherein the first audio channel signal and the second audio channel signal are associated with the left side of the audio scene, and
wherein the third audio channel signal and the fourth audio channel signal are associated with the right side of the audio scene.

### Claim 35 (depends on 34)

The audio encoder according to claim 34, wherein the first audio channel signal is associated with a lower left position of the audio scene,
wherein the second audio channel signal is associated with an upper left position of the audio scene,
wherein the third audio channel signal is associated with a lower right position of the audio scene, and
wherein the fourth audio channel signal is associated with an upper right position of the audio scene.

### Claim 36 (depends on 27)

The audio encoder according to claim 27, wherein the audio encoder is configured to jointly encode the first downmix signal and the second downmix signal using a multi-channel encoding, to obtain a jointly encoded representation of the downmix signals, wherein the first downmix signal is associated with a left side of an audio scene and the second downmix signal is associated with a right side of the audio scene.

### Claim 37

A method for providing at least four audio channel signals on the basis of an encoded representation, the method comprising:
providing a first residual signal and a second residual signal on the basis of a jointly encoded representation of the first residual signal and the second residual signal using a multi-channel decoding;
providing a first audio channel signal and a second audio channel signal on the basis of a first downmix signal and the first residual signal using a residual-signal-assisted multi-channel decoding; and
providing a third audio channel signal and a fourth audio channel signal on the basis of a second downmix signal and the second residual signal using a residual-signal-assisted multi-channel decoding.

### Claim 38

A method for providing an encoded representation on the basis of at least four audio channel signals, the method comprising:
jointly encoding at least a first audio channel signal and a second audio channel signal using a residual-signal assisted multi-channel encoding, to obtain a first downmix signal and a first residual signal;
jointly encoding at least a third audio channel signal and a fourth audio channel signal using a residual-signal-assisted multi-channel encoding, to obtain a second downmix signal and a second residual signal; and
jointly encoding the first residual signal and the second residual signal using a multi-channel encoding, to obtain an encoded representation of the residual signals.

### Claim 39

A non-transitory digital storage medium having stored thereon computer program for performing a method for providing at least four audio channel signals on the basis of an encoded representation, the method comprising:
providing a first residual signal and a second residual signal on the basis of a jointly encoded representation of the first residual signal and the second residual signal using a multi-channel decoding;
providing a first audio channel signal and a second audio channel signal on the basis of a first downmix signal and the first residual signal using a residual-signal-assisted multi-channel decoding; and
providing a third audio channel signal and a fourth audio channel signal on the basis of a second downmix signal and the second residual signal using a residual-signal-assisted multi-channel decoding,
when said computer program is run by a computer.

### Claim 40

A non-transitory digital storage medium having stored thereon computer program for performing a method for providing an encoded representation on the basis of at least four audio channel signals, the method comprising:
jointly encoding at least a first audio channel signal and a second audio channel signal using a residual-signal assisted multi-channel encoding, to obtain a first downmix signal and a first residual signal;
jointly encoding at least a third audio channel signal and a fourth audio channel signal using a residual-signal-assisted multi-channel encoding, to obtain a second downmix signal and a second residual signal; and
jointly encoding the first residual signal and the second residual signal using a multi-channel encoding, to obtain an encoded representation of the residual signals,
when said computer program is run by a computer.
