# US 2012/0250760 A1 · Apparatus and Method for Generating a Coded Video Sequence and for Decoding a Coded Video Sequence by Using an Intermediate Layer Residual Value Prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0250760 A1
- Document ID: US20120250760A1
- Status: Published application
- Kind: A1
- Application number: 13/523,981
- Filed: 2012-06-15
- Published: 2012-10-04
- Earliest claimed: 2004-10-15
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Heiko Schwarz; Detlev Marpe; Thomas Wiegand
- Web page: https://patentgrep.com/patent/US20120250760A1

## CPC

- H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
- H04N 19/103: Selection of coding mode or of prediction mode
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/109: among a plurality of temporal predictive coding modes
- H04N 19/11: among a plurality of spatial predictive coding modes
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/19: using optimisation based on Lagrange multipliers
- H04N 19/31: in the temporal domain
- H04N 19/33: in the spatial domain
- H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/51: Motion estimation or motion compensation
- H04N 19/52: by predictive encoding
- H04N 19/567: Motion estimation based on rate distortion criteria
- H04N 19/61: in combination with predictive coding
- H04N 19/615: using motion compensated temporal filtering [MCTF]
- H04N 19/63: using sub-band based transform, e.g. wavelets
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video coder performs a motion-compensated prediction both in the base layer and in an enhancement layer to determine motion data of the enhancement layer by using the motion data from the base layer and/or to predict sequences of residual error pictures after the motion-compensated prediction in the enhancement layer by using sequences of residual error pictures from the base layer via an intermediate layer predictor. On the decoder side, an intermediate layer combiner is used for canceling this intermediate layer prediction. Thereby, the data rate is improved compared to scalability schemes without intermediate layer prediction with the same picture quality.

## Claims

### Claim 1

An apparatus for decoding a coded video sequence having a base scaling layer and an enhancement scaling layer, comprising:
a base picture decoder configured for decoding the base scaling layer to obtain a decoded base sequence of residual error pictures and base layer motion data, the base layer motion data indicating how a macroblock in a current picture has moved in relation to another picture in a group of pictures;
an enhancement picture decoder configured for decoding the enhancement scaling layer to obtain enhancement layer prediction residual error pictures;
an intermediate layer combiner configured for combining the decoded base sequence of residual error pictures or an interpolated base sequence of residual error pictures with the enhancement layer prediction residual error pictures to obtain an enhancement sequence of residual error pictures; and
an enhancement motion combiner configured for obtaining a sequence of pictures of the enhancement scaling layer by using the enhancement sequence of residual error pictures and enhancement layer motion data,
wherein the pictures of the base scaling layer have a lower resolution than the pictures of the enhancement scaling layer, and
wherein the apparatus further comprises a residual value interpolator to interpolate the decoded base sequence of residual error pictures to the higher resolution.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the base picture decoder is formed to dequantize by using a base quantization parameter which is larger than an enhancement quantization parameter, and
wherein the intermediate layer combiner is formed to use the decoded base sequence of residual error pictures or an interpolated version of the same.

### Claim 3 (depends on 1)

The apparatus according to claim 1, wherein the base motion combiner and the enhancement motion combiner are formed to perform only an inverse prediction operation or an inverse prediction operation and an inverse update operation.

### Claim 4 (depends on 1)

The apparatus according to claim 1, wherein an interpolator is formed to use an interpolation filter operating by using a plurality of pixels in an environment of the pixel to be interpolated.

### Claim 5 (depends on 4)

The apparatus according to claim 4, wherein the interpolation filter is formed to use a synthesized value instead of the pixel of the other transformation block when a pixel of another transformation block is present in the proximity of the pixel to be interpolated.

### Claim 6 (depends on 1)

The apparatus according to claim 1, wherein the coded video sequence has enhancement motion data for macroblocks, wherein a motion data control signal is associated to the enhancement motion data, which indicates whether a prediction signal is underlying the enhancement motion data, which is derived from a scaled motion vector of the base motion data, or whether a prediction signal is underlying the enhancement motion data, which is not derived from the base motion data, and wherein the apparatus is formed to read the motion data control signal and to calculate the enhancement motion data in dependence on the control signal by using the scaled version of the base motion data.

### Claim 7 (depends on 1)

The apparatus according to claim 1, wherein the coded video sequence for macroblocks has an intermediate layer prediction control signal in the enhancement scaling layer, which indicates whether a macroblock has been generated due to an intermediate layer prediction or without an intermediate layer prediction, and
wherein the apparatus is further formed to activate the intermediate layer combiner only when the intermediate layer prediction control signal indicates an intermediate layer prediction for a considered macroblock.

### Claim 8 (depends on 1)

Apparatus according to claim 1, further comprising a base motion combiner to obtain a sequence of pictures of the base scaling layer by using the base motion data and the decoded base sequence of residual error pictures, wherein the base motion combiner and the enhancement motion combiner are formed to perform an inverse prediction operation and an inverse update operation.

### Claim 9

A method for decoding a coded video sequence with a base scaling layer and an enhancement scaling layer, comprising:
decoding, by a base picture decoder, the base scaling layer to obtain a decoded base sequence of residual error pictures and base layer motion data, the base layer motion data indicating how a macroblock in a current picture has moved in relation to another picture in a group of pictures;
decoding, by an enhancement picture decoder, the enhancement scaling layer to obtain enhancement prediction layer residual error pictures;
combining, by an intermediate layer combiner, the decoded base sequence of residual error pictures or an interpolated base sequence of residual error pictures with the enhancement layer prediction residual error pictures to obtain an enhancement sequence of residual error pictures; and
performing, by an enhancement motion combiner, an enhancement motion combination to obtain a sequence of pictures of the enhancement scaling layer by using the enhancement sequence of residual error pictures and enhancement motion data,
wherein the pictures of the base scaling layer have a lower resolution than the pictures of the enhancement scaling layer, and
wherein the method further comprises interpolating the decoded base sequence of residual error pictures to the higher resolution.

### Claim 10

Non-transitory storage medium having stored thereon a computer-readable storage medium encoded with a computer program, causing a computer to execute a method for decoding a coded video sequence with a base scaling layer and an enhancement scaling layer, the method comprising:
decoding the base scaling layer to obtain a decoded base sequence of residual error pictures and base layer motion data, the base layer motion data indicating how a macroblock in a current picture has moved in relation to another picture in a group of pictures;
decoding the enhancement scaling layer to obtain enhancement layer prediction residual error pictures;
combining the decoded base sequence of residual error pictures or an interpolated base sequence of residual error pictures with the enhancement layer prediction residual error pictures to obtain an enhancement sequence of residual error pictures; and
performing an enhancement motion combination to obtain a sequence of pictures of the enhancement scaling layer by using the enhancement sequence of residual error pictures and enhancement motion data,
wherein the pictures of the base scaling layer have a lower resolution than the pictures of the enhancement scaling layer, and
wherein the method further comprises interpolating the decoded base sequence of residual error pictures to the higher resolution.
