# US 12,120,321 B2 · Refined block-based predictive coding and decoding of a picture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,120,321 B2
- Document ID: US12120321B2
- Status: Granted
- Kind: B2
- Application number: 17/036,828
- Filed: 2020-09-29
- Published: 2021-03-18
- Granted: 2024-10-15
- Earliest claimed: 2018-03-29
- Assignees: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
- Inventors: Phan Hoang Tung Nguyen; Gerhard Tech; Jonathan Pfaff; Michael Schäfer; Jennifer Rasch; Tobias Hinz; Heiko Schwarz; Detlev Marpe; Thomas Wiegand
- Web page: https://patentgrep.com/patent/US12120321B2

## CPC

- H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks
- H04N 19/82: involving filtering within a prediction loop
- G06T 5/10: using non-spatial domain filtering
- G06T 5/70: Denoising; Smoothing
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/176: the region being a block, e.g. a macroblock
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20052: Discrete cosine transform [DCT]
- G06T 2207/20056: Discrete and fast Fourier transform, [DFT, FFT]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The refinement of block-based predictive coding and decoding of pictures as applicable in, e.g., hybrid video codecs, using additional reconstructed samples or residuals is described. An apparatus for block-based predictive decoding of a picture is described. The apparatus includes a combiner configured to combine a residual signal a predetermined block of the picture and a reference signal for the predetermined block so as to obtain a first spectrum, the residual signal correcting a prediction error of a prediction of the predetermined block of the picture; a reducer configured to perform thresholding on the first spectrum to obtain a second spectrum so that coefficients below a threshold value are set to a predefined value; an extractor configured to obtain from the second spectrum a modified version of the residual signal; and a reconstructor block configured to decode the predetermined block of the picture from the data stream on the basis of the modified version of the residual signal.

## Claims

### Claim 1

An apparatus for block-based predictive decoding of a picture, the apparatus comprising:
a processor and memory, the processor configured to:
decode, from a data stream, an index and a residual signal for a current block of the picture, the index indicating a threshold value for the current block, the residual signal for correcting a prediction error of a prediction signal that is derived for the current block;
modify an employed signal comprising at least one of the prediction signal and the residual signal, wherein the modified employed signal is obtained by:
comparing, for each coefficient position within a reference signal for the current block, a respective reference signal coefficient against the threshold value, and
when the respective reference signal coefficient is below the threshold value, setting a coefficient of the employed signal at that coefficient position to a predefined value; and
decode the current block from the data stream based on the modified employed signal.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the processor is further configured to:
combine the employed signal and the reference signal so as to obtain a first set of coefficients;
set, at the corresponding coefficient position within the first set of coefficients, the coefficient to the predefined value, to obtain a modified first set of coefficients; and
obtain from the modified first set of coefficients the modified employed signal.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the processor is further configured to:
perform a transform on a region composed of the prediction signal and the reference signal so as to obtain a first set of coefficients of the region; and
subject the modified first set of coefficients to a re-transform so as to obtain a modified version of the region including the modified prediction signal.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the reference signal comprises a neighboring reconstructed residual.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the threshold value is based on a quantization parameter associated with the residual signal.

### Claim 6

A method for block-based predictive decoding of a picture, comprising:
decoding, from a data stream, an index indicating a threshold value and a residual signal for a current block of the picture, the residual signal correcting a prediction error of a prediction signal that is derived for the current block of the picture;
modifying an employed signal comprising at least one of the prediction signal and the residual signal, wherein the modified employed signal is obtained by:
comparing, for each coefficient position within a reference signal for the current block, a respective reference signal coefficient against the threshold value, and
when the respective reference signal coefficient is below the threshold value, setting a coefficient of the employed signal at that coefficient position to a predefined value; and
decoding the current block from the data stream based on the modified employed signal.

### Claim 7 (depends on 6)

The method of claim 6, wherein the reference signal comprises a neighboring reconstructed residual.

### Claim 8 (depends on 6)

The method of claim 6, wherein the threshold value is based on a quantization parameter associated with the residual signal.

### Claim 9

An apparatus for block-based predictive encoding of a picture, the apparatus comprising:
a processor and memory, the processor configured to:
encode a residual signal for a current block of the picture and an index indicating a threshold value for the current block into a data stream, the residual signal for correcting a prediction error of a prediction signal for the current block;
modify an employed signal comprising at least one of the prediction signal and the residual signal, wherein the modified employed signal is obtained by:
comparing, for each coefficient position within a reference signal for the current block, a respective reference signal coefficient against the threshold value, and
when the respective reference signal coefficient is below the threshold value, setting a coefficient of the employed signal at that coefficient position to a predefined value; and
encode the current block to the data stream based on the modified employed signal.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the processor is further configured to:
combine the employed signal and the reference signal so as to obtain a first set of coefficients;
set, at the corresponding coefficient position within the first set of coefficients, the coefficient to the predefined value, to obtain a modified first set of coefficients; and
obtain from the modified first set of coefficients the modified employed signal.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to:
perform a transform on a region composed of the prediction signal and the reference signal so as to obtain a first set of coefficients of the region; and
subject the modified first set of coefficients to a re-transform so as to obtain a modified version of the region including the modified prediction signal.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the threshold value is based on a quantization parameter associated with the residual signal.

### Claim 13

A method for block-based predictive encoding of a picture, comprising:
encoding a residual signal for a current block of the picture and an index indicating a threshold value for the current block into a data stream, the residual signal for correcting a prediction error of a prediction signal for the current block of the picture;
modifying an employed signal comprising at least one of the prediction signal and the residual signal, wherein the modified employed signal is obtained by:
comparing, for each coefficient position within a reference signal for the current block, a respective reference signal coefficient against the threshold value, and
when the respective reference signal coefficient is below the threshold value, setting a coefficient of the employed signal at that coefficient position to a predefined value; and
encoding the current block to the data stream based on the modified employed signal.

### Claim 14 (depends on 13)

The method of claim 13, wherein the threshold value is based on a quantization parameter associated with the residual signal.