# US 2024/0205458 A1 · Transform prediction with parsing independent coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0205458 A1
- Document ID: US20240205458A1
- Status: Published application
- Kind: A1
- Application number: 18/542,850
- Filed: 2023-12-18
- Published: 2024-06-20
- Earliest claimed: 2022-12-20
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Onur Guleryuz; Zeyu Deng; Debargha Mukherjee; Lester Lu; Yue Chen
- Web page: https://patentgrep.com/patent/US20240205458A1

## CPC

- H04N 19/61: in combination with predictive coding
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N 19/124: Quantisation
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/172: the region being a picture, frame or field
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Transform prediction with parsing independent coding includes generating a reconstructed frame and outputting the reconstructed frame. Generating the reconstructed frame includes entropy decoding transform blocks for the reconstructed frame, entropy decoding decoded transform identifiers for the transform blocks, obtaining transform-specific probability distributions for available transforms, and, for a current transform block from the transform blocks, identifying a current remapped transform identifier from the decoded transform identifiers, identifying a current transform identifier in accordance with the current remapped transform identifier, the transform coefficients from the current transform block, and the transform-specific probability distributions, identifying a current transform in accordance with the current transform identifier; inverse transforming, in accordance with the current transform, the current transform block to obtain a current residual block and obtaining a current reconstructed block using the current residual block. Generating the reconstructed frame includes including the current reconstructed block in the reconstructed frame.

## Claims

### Claim 1

A method comprising:
obtaining an encoded bitstream;
generating a reconstructed frame, wherein the reconstructed frame includes a plurality of blocks, wherein generating the reconstructed frame includes:
entropy decoding, from the encoded bitstream, transform blocks for the reconstructed frame;
entropy decoding, from the encoded bitstream, decoded transform identifiers for the transform blocks;
obtaining transform-specific probability distributions for available transforms;
for a current transform block from the transform blocks:
identifying a current remapped transform identifier from the decoded transform identifiers;
identifying a current transform identifier, wherein identifying the current transform identifier includes identifying the current transform identifier in accordance with the current remapped transform identifier, the transform coefficients from the current transform block, and the transform-specific probability distributions;
identifying a current transform in accordance with the current transform identifier;
inverse transforming, in accordance with the current transform, the current transform block to obtain a current residual block; and
obtaining a current reconstructed block using the current residual block; and
including the current reconstructed block in the reconstructed frame; and
outputting the reconstructed frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein entropy decoding the transform identifiers includes entropy decoding the transform identifiers subsequent to entropy decoding the transform blocks.

### Claim 3 (depends on 1)

The method of claim 1, wherein entropy decoding the transform identifiers includes entropy decoding the transform identifiers independent of entropy decoding the transform blocks.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying the current transform identifier includes inverse remapping the current remapped transform identifier.

### Claim 5

A method comprising:
obtaining a current frame from an input video stream;
obtaining an encoded frame by encoding the current frame, wherein encoding the current frame includes:
for a current block from the current frame:
obtaining, as a residual block, a difference between the current block and a prediction reference block obtained for coding the current block;
determining a transform identifier of a transform, from a plurality of available transforms, for transforming the residual block;
obtaining an entropy coded block by entropy coding a transform block obtained by transforming the residual block using the transform;
including the entropy coded block in a plurality of entropy coded blocks for the current frame;
obtaining a transform-specific probability distribution for the transform;
obtaining an entropy coded transform identifier by entropy coding the transform identifier using the transform-specific probability distribution; and
including the entropy coded transform identifier in a plurality of entropy coded transform identifiers for the current frame;
including the plurality of entropy coded blocks in an output bitstream;
including the plurality of entropy coded transform identifiers in the output bitstream; and
outputting the output bitstream.

### Claim 6 (depends on 5)

The method of claim 5, wherein obtaining the entropy coded block includes:
obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and
entropy coding the quantized transform coefficients.

### Claim 7 (depends on 5)

The method of claim 5, wherein obtaining the transform-specific probability distribution includes:
generating the transform-specific probability distribution using transform domain training data.

### Claim 8 (depends on 5)

The method of claim 5, wherein obtaining the transform-specific probability distribution includes:
generating the transform-specific probability distribution using pixel domain training data.

### Claim 9 (depends on 5)

The method of claim 5, wherein determining the transform identifier includes determining the transform for transforming the residual block from available transforms, wherein a respective transform identifier is associated with a respective transform from the available transforms.

### Claim 10 (depends on 9)

The method of claim 9, wherein obtaining the transform-specific probability distribution includes obtaining the transform-specific probability distribution in accordance with the transform block and trained statistical probability distributions for the available transforms.

### Claim 11 (depends on 10)

The method of claim 10, wherein obtaining the transform-specific probability distribution includes:
obtaining updated trained statistical probability distributions for the available transforms by updating the trained statistical probability distributions in accordance with the prediction reference block.

### Claim 12 (depends on 10)

The method of claim 10, wherein:
obtaining the entropy coded block includes:
obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and
entropy coding the quantized transform coefficients; and
obtaining the transform-specific probability distribution includes:
obtaining a plurality of reconstructed residual block candidates by:
for a respective transform from the available transforms:
obtaining a reconstructed residual block candidate by inverse transforming the quantized transform coefficients in accordance with the respective transform; and
including the reconstructed residual block candidate in the plurality of reconstructed residual block candidates;
obtaining the transform-specific probability distribution output by an artificial neural network in response to the prediction reference block and the plurality of reconstructed residual block candidates.

### Claim 13

An apparatus comprising:
a non-transitory computer readable medium; and
a processor configured to execute instructions stored on the non-transitory computer readable medium to:
obtain a current frame from an input video stream;
obtain an encoded frame, wherein, to obtain the encoded frame the processor executes the instructions to encode the current frame, wherein to encode the current frame the processor executes the instructions to:
for a current block from the current frame:
obtain as a residual block, a difference between the current block and a prediction reference block obtained for coding the current block;
determine a transform identifier of a transform, from a plurality of available transforms, for transforming the residual block;
obtain an entropy coded block, wherein, to obtain the entropy coded block, the processor executes the instructions to entropy code a transform block obtained by transformation of the residual block using the transform;
include the entropy coded block in a plurality of entropy coded blocks for the current frame;
obtain a transform-specific probability distribution for the transform;
obtain an entropy coded transform identifier, wherein to obtain the entropy coded transform identifier the processor executes the instructions to entropy code the transform identifier using the transform-specific probability distribution; and
include the entropy coded transform identifier in a plurality of entropy coded transform identifiers for the current frame;
include the plurality of entropy coded blocks in an output bitstream;
include the plurality of entropy coded transform identifiers in the output bitstream; and
output the output bitstream.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein obtaining the entropy coded block includes:
obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and
entropy coding the quantized transform coefficients.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein obtaining the transform-specific probability distribution includes:
generating the transform-specific probability distribution using transform domain training data.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein obtaining the transform-specific probability distribution includes:
generating the transform-specific probability distribution using pixel domain training data.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein determining the transform identifier includes determining the transform for transforming the residual block from available transforms, wherein a respective transform identifier is associated with a respective transform from the available transforms.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein obtaining the transform-specific probability distribution includes obtaining the transform-specific probability distribution in accordance with the transform block and trained statistical probability distributions for the available transforms.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein obtaining the transform-specific probability distribution includes:
obtaining updated trained statistical probability distributions for the available transforms by updating the trained statistical probability distributions in accordance with the prediction reference block.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein:
obtaining the entropy coded block includes:
obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and
entropy coding the quantized transform coefficients; and
obtaining the transform-specific probability distribution includes:
obtaining a plurality of reconstructed residual block candidates by:
for a respective transform from the available transforms:
obtaining a reconstructed residual block candidate by inverse transforming the quantized transform coefficients in accordance with the respective transform; and
including the reconstructed residual block candidate in the plurality of reconstructed residual block candidates;
obtaining the transform-specific probability distribution output by an artificial neural network in response to the prediction reference block and the plurality of reconstructed residual block candidates.