# US 12,003,706 B2 · Combination of mode-dependent and fixed transform types in video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,706 B2
- Document ID: US12003706B2
- Status: Granted
- Kind: B2
- Application number: 17/700,298
- Filed: 2022-03-21
- Published: 2022-07-07
- Granted: 2024-06-04
- Earliest claimed: 2019-12-12
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Lester Lu; Debargha Mukherjee; Elliott Karpilovsky
- Web page: https://patentgrep.com/patent/US12003706B2

## CPC

- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N 19/122: Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/50: using predictive coding (H04N19/61 takes precedence)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Decoding video data includes, for a block encoded using a prediction mode, determining a transform mode for the block using the prediction mode. The transform mode is a first mode when the prediction mode is an inter-prediction mode and is a second mode when the prediction mode is an intra-prediction mode. The first mode is an available first transform type that is a combination of transforms selected from first fixed transforms and first learned transforms that each comprise a respective transformation matrix generated iteratively using blocks predicted using the inter-prediction mode. The second mode is an available second transform type that is a combination of transforms selected from second fixed transforms, which is a proper subset of the first fixed transforms, and a second learned transform comprising a transformation matrix that is generated iteratively using blocks predicted using the intra-prediction mode. Decoding the block uses the prediction and transform modes.

## Claims

### Claim 1

A method for decoding video data from a video bitstream, the method comprising:
for a respective block of a plurality of blocks encoded into the video bitstream using a prediction mode:
determining a transform mode for the block using the prediction mode, wherein:
the transform mode comprises a first transform mode responsive to the prediction mode being an inter-prediction mode,
the transform mode comprises a second transform mode responsive to the prediction mode being an intra-prediction mode of a plurality of available intra-prediction modes,
the first transform mode comprises one of a plurality of available first transform types, each of the plurality of available first transform types comprising respective combinations of transforms selected from the group consisting of:
first fixed transforms, each comprising a respective fixed transformation matrix, and
first mode-dependent, learned transforms, each comprising a respective transformation matrix that was generated iteratively using blocks predicted using the inter-prediction mode,
the second transform mode comprises one of a plurality of available second transform types, each of the plurality of available second transform types comprising respective combinations of transforms selected from the group consisting of:
second fixed transforms, each comprising a respective fixed transformation matrix, wherein the second fixed transforms comprise a proper subset of the first fixed transforms, and
a second mode-dependent, learned transform comprising a transformation matrix that is generated iteratively using blocks predicted using the intra-prediction mode; and
decoding the block using the prediction mode and the transform mode.

### Claim 2 (depends on 1)

The method of claim 1, wherein the available first transform types is greater in number than the available second transform types.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first mode-dependent, learned transforms include a first learned transform and a flipped version of the first learned transform, the second mode-dependent, learned transform includes a second learned transform, and the available second transform types omit a flipped version of the second learned transform.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
each of the first mode-dependent, learned transforms comprises a one-dimensional (1D) transform,
the available first transform types comprise combinations of a vertical transform and a horizontal transform, and
the vertical transform of a proper subset of the available first transform types includes at least one of the 1D transform or a flipped transform of the 1D transform.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
the first fixed transforms include a discrete cosine transform (DCT), and
the horizontal transform of the proper subset of the available first transform types includes the DCT.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
each of the first mode-dependent, learned transforms comprises a one-dimensional (1D) transform,
the available first transform types comprise combinations of a vertical transform and a horizontal transform, and
the horizontal transform of a proper subset of the available first transform types includes one of the 1D transform or a flipped transform of the 1D transform.

### Claim 7 (depends on 6)

The method of claim 6, wherein:
the first fixed transforms include a discrete cosine transform (DCT), and
the vertical transform of the proper subset of the available first transform types includes the DCT.

### Claim 8

An apparatus for decoding video data from a video bitstream, the apparatus comprising:
a processor configured to:
for a block of a plurality of blocks encoded into the video bitstream using a prediction mode:
determine a transform mode for the block using the prediction mode, wherein:
the transform mode comprises a first transform mode responsive to the prediction mode being an inter-prediction mode,
the transform mode comprises a second transform mode responsive to the prediction mode being an intra-prediction mode of a plurality of available intra-prediction modes,
the first transform mode comprises one of a plurality of available first transform types, each of the plurality of available first transform types comprising respective combinations of transforms selected from the group consisting of:
first fixed transforms, each comprising a respective fixed transformation matrix, and
first mode-dependent, learned transforms, each comprising a respective transformation matrix that was generated iteratively using blocks predicted using the inter-prediction mode,
the second transform mode comprises one of a plurality of available second transform types, each of the plurality of available second transform types comprising respective combinations of transforms selected from the group consisting of:
second fixed transforms, each comprising a respective fixed transformation matrix, wherein the second fixed transforms comprise a proper subset of the first fixed transforms, and
a second mode-dependent, learned transform comprising a transformation matrix that is generated iteratively using blocks predicted using the intra-prediction mode; and
decode the block using the prediction mode and the transform mode.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the available first transform types is greater in number than the available second transform types.

### Claim 10 (depends on 8)

The apparatus of claim 8, wherein:
the first mode-dependent, learned transforms include a first learned transform,
the available first transform types each include at least one of first learned transform or a flipped version of the first learned transform,
the second mode-dependent, learned transform includes a second learned transform, and
the available second transform types omit a flipped version of the second learned transform.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein:
the second, mode-dependent learned transform is one a plurality of second, mode-dependent learned transforms, each generated iteratively using blocks predicted using a respective intra-prediction mode of the available intra-prediction modes,
the second mode-dependent, learned transform comprises a one-dimensional (1D) transform,
the available second transform types comprise combinations of a vertical transform and a horizontal transform, and
the vertical transform of a proper subset of the available second transform types includes the 1D transform.

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein:
the second, mode-dependent learned transform is one a plurality of second, mode-dependent learned transforms, each generated iteratively using blocks predicted using a respective intra-prediction mode of the plurality of available intra-prediction modes,
the second mode-dependent, learned transform comprises a one-dimensional (1D) transform,
the available second transform types comprise combinations of a vertical transform and a horizontal transform, and
the horizontal transform of a proper subset of the available second transform types includes the 1D transform.

### Claim 13 (depends on 8)

The apparatus of claim 8, wherein the processor is configured to:
receive, from the video bitstream, a header associated with the block; and
determine, from the header, the prediction mode, wherein to determine the transform mode comprises to:
determine, from the prediction mode, that the transform mode is the first transform mode, and
determine, from the header, that the first transform mode includes a first transform of the first mode-dependent, learned transforms.

### Claim 14 (depends on 8)

The apparatus of claim 8, wherein:
the first fixed transforms comprise a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (FLIPADST), and an identity transform,
a first mode-dependent, learned transforms comprise a first learned transform (MDTX1) and a flipped MDTX1 (FMDTX1),
the available first transform types comprise combinations of transforms selected from the group consisting of the DCT, the ADST, the FLIPADST, and the identity transform, and combinations of transforms selected from the group consisting of the MDTX1, the FMDTX1, and the DCT,
the second fixed transforms comprise the DCT, the ADST, and the identity transform, and omit the FLIPADST,
the second mode-dependent, learned transform comprises a second learned transform (MDTX2), and
the available second transform types comprise combinations of transforms selected from the group consisting of the DCT, the ADST, and the identity transform, and combinations of transforms selected from the group consisting of the MDTX2 and the DCT.

### Claim 15 (depends on 8)

The apparatus of claim 8, wherein:
the available first transform types for a proper subset of the available intra-prediction modes comprise only the first fixed transforms, and
the available first transform types for remaining intra-prediction modes of the available intra-prediction modes comprise at least a first learned transform of the first mode-dependent, learned transforms.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein:
at least some of the available first transform types for remaining intra-prediction modes of the available intra-prediction modes comprise a discrete cosine transform (DCT).

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the first fixed transforms comprise the DCT, an asymmetric discrete sine transform (ADST), a flipped ADST (FLIPADST), and an identity transform.

### Claim 18

An apparatus for encoding video data into a video bitstream, the apparatus comprising:
a processor configured to, for a respective block of a plurality of blocks of the video data:
determine a transform mode for the block using a prediction mode for the block, wherein:
the transform mode comprises a first transform mode responsive to the prediction mode being an inter-prediction mode,
the transform mode comprises a second transform mode responsive to the prediction mode being an intra-prediction mode of a plurality of available intra-prediction modes,
the first transform mode comprises one of a plurality of available first transform types, each of the available first transform types comprising respective combinations of transforms selected from the group consisting of:
first fixed transforms, each comprising a respective fixed transformation matrix, and
first mode-dependent, learned transforms, each comprising a respective transformation matrix that was generated iteratively using blocks predicted using the inter-prediction mode,
the second transform mode comprises one of a plurality of available second transform types, each of the available second transform types comprising respective combinations of transforms selected from the group consisting of:
second fixed transforms, each comprising a respective fixed transformation matrix, wherein the second fixed transforms comprise a proper subset of the first fixed transforms, and
a second mode-dependent, learned transform comprising a transformation matrix that is generated iteratively using blocks predicted using the intra-prediction mode; and
encode the block into the video bitstream using the prediction mode and the transform mode.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein the available first transform types is greater in number than the available second transform types.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein the processor is configured to:
when the transform mode is the second transform mode:
encode, in a header associated with the block, a flag that indicates whether the transform mode includes the second mode-dependent transform; and
encode, in the header associated with the block, bits indicating the transform mode from an index including those of the available second transform types including only the first fixed transforms or an index including those of the available second transform types including the second mode-dependent transform.