# US 12,120,351 B2 · Image data compression using switching between a set of transforms

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,120,351 B2
- Document ID: US12120351B2
- Status: Granted
- Kind: B2
- Application number: 16/246,134
- Filed: 2019-01-11
- Published: 2019-05-16
- Granted: 2024-10-15
- Earliest claimed: 2016-07-14
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Santiago De Luxán Hernández; Detlev Marpe; Klaus-Robert Mueller; Heiko Schwarz; Thomas Wiegand
- Web page: https://patentgrep.com/patent/US12120351B2

## CPC

- H04N 19/61: in combination with predictive coding
- H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/463: by compressing encoding parameters before transmission
- H04N 19/625: using discrete cosine transform [DCT]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The coding efficiency of predictive picture codecs using transform-based residual coding is enhanced by allowing the codec to switch between a set of transforms for the sake of the transform-based residual coding. It turned out that even when using explicit signaling from encoder to decoder in order to signal the actual transform to be used out of the set of transforms from encoder to decoder, an increase in coding efficiency may result. Alternatively, the switching between the set of transforms may be performed without explicit signaling at all, or using a combination of explicit signaling and some sort of prediction of the switching.

## Claims

### Claim 1

An apparatus for predictively decoding a picture that is subdivided into a set of segments from a data stream, the apparatus comprising:
a processor; and
a memory having instructions which, when executed by the processor, cause the processor to:
decode, from the data stream, an intra-coding mode signalization for a segment from the set of segments;
select an intra-coding mode for the segment using the intra-coding mode signalization;
decode, from the data stream, a transform class signalization corresponding to the segment, the transform class signalization comprising an even a number of bins;
determine that a prefix portion of the transform class signalization indicates a first state or a second state;
in response to the prefix portion indicating the first state, select, for the segment, a transform from a set of transforms based on a previous transform selection of a neighboring segment without decoding a suffix portion of the transform class signalization, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spectral-to-spatial transforms applied horizontally and a second one of the set of one-dimensional spectral-to-spatial transforms applied vertically;
in response to the prefix portion indicating the second state, decode the suffix portion from the data stream and select, for the segment, a transform from the set of transforms identified by the suffix portion, the suffix portion corresponding to a fixed-length binarization;
continue to parse the data stream for decoding subsequent segments from the set of segments; and
use a first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically,
wherein the intra-coding mode is used to read a first bin of the first plurality of the bins and the intra-coding mode is used to read a first bin of the second plurality of the bins, and wherein the first bin of the first plurality of the bins is used to read a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to read a second bin of the second plurality of bins.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the processor is further configured to:
switch between the transforms in the set of transforms for intra-coded segments, and
use one transform or switch between transforms in a different set of transforms for inter-coded segments.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein in switching between transforms in the set of transforms, the processor is configured to:
select the transform using a mapping of a domain of values, which are indicated by the transform class signalization, onto the transforms in the set of transforms,
wherein each transform in the set of transforms is selectable in a manner independent from the segment's coding mode, and wherein every transform of the set of transforms is selectable.

### Claim 4 (depends on 2)

The apparatus according to claim 2, wherein in switching between the transforms in the set of transforms the processor is configured to:
when the segment is an intra-code segment, select, for the intra-coded segment, the transform depending on the transform class signalization for the intra-coded segment.

### Claim 5 (depends on 1)

The apparatus according to claim 1, wherein the second spectral-to-spatial transform is different from the first spectral-to-spatial transform.

### Claim 6 (depends on 1)

The apparatus according to claim 1, wherein the set of transforms comprises:
a spectral-to-spatial transform which is concatenation of a first one-dimensional spectral-to-spatial transform applied horizontally and a second one-dimensional spectral-to-spatial transform applied vertically,
wherein the first one-dimensional spectral-to-spatial transform is different than the second one-dimensional spectral-to-spatial transform.

### Claim 7 (depends on 1)

The apparatus according to claim 1, wherein the set of transforms is composed of different pairs of one of a set of horizontal transforms and one of a set of vertical transforms.

### Claim 8 (depends on 1)

The apparatus according to claim 1, wherein the set of one-dimensional spectral-to-spatial transforms comprises an identity transform and one or more discrete cosine/sine transforms.

### Claim 9 (depends on 1)

The apparatus according to claim 1, wherein to use the first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and the second plurality of the bins to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically, the processor is configured to:
use the first and second pluralities of bins such that the first one and the second one of the one-dimensional spectral-to-spatial transforms are equal to each other if the first and second pluralities of bins are equal to each other.

### Claim 10 (depends on 1)

The apparatus according to claim 1, wherein to use the first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and the second plurality of the bins to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically, the processor is configured to:
use a first bin of the first plurality of bins and a first bin of the second plurality of bins to discriminate between {identity transform, DST-IV} on the one hand and {DCT-II/III, DCT-IV} on the other hand and a second bin of the first plurality of bins and a second bin of the second plurality of bins to discriminate between {identity transform, DCT-II/III} on the one hand and {DCT-IV, DST-IV} on the other hand.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein the processor is further configured to select a context for context-adaptively entropy decoding the first bin of the first and second pluralities depending on the second bin of the first and second pluralities, respectively.

### Claim 12 (depends on 10)

The apparatus according to claim 10, wherein the processor is further configured to select the context for context-adaptively entropy decoding the second bin of the first and second pluralities depending on the intra-coding mode of the segment for which the transform class signalization is comprised by the data stream.

### Claim 13

An apparatus for predictively encoding a picture that is subdivided into a set of segments into a data stream, the apparatus comprising:
a processor; and
a memory having instructions which, when executed by the processor, cause the apparatus to:
select at least one transform from a set of transforms and an intra-coding mode from a set of intra-coding modes for encoding a segment from the set of segments, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spatial-to-spectral transforms applied horizontally and a second one of the set of one-dimensional spatial-to-spectral transforms applied vertically;
signal the intra-coding mode in the data stream using an intra-coding mode signalization;
use a transform class signalization signal for indicating the selected transform in the data stream, wherein the transform class signalization comprises a number of bins, and wherein the transform class signalization indicates the selected at least one transform using a prefix portion indicating a first state or a second state, the first state indicating that the selected at least one transform is a previous transform selection of a neighboring segment, the second state indicating a switching to at least one different transform specified by a suffix portion of the transform class signalization;
context-adaptively encode the transform class signalization using a context based on the intra-coding mode; and
use a first plurality of the bins to index the first one of the set of one-dimensional spatial-to-spectral transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spatial-to-spectral transforms applied vertically;
wherein the intra-coding mode is used to encode a first bin of the first plurality of the bins and the intra-coding mode is used to encode a first bin of the second plurality of the bins, and wherein the first bin of the first plurality of the bins is used to encode a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to encode a second bin of the second plurality of bins.

### Claim 14 (depends on 13)

The apparatus according to claim 13, wherein the processor is further configured to:
switch between the set of transforms for intra-coded segments and
steadily use one transform or switch between a different set of transforms for inter-coded segments in spatial-to-spectral transforming a prediction residual signal.

### Claim 15 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to:
signal, for each of a set of the segments, one transform of the set of transforms by way of the transform class signalization for the respective segment in the data stream, wherein every transform of the set of transforms is able to be signaled.

### Claim 16 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to:
signal, for each of intra-coded segments, one of the set of transforms by way of the transform class signalization for the respective intra-coded segment in the data stream by encoding the transform class signalization into the data stream using context-based entropy encoding using a context which depends on an intra-coding mode of the respective intra-coded segment.

### Claim 17 (depends on 13)

The apparatus according to claim 13, wherein the first spatial-to-spectral transform is different from the second spatial-to-spectral transform.

### Claim 18

A method for predictively decoding a picture from a data stream, wherein the picture is subdivided into a set of segments, the method comprising:
decoding, from the data stream, an intra-coding mode signalization for a segment from the set of segments;
selecting an intra-coding mode for the segment using the intra-coding mode signalization;
decoding, from the data stream, a transform class signalization corresponding to the segment based on a context dependent on the intra-coding mode, wherein the transform class signalization comprises a number of bins;
determining that a prefix portion of the transform class signalization indicates a first state or a second state;
in response to the prefix portion indicating the first state, selecting, for the segment, a transform from a set of transforms based on a previous transform selection of a neighboring segment without decoding a suffix portion of the transform class signalization decoding, from the data stream, a transform class signalization corresponding to the segment based on a context dependent on the intra-coding mode, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spectral-to-spatial transforms applied horizontally and a second one of the set of one-dimensional spectral-to-spatial transforms applied vertically;
in response to the prefix portion indicating the second state, decoding the suffix portion from the data stream and selecting, for the segment, a transform from the set of transforms identified by the suffix portion, the suffix portion corresponding to a fixed-length binarization;
continue parsing the data stream for decoding subsequent segments from the set of segments; and
using a first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically;
wherein the intra-coding mode is used to read a first bin of the first plurality of the bins and the intra-coding mode is used to read a first bin of the second plurality of the bins, and wherein the first bin of the first plurality of the bins is used to read a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to read a second bin of the second plurality of bins.

### Claim 19 (depends on 18)

The method according to claim 18, further comprising:
switching between the transforms in the set of transforms for intra-coded segments, and using one transform or switching between transforms in a different set of transforms for inter-coded segments.

### Claim 20 (depends on 19)

The method according to claim 19, wherein in switching between transforms in the set of transforms, the method further comprises:
selecting the transform using a mapping of a domain of values, which are indicated by the transform class signalization, onto the transforms in the set of transforms,
wherein each transform in the set of transforms is selectable in a manner independent from the segment's coding mode, and wherein every transform of the set of transforms is selectable.

### Claim 21 (depends on 19)

The method according to claim 19, wherein in the switching between the transforms in the set of transforms, where the segment is an intra-code segment, the method further comprises:
when the segment is an intra-code segment, selecting, for the intra-coded segment, the transform depending on the transform class signalization for the intra-coded segment.

### Claim 22 (depends on 18)

The method according to claim 18, wherein the first spectral-to-spatial transform is different from the second spectral-to-spatial transform.

### Claim 23 (depends on 18)

The method according to claim 18, wherein the set of transforms comprises:
a spectral-to-spatial transform which is concatenation of a first one-dimensional spectral-to-spatial transform applied horizontally and a second one-dimensional spectral-to-spatial transform applied vertically,
wherein the first one-dimensional spectral-to-spatial transform is different than the second one-dimensional spectral-to-spatial transform.

### Claim 24 (depends on 18)

The method according to claim 18, wherein the set of transforms is composed of different pairs of one of a set of horizontal transforms and one of a set of vertical transforms.

### Claim 25 (depends on 18)

The method according to claim 18, wherein the set of one-dimensional spectral-to-spatial transforms comprises an identity transform and one or more discrete cosine/sine transforms.

### Claim 26 (depends on 18)

The method according to claim 18, wherein using the first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and the second plurality of the bins to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically, comprises:
using the first and second pluralities of bins such that the first one and the second one of the one-dimensional spectral-to-spatial transforms are equal to each other if the first and second pluralities of bins are equal to each other.

### Claim 27 (depends on 18)

The method according to claim 18, wherein using the first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and the second plurality of the bins to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically, comprises:
using a first bin of the first plurality of bins and a first bin of the second plurality of bins to discriminate between {identity transform, DST-IV} on the one hand and {DCT-II/III, DCT-IV} on the other hand and a second bin of the first plurality of bins and a second bin of the second plurality of bins to discriminate between {identity transform, DCT-II/III} on the one hand and { DCT-IV, DST-IV} on the other hand.

### Claim 28 (depends on 27)

The method according to claim 27, further comprising selecting a context for context-adaptively entropy decoding the first bin of the first and second pluralities depending on the second bin of the first and second pluralities, respectively.

### Claim 29 (depends on 27)

The method according to claim 27, further comprising selecting the context for context-adaptively entropy decoding the second bin of the first and second pluralities depending on the intra-coding mode of the segment for which the transform class signalization is comprised by the data stream.

### Claim 30

A method for predictively encoding a picture that is subdivided into a set of segments, the method comprising:
selecting at least one transform from a set of transforms and an intra-coding mode from a set of intra-coding modes for encoding a segment from the set of segments, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spatial-to-spectral transforms applied horizontally and a second one of the set of one-dimensional spatial-to- spectral transforms applied vertically;
signaling the intra-coding mode in the data stream using an intra-coding mode signalization;
using a transform class signalization signal for indicating the selected transform in the data stream, wherein the transform class signalization comprises a number of bins, and wherein the transform class signalization indicates the selected at least one transform using a prefix portion indicating a first state or a second state, the first state indicating that the selected at least one transform is a previous transform selection of a neighboring segment, the second state indicating a switching to at least one different transform specified by a suffix portion of the transform class signalization;
context-adaptively encoding the transform class signalization using a context based on the intra-coding mode; and
using a first plurality of the bins to index the first one of the set of one-dimensional spatial-to-spectral transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spatial-to-spectral transforms applied vertically;
wherein the intra-coding mode is used to encode a first bin of the first plurality of the bins and the intra-coding mode is used to encode a first bin of the second plurality of the bins, and wherein the first bin of the first plurality of the bins is used to encode a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to encode a second bin of the second plurality of bins.

### Claim 31 (depends on 30)

The method according to claim 30, further comprising:
switching between the set of transforms for intra-coded segments; and
steadily using one transform or switching between a different set of transforms for inter-coded segments in spatial-to-spectral transforming a prediction residual signal.

### Claim 32 (depends on 30)

The method according to claim 30, further comprising signaling, for each of a set of the segments, one transform of the set of transforms by way of the transform class signalization for the respective segment in the data stream, wherein every transform of the set of transforms is able to be signaled.

### Claim 33 (depends on 30)

The method according to claim 30, further comprising signaling, for each of intra-coded segments, one of the set of transforms by way of the transform class signalization for the respective intra-coded segment in the data stream by encoding the transform class signalization into the data stream using context-based entropy encoding using a context which depends on an intra-coding mode of the respective intra-coded segment.

### Claim 34 (depends on 30)

The method according to claim 30, wherein the first spatial-to-spectral transform is different from the second spatial-to-spectral transform.

### Claim 35

A non-transitory computer readable medium encoded with a computer program that, when executed by a computer, causes the computer to implement a method for predictively decoding a picture that is subdivided into a set of segments from a data stream, the method comprising:
decoding, from the data stream, an intra-coding mode signalization for a segment from the set of segments;
selecting an intra-coding mode for the segment using the intra-coding mode signalization;
decoding, from the data stream, a transform class signalization corresponding to the segment based on a context dependent on the intra-coding mode, the transform class signalization comprising a number of bins;
determining that a prefix portion of the transform class signalization indicates a first state or a second state;
in response to the prefix portion indicating the first state, selecting for the segment, a transform from a set of transforms based on a previous transform selection of a neighboring segment without decoding a suffix portion of the transform class signalization, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spectral-to-spatial transforms applied horizontally and a second one of the set of one-dimensional spectral-to-spatial transforms applied vertically;
in response to the prefix portion indicating the second state, decoding the suffix portion from the data stream and select, for the segment, a transform from the set of transforms identified by the suffix portion, the suffix portion corresponding to a fixed-length binarization;
continue to parse the data stream for decoding subsequent segments from the set of segments; and
using a first plurality of the bins to index the first one of the set of one-dimensional spectral-to-spatial transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spectral-to-spatial transforms applied vertically;
wherein the intra-coding mode is used to read a first bin of the first plurality of the bins and the intra-coding mode is used to read a first bin of the second plurality of the bins. and wherein the first bin of the first plurality of the bins is used to read a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to read a second bin of the second plurality of bins.

### Claim 36

A non-transitory computer readable medium encoded with a computer program that, when executed by a computer, causes the computer to implement a method for predictively encoding a picture that is subdivided into a set of segments into a data stream, the method comprising:
selecting at least one transform from a set of transforms and an intra-coding mode from a set of intra-coding modes for encoding a segment from the set of segments, wherein the set of transforms is comprised of concatenations of a first one of a set of one-dimensional spatial-to-spectral transforms applied horizontally and a second one of the set of one-dimensional spatial-to-spectral transforms applied vertically;
signaling the intra-coding mode in the data stream using an intra-coding mode signalization ;
using a transform class signalization signal for indicating the selected transform in the data stream, wherein the transform class signalization comprises a number of bins, and wherein the transform class signalization indicates the selected at least one transform using a prefix portion indicating a first state or a second state, the first state indicating that the selected at least one transform is a previous transform selection of a neighboring segment, the second state indicating a switching to at least one different transform specified by a suffix portion of the transform class signalization;
context-adaptively encode the transform class signalization using a context based on the intra-coding mode; and
using a first plurality of the bins to index the first one of the set of one-dimensional spatial-to-spectral transforms applied horizontally and a second plurality of the bins, that is different than the first plurality of the bins, to index the second one of the set of one-dimensional spatial-to-spectral transforms applied vertically;
wherein the intra-coding mode is used to encode a first bin of the first plurality of the bins and the intra-coding mode is used to encode a first bin of the second plurality of the bins, and wherein the first bin of the first plurality of the bins is used to encode a second bin of the first plurality of bins and the first bin of the second plurality of the bins is used to encode a second bin of the second plurality of bins.