# US 2022/0286686 A1 · Video encoding and decoding using differential modulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0286686 A1
- Document ID: US20220286686A1
- Status: Published application
- Kind: A1
- Application number: 17/826,302
- Filed: 2022-05-27
- Published: 2022-09-08
- Earliest claimed: 2019-12-02
- Assignees: Hyundai Motor Company; Kia Corporation; Ewha University-Industry Collaboration Foundation
- Applicants: Hyundai Motor Company; Kia Corporation; Ewha University - Industry Collaboration Foundation
- Inventors: Je Won Kang; Seung Wook Park; Wha Pyeong Lim
- Web page: https://patentgrep.com/patent/US20220286686A1

## CPC

- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/593: involving spatial prediction techniques
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed is a video encoding and decoding technology using a differential modulation technique for a residual signal in order to enhance compression performance in a transform skip mode. The differential modulation technique is applicable to both luma and chroma components as long as the chroma format is not monochrome (i.e. as long as chroma_format_idc=0 is not true).

## Claims

### Claim 1

A method for decoding a bitstream encoded from a video sequence comprising a plurality of pictures, the method comprising:
decoding, from a sequence parameter set of the bitstream, information on a chroma sampling format and high level syntax elements related to differential modulation for residual signals, wherein the high level syntax elements comprise a control flag indicating whether the differential modulation is permitted for the residual signals of the video sequence and information on a maximum block size for which the differential modulation is permitted;
decoding a first chroma syntax element indicating whether the differential modulation has been applied to a chroma block corresponding to a first block to be currently decoded, when the first block satisfies conditions defined by the information on the chroma sampling format and the high level syntax elements;
decoding a second chroma syntax element indicating a differential modulation direction applied to the chroma block in response to the first chroma syntax element indicating that the differential modulation has been applied to the chroma block;
determining a differential-modulated residual block for the chroma block from the bitstream;
determining quantized residual values by differential-demodulating the differential-modulated residual block in the differential modulation direction indicated by the second chroma syntax element;
inversely quantizing the quantized residual values for the chroma block;
generating predicted values for the chroma block; and
reconstructing the chroma block corresponding to the first block based on the inversely quantized residual values and the predicted values,
wherein the conditions defined by the information on the chroma sampling format and the high level syntax elements are satisfied when
(1) the control flag indicates that the differential modulation is permitted for the video sequence,
(2) a value obtained by dividing a width of the first block by a horizontal scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size, and
(3) a value obtained by dividing a height of the first block by a vertical scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the predicted values for the chroma block comprises:
setting, as an intra prediction mode for the chroma block, a prediction mode having a direction identical with the differential modulation direction indicated by the second chroma syntax element among a vertical prediction mode and a horizontal prediction mode; and
generating the predicted values for the chroma block based on the intra prediction mode for the chroma block.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the prediction values for the chroma block comprises:
generating the prediction values for the chroma block by performing intra prediction on the chroma block by using samples not filtered from edge samples of a top or left block of the chroma block.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the quantized residual values comprises:
determining, as in an equation below, the quantized residual values when the second chroma syntax element indicates that the differential modulation direction is horizontal.
Q(ri,j)=Σk=0j{tilde over (r)}i,k, 0≤i≤(M−1), 0≤j≤(N−1)
(wherein Q(ri,j) is a quantized residual value at a position [i, j] within the chroma block, {tilde over (r)}i,k is a sample value at a position [i, k] within the differential-modulated residual block, M is the number of rows of the chroma block, and N is the number of columns of the chroma block.)

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the quantized residual values comprises, determining, as in an equation below, the quantized residual values when the second chroma syntax element indicates that the differential modulation direction is vertical.
Q(ri,j)=Σk=0i{tilde over (r)}k,j, 0≤i≤(M−1), 0≤j≤(N−1)
(wherein Q(ri,j) is a quantized residual value at a position [i, j] within the chroma block, {tilde over (r)}k,j is a sample value at a position [k, j] within the differential-modulated residual block, M is the number of rows of the chroma block, and N is the number of columns of the chroma block.)

### Claim 6 (depends on 1)

The method of claim 1, wherein the information on the maximum block size for which the differential modulation is permitted is information on a maximum transform size for which a transform skip is permitted.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
decoding a first luma syntax element indicating whether the differential modulation has been applied to a luma block corresponding to the first block, when the first block satisfies the condition defined by the high level syntax elements;
decoding a second luma syntax element indicating a differential modulation direction applied to the luma block, in response to the first luma syntax element indicating that the differential modulation has been applied to the luma block;
determining a differential-modulated residual block for the luma block from the bitstream;
determining quantized residual values by differential-demodulating the differential-modulated residual block for the luma block in a differential modulation direction indicated by the second luma syntax element;
inversely quantizing the quantized residual values for the luma block;
generating predicted values for the luma block; and
reconstructing the luma block corresponding to the first block based on the inversely quantized residual values and the predicted values.

### Claim 8 (depends on 7)

The method of claim 7, wherein the condition defined by the high level syntax elements is satisfied when
the control flag indicates that the differential modulation is permitted for the video sequence,
a width of the first block is smaller than or equal to the maximum block size, and
a height of the first block is smaller than or equal to the maximum block size.

### Claim 9 (depends on 7)

The method of claim 7, wherein in determining an MPM list for a luma block corresponding to a second block neighboring the luma block corresponding to the first block, an intra prediction mode of the luma block corresponding to the first block is set as a prediction mode having a direction identical with the differential modulation direction indicated by the second luma syntax element among a vertical prediction mode and a horizontal prediction mode.

### Claim 10

An apparatus for decoding a video sequence composed of a plurality of pictures, the apparatus comprises:
means for decoding, from a sequence parameter set of the bitstream, information on a chroma sampling format and high level syntax elements related to differential modulation for residual signals, wherein the high level syntax elements comprise a control flag indicating whether the differential modulation is permitted for the residual signals of the video sequence and information on a maximum block size for which the differential modulation is permitted;
means for decoding a first chroma syntax element indicating whether the differential modulation has been applied to a chroma block corresponding to a first block, when the first block to be currently decoded satisfies conditions defined by the information on the chroma sampling format and the high level syntax elements;
means for decoding a second chroma syntax element indicating a differential modulation direction applied to the chroma block in response to the first chroma syntax element indicating that the differential modulation has been applied to the chroma block;
means for determining a differential-modulated residual block for the chroma block from the bitstream;
means for determining quantized residual values by differential-demodulating the differential-modulated residual block in the differential modulation direction indicated by the second chroma syntax element;
means for inversely quantizing the quantized residual values for the chroma block;
means for generating predicted values for the chroma block; and
means for reconstructing the chroma block corresponding to the first block based on the inversely quantized residual values and the predicted values,
wherein the conditions defined by the information on the chroma sampling format and the high level syntax elements are satisfied when
(1) the control flag indicates that the differential modulation is permitted for the video sequence,
(2) a value obtained by dividing a width of the first block by a horizontal scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size, and
(3) a value obtained by dividing a height of the first block by a vertical scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the means for generating the predicted values for the chroma block
sets, as an intra prediction mode for the chroma block, a prediction mode having a direction identical with the differential modulation direction indicated by the second chroma syntax element among a vertical prediction mode and a horizontal prediction mode, and
generates the predicted values for the chroma block based on the intra prediction mode for the chroma block.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the means for generating the prediction values for the chroma block generates the predicted values for the chroma block by performing intra prediction on the chroma block by using samples not filtered from edge samples of a top or left block of the chroma block.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the information on the maximum block size for which the differential modulation is permitted is information on a maximum transform size for which a transform skip is permitted.

### Claim 14 (depends on 10)

The apparatus of claim 10, further comprising:
means for decoding a first luma syntax element indicating whether the differential modulation has been applied to a luma block corresponding to the first block, when the first block satisfies the condition defined by the high level syntax elements;
means for decoding a second luma syntax element indicating a differential modulation direction applied to the luma block, in response to the first luma syntax element indicating that the differential modulation has been applied to the luma block;
means for determining a differential-modulated residual block for the luma block from the bitstream;
means for determining quantized residual values by differential-demodulating the differential-modulated residual block for the luma block in a differential modulation direction indicated by the second luma syntax element;
means for inversely quantizing the quantized residual values for the luma block;
means for generating predicted values for the luma block; and
means for reconstructing the luma block corresponding to the first block based on the inversely quantized residual values and the predicted values.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the condition defined by the high level syntax elements is satisfied when
the control flag indicates that the differential modulation is permitted for the video sequence,
a width of the first block is smaller than or equal to the maximum block size, and
a height of the first block is smaller than or equal to the maximum block size.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein in determining an MPM list for a luma block corresponding to a second block neighboring the luma block corresponding to the first block, an intra prediction mode of the luma block corresponding to the first block is set as a prediction mode having a direction identical with the differential modulation direction indicated by the second luma syntax element among a vertical prediction mode and a horizontal prediction mode.

### Claim 17

A method for encoding a video sequence comprising_a plurality of pictures, the method comprising:
encoding, in a sequence parameter set of a bitstream, information on a chroma sampling format and high level syntax elements related to differential modulation for residual signals, wherein the high level syntax elements comprise a control flag indicating whether the differential modulation is permitted for the residual signals of the video sequence and information on a maximum block size for which the differential modulation is permitted;
encoding a first chroma syntax element indicating whether the differential modulation has been applied to a chroma block corresponding to a first block, when the first block to be currently decoded satisfies conditions defined by the information on the chroma sampling format and the high level syntax elements;
encoding a second chroma syntax element indicating a differential modulation direction applied to the chroma block in response to the first chroma syntax element indicating that the differential modulation has been applied to the chroma block;
generating predicted values for the chroma block;
determining a residual block for the chroma block based on original sample values and the predicted values of the chroma block;
generating quantized residual values by quantizing the residual block for the chroma block;
determining differential-modulated residual values by differential-modulating the quantized residual values in the differential modulation direction indicated by the second chroma syntax element; and
encoding the differential-modulated residual values in the bitstream, wherein the conditions defined by the information on the chroma sampling format and the high level syntax elements are satisfied when
(1) the control flag indicates that the differential modulation is permitted for the video sequence,
(2) a value obtained by dividing a width of the first block by a horizontal scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size, and
(3) a value obtained by dividing a height of the first block by a vertical scaling factor determined by the chroma sampling format is smaller than or equal to the maximum block size.

### Claim 18 (depends on 17)

The method of claim 17, wherein generating the predicted values for the chroma block comprises:
setting, as an intra prediction mode for the chroma block, a prediction mode having a direction identical with the differential modulation direction indicated by the second chroma syntax element among a vertical prediction mode and a horizontal prediction mode; and
generating the predicted values for the chroma block based on the intra prediction mode for the chroma block.

### Claim 19 (depends on 17)

The method of claim 17, wherein generating the predicted values for the chroma block comprises:
generating the predicted values for the chroma block by performing intra prediction on the chroma block by using samples not filtered from edge samples of a top or left block of the chroma block.

### Claim 20 (depends on 17)

The method of claim 17, wherein the information on the maximum block size for which the differential modulation is permitted is information on a maximum transform size for which a transform skip is permitted.
