Video coding method based on secondary transform, and device therefor

An encoding apparatus for image encoding, the encoding apparatus comprising:

• a memory;
• and
• at least one processor connected to the memory, the at least one processor configured to:
• derive prediction samples based on an intra prediction mode applied to a target block;
• derive residual samples for the target block based on the prediction samples;
• derive transform coefficients for the target block based on a primary transform for the residual samples;
• derive modified transform coefficients based on a non-separable transform using a transform kernel matrix for the transform coefficients;
• derive quantized transform coefficients by performing quantization based on the modified transform coefficients;
• and
• generate flag information related to whether or not a transform index is present and the transform index, wherein the transform index is related to whether the non-separable transform is applied to the target block and which one of transform kernel matrices is applied to the target block,
• wherein the flag information is signaled in a sequence parameter set, and the transform index is present in a coding unit syntax,
• wherein based on the non-separable transform being applied to 8×8 area of the target block, 16 modified transform coefficients are derived by performing a matrix operation with 48 transform coefficients in the target block,
• wherein based on the intra prediction mode for the target block being less than or equal to mode 34, the 48 transform coefficients for the matrix operation are ordered in a horizontal scan order from an area excluding a bottom-right 4×4 area in the 8×8 area of the target block,
• wherein based on the intra prediction mode for the target block being more than the mode 34, the 48 transform coefficients are ordered in a vertical scan order in the area excluding the bottom-right 4×4 area from the 8×8 area of the target block, and
• wherein the 16 modified transform coefficients are arranged in a diagonal scan order in a top-left 4×4 area of the 8×8 area of the target block.

An apparatus for transmitting data for an image, the apparatus comprising:

• at least one processor configured to obtain a bitstream for the image, wherein the bitstream is generated based on deriving prediction samples based on an intra prediction mode applied to a target block;
• deriving residual samples for the target block based on the prediction samples, deriving transform coefficients for the target block based on a primary transform for the residual samples, deriving modified transform coefficients based on a non-separable transform using a transform kernel matrix for the transform coefficients, deriving quantized transform coefficients by performing quantization based on the modified transform coefficients, and generating flag information related to whether or not a transform index is present, and the transform index, wherein the transform index is related to whether the non-separable transform is applied to the target block and which one of transform kernel matrices is applied to the target block;
• and
• a transmitter configured to transmit the data comprising the bitstream,
• wherein the flag information is signaled in a sequence parameter set, and the transform index is present in a coding unit syntax,
• wherein based on the non-separable transform being applied to 8×8 area of the target block, 16 modified transform coefficients are derived by performing a matrix operation with 48 transform coefficients in the target block,
• wherein based on the intra prediction mode for the target block being less than or equal to mode 34, the 48 transform coefficients for the matrix operation are ordered in a horizontal scan order from an area excluding a bottom-right 4×4 area in the 8×8 area of the target block,
• wherein based on the intra prediction mode for the target block being more than the mode 34, the 48 transform coefficients are ordered in a vertical scan order in the area excluding the bottom-right 4×4 area from the 8×8 area of the target block, and
• wherein the 16 modified transform coefficients are arranged in a diagonal scan order in a top-left 4×4 area of the 8×8 area of the target block.