Method and device for processing a video signal by using an adaptive separable graph-based transform

A method for encoding a video signal using an adaptive separable graph-based transform (GBT), the method comprising:

• partitioning a target block into a plurality of subblocks in a horizontal or vertical direction;
• generating a 2-dimensional (2D) separable GBT kernel for each of the plurality of subblocks based on a combination of at least two 1-dimensional (1D) GBTs;
• performing a transform on the plurality of subblocks using the 2D separable GBT kernel, and
• encoding a transform index corresponding to the 2D separable GBT kernel,
• wherein when the target block is comprised of the plurality of subblocks partitioned in a horizontal or vertical direction, the transform index corresponds to each of the subblocks,
• wherein the 1D GBTs are generated for each line graph corresponding to a row direction and a column direction of each of the subblocks,
• wherein transform types of the 1D GBTs are determined based on graph types of the line graphs, edge weights of the line graphs, and self-loop weights of the line graphs,
• wherein a direction of energy concentration in the subblocks is determined by a combination of the transform types for each of the row direction and column direction, and
• wherein the edge weights of the line graphs are given based on prediction direction of the target block.

A device for decoding a video signal using an adaptive separable graph-based transform (GBT), the device comprising:

• a processor configured to:
• receive, from the video signal, a transform index for a target block;
• and
• derive a 2-dimensional (2D) separable GBT kernel corresponding to the transform index and decode the target block based on the 2D separable GBT kernel,
• wherein when the target block is comprised of a plurality of subblocks partitioned in a horizontal or vertical direction, the transform index corresponds to each of the subblocks,
• wherein when the target block is comprised of a plurality of subblocks partitioned in a horizontal or vertical direction, the transform index corresponds to each of the subblocks,
• wherein the 2D separable GBT kernel is generated based on a combination of at least two 1-dimensional (1D) GBTs,
• wherein the 1D GBTs are generated for each line graph corresponding to a row direction and a column direction of each of the subblocks,
• wherein transform types of the 1D GBTs are determined based on graph types of the line graphs, edge weights of the line graphs, and self-loop weights of the line graphs,
• wherein a direction of energy concentration in the subblocks is determined by a combination of the transform types for each of the row direction and column direction, and
• wherein the edge weights of the line graphs are given based on prediction direction of the target block.

A device for encoding a video signal using an adaptive separable graph-based transform (GBT), the device comprising:

• a processor configured to:
• partition a target block into a plurality of subblocks in a horizontal or vertical direction;
• generate a 2-dimensional (2D) separable GBT kernel for each of the plurality of subblocks based on a combination of at least two 1-dimensional (1D) GBTs;
• and
• perform a transform on the plurality of subblocks using 2D separable GBT kernel,
• wherein when the target block is comprised of the plurality of subblocks partitioned in a horizontal or vertical direction, a transform index corresponds to each of the subblocks,
• wherein the 1D GBTs are generated for each line graph corresponding to a row direction and a column direction of each of the subblocks,
• wherein transform types of the 1D GBTs are determined based on graph types of the line graphs, edge weights of the line graphs, and self-loop weights of the line graphs,
• wherein a direction of energy concentration in the subblocks is determined by a combination of the transform types for each of the row direction and column direction, and
• wherein the edge weights of the line graphs are given based on prediction direction of the target block.