Method and apparatus for performing non-separable transform based on modified intra prediction mode

A method for encoding an image signal by an apparatus, comprising:

• determining a modified intra prediction mode of a current block having a different width and height from an intra prediction mode, based on a ratio between the width and height of the current block and the intra prediction mode;
• determining a non-separable transform set based on the modified intra prediction mode;
• and
• applying a non-separable transform matrix selected from the non-separable transform set, to a top-left region of the current block determined based on the width and height of the current block,
• wherein determining the non-separable transform set comprises determining an index of the non-separable transform set related to the modified intra prediction mode,
• wherein based on that the modified intra prediction mode is greater than or equal to 0 and smaller than or equal to 1, the index is determined as a first index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 2 and smaller than or equal to 12, the index is determined as a second index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 13 and smaller than or equal to 23, the index is determined as a third index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 24 and smaller than or equal to 44, the index is determined as a fourth index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 45 and smaller than or equal to 55, the index is determined as the third index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 56, the index is determined as the second index value, and
• wherein based on that the modified intra prediction mode is smaller than 0, the index is determined as the second index value.

A non-transitory computer-readable medium storing a bitstream causing an image decoding apparatus to perform an image decoding method, the image decoding method comprising:

• determining a modified intra prediction mode of a current block having a different width and height from an intra prediction mode, based on a ratio between the width and height of the current block and the intra prediction mode;
• determining an inverse non-separable transform set based on the modified intra prediction mode;
• and
• applying an inverse non-separable transform matrix selected from the inverse non-separable transform set, to a top-left region of the current block determined based on the width and height of the current block,
• wherein determining the inverse non-separable transform set comprises determining an index of the inverse non-separable transform set related to the modified intra prediction mode,
• wherein based on that the modified intra prediction mode is greater than or equal to 0 and smaller than or equal to 1, the index is determined as a first index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 2 and smaller than or equal to 12, the index is determined as a second index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 13 and smaller than or equal to 23, the index is determined as a third index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 24 and smaller than or equal to 44, the index is determined as a fourth index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 45 and smaller than or equal to 55, the index is determined as the third index value,
• wherein based on that the modified intra prediction mode is greater than or equal to 56, the index is determined as the second index value, and
• wherein based on that the modified intra prediction mode is smaller than 0, the index is determined as the second index value.