Intra-mode dependent multiple transform selection for video coding

A device for decoding video data, the device comprising:

• a memory configured to store video data;
• and
• one or more processors implemented in circuitry and configured to:
• determine a size of a current block of video data;
• determine an intra-prediction mode for the current block of video data, wherein the determined intra-prediction mode comprises a matrix intra-prediction (MIP) mode having a transpose flag value;
• determine a mode group including the determined intra-prediction mode, the mode group being one of a plurality of mode groups, each of the mode groups in the plurality of mode groups including respective sets of intra-prediction modes such that each possible intra-prediction mode is included in no more than one of the mode groups;
• determine a set of available multiple transform selection (MTS) schemes for the current block according to the size and the intra-prediction mode for the current block, the set of available MTS schemes being one set of available MTS schemes of a plurality of sets of MTS schemes, each of the sets of MTS schemes of the plurality of sets of MTS schemes including a common number of MTS schemes, the common number being greater than one;
• determine an MTS scheme from the set of available MTS schemes according to the determined mode group, wherein each of the MTS schemes of the plurality of sets of MTS schemes includes a respective transform pair including a respective horizontal transform and a respective vertical transform, wherein the current block has a size of W×H, wherein W is not equal to H;
• apply transforms of the MTS scheme to a transform block of the current block to produce a residual block for the current block, wherein to apply the transforms, the processors are further configured to:
• based on the transpose flag value being a first value:
• apply the respective horizontal transform of the MTS scheme and the respective vertical transform of the MTS scheme to the transform block;
• or
• based on the transpose flag value being a second value, different than the first value:
• transpose the respective horizontal transform of the MTS scheme to form a transposed vertical transform;
• transpose the respective vertical transform of the MTS scheme to form a transposed horizontal transform;
• and
• apply the transposed horizontal transform and the transposed vertical transform to the transform block;
• and
• decode the current block using the residual block.

A non-transitory computer-readable medium having stored thereon instructions that when executed cause one or more processors to:

• determine a size of a current block of video data;
• determine an intra-prediction mode for the current block of video data wherein the determined intra-prediction mode comprises a matrix intra-prediction (MIP) mode having a transpose flag value;
• determine a mode group including the determined intra-prediction mode, the mode group being one of a plurality of mode groups, each of the mode groups in the plurality of mode groups including respective sets of intra-prediction modes such that each possible intra-prediction mode is included in no more than one of the mode groups;
• determine a set of available multiple transform selection (MTS) schemes for the current block according to the size and the intra-prediction mode for the current block, the set of available MTS schemes being one set of available MTS schemes of a plurality of sets of MTS schemes, each of the sets of MTS schemes of the plurality of sets of MTS schemes including a common number of MTS schemes, the common number being greater than one;
• determine an MTS scheme from the set of available MTS schemes according to the determined mode group, wherein each of the MTS schemes of the plurality of sets of MTS schemes includes a respective transform pair including a respective horizontal transform and a respective vertical transform, wherein the current block has a size of W×H, wherein W is not equal to H;
• apply transforms of the MTS scheme to a transform block of the current block to produce a residual block for the current block, wherein to apply the transforms, the instructions further cause the processors to:
• based on the transpose flag value being a first value:
• apply the respective horizontal transform of the MTS scheme and the respective vertical transform of the MTS scheme to the transform block;
• or based on the transpose flag value being a second value, different than the first value:
• transpose the respective horizontal transform of the MTS scheme to form a transposed vertical transform;
• transpose the respective vertical transform of the MTS scheme to form a transposed horizontal transform;
• and
• apply the transposed horizontal transform and the transposed vertical transform to the transform block;
• and
• decode the current block using the residual block.