Low frequency non-separable transform for video coding

A device for decoding video data, the device comprising:

• memory;
• and
• processing circuitry coupled to the memory and configured to:
• determine a transform unit (TU) class from a plurality of TU classes based on a height and a width of a current block of the video data, wherein the plurality of TU classes includes more than two TU classes;
• determine an intra prediction mode from a plurality of intra prediction modes for the current block of the video data;
• determine an intra prediction mode cluster based on the intra prediction mode and the TU class;
• determine a low frequency non-separable transform (LFNST) kernel from a plurality of LFNST kernels for the current block based on the intra prediction mode cluster and the TU class, wherein each LFNST kernel of the plurality of LFNST kernels is assigned to at least one intra prediction mode cluster from a plurality of intra prediction mode clusters and at least one TU class from the plurality of TU classes;
• apply an inverse of the determined LFNST kernel to coefficient values generated from a transform unit (TU) of the current block to generate intermediate values;
• apply an inverse primary transform on the intermediate values to generate residual data;
• and
• reconstruct the current block based on the residual data.

A method of encoding video data, the method comprising:

• applying a primary transform on residual data indicative of a difference between a current block of the video data and a prediction block to generate intermediate coefficients;
• determining a transform unit (TU) class from a plurality of TU classes based on a height and a width of the current block of the video data, wherein the plurality of TU classes includes more than two TU classes;
• determining an intra prediction mode from a plurality of intra prediction modes for the current block;
• determining an intra prediction mode cluster based on the intra prediction mode and the TU class;
• determining a low frequency non-separable transform (LFNST) kernel from a plurality of LFNST kernels for the current block based on the intra prediction mode cluster and the TU class, wherein each LFNST kernel of the plurality of LFNST kernels is assigned to at least one intra prediction mode cluster from a plurality of intra prediction mode clusters and at least one TU class from the plurality of TU classes;
• applying the determined LFNST kernel to the intermediate coefficients to generate coefficients of a transform unit (TU) of the current block;
• and
• signaling information indicative of the coefficients of the TU.

A device for encoding video data, the device comprising:

• memory;
• and
• processing circuitry coupled to the memory and configured to:
• apply a primary transform on residual data indicative of a difference between a current block of the video data and a prediction block to generate intermediate coefficients;
• determine a transform unit (TU) class from a plurality of TU classes based on a height and a width of the current block of the video data, wherein the plurality of TU classes includes more than two TU classes;
• determine an intra prediction mode from a plurality of intra prediction modes for the current block;
• determine an intra prediction mode cluster based on the intra prediction mode and the TU class;
• determine a low frequency non-separable transform (LFNST) kernel from a plurality of LFNST kernels for the current block based on the intra prediction mode cluster and the TU class, wherein each LFNST kernel of the plurality of LFNST kernels is assigned to at least one intra prediction mode cluster from a plurality of intra prediction mode clusters and at least one TU class from the plurality of TU classes;
• apply the determined LFNST kernel to the intermediate coefficients to generate coefficients of a transform unit (TU) of the current block;
• and
• signal information indicative of the coefficients of the TU.