Non-separable secondary transform for video coding

A method for encoding video data, the method comprising:

• forming a residual video block, wherein forming the residual video block comprises subtracting one or more predictive blocks from a video block being coded;
• applying a first transform to the residual video block to generate a first coefficient block, the first transform converting the residual video block from a pixel domain to a frequency domain;
• formulating a subset of a set of non-separable transforms, wherein formulating the subset comprises determining the subset based on one or more of an intra prediction mode, a Coding Unit (CU) level Enhanced Multiple Transforms (EMT) index, a Transform Unit (TU) level EMT index, a residual quadtree depth, quantized coefficients, a reference prediction block, and a relative location of a current TU inside a prediction unit (PU);
• selecting one candidate from the subset of the set of non-separable transforms as a second transform;
• applying the second transform to at least part of the first coefficient block to generate a second coefficient block, the second transform being a non-separable transform;
• and
• quantizing the second coefficient block.

An apparatus for decoding video data comprising:

• a memory configured to store the video data;
• and
• a video decoder configured to:
• inverse quantize a first coefficient block;
• formulate a subset of a set of non-separable transforms, wherein to formulate the subset, the video decoder is configured to determine the subset based on decoded information, the decoded information comprising one or more of an intra prediction mode, a Coding Unit (CU) level Enhanced Multiple Transforms (EMT) index, a Transform Unit (TU) level EMT index, a residual quadtree depth, quantized coefficients, a reference prediction block, and a relative location of a current TU inside a prediction unit (PU);
• select one candidate from the subset of the set of non-separable transforms as a first inverse transform;
• apply the first inverse transform to at least part of the inverse quantized first coefficient block to generate a second coefficient block, the first inverse transform being a non-separable transform;
• apply a second inverse transform to the second coefficient block to generate a residual video block, the second inverse transform converting the second coefficient block from a frequency domain to a pixel domain;
• sum the residual video block with one or more predictive blocks;
• and
• form a decoded video block based on the sum of the residual video block with one or more predictive blocks.

An apparatus for encoding video data comprising:

• a memory configured to store the video data;
• and
• a video encoder configured to:
• form a residual video block, wherein forming the residual video block comprises subtracting one or more predictive blocks from a video block being coded;
• apply a first transform to the residual video block to generate a first coefficient block, the first transform converting the residual video block from a pixel domain to a frequency domain;
• formulate a subset of a set of non-separable transforms, wherein, to formulate the subset, the video encoder is configured to determine the subset based on one or more of an intra prediction mode, a Coding Unit (CU) level Enhanced Multiple Transforms (EMT) index, a Transform Unit (TU) level EMT index, a residual quadtree depth, quantized coefficients, a reference prediction block, and a relative location of a current TU inside a prediction unit (PU);
• select one candidate from the subset of the set of non-separable transforms as a second transform;
• apply the second transform to at least part of the first coefficient block to generate a second coefficient block, the second transform being a non-separable transform;
• and
• quantize the second coefficient block.

An apparatus for decoding video data comprising:

• means for inverse quantizing a first coefficient block;
• means for formulating a subset of a set of non-separable transforms, wherein the means for formulating the subset comprises means for determining the subset based on decoded information, the decoded information comprising one or more of an intra prediction mode, a Coding Unit (CU) level Enhanced Multiple Transforms (EMT) index, a Transform Unit (TU) level EMT index, a residual quadtree depth, quantized coefficients, a reference prediction block, and a relative location of a current TU inside a prediction unit (PU);
• means for selecting one candidate from the subset of the set of non-separable transforms as a first inverse transform;
• means for applying the first inverse transform to at least part of the inverse quantized first coefficient block to generate a second coefficient block, the first inverse transform being a non-separable transform;
• means for applying a second inverse transform to the second coefficient block to generate a residual video block, the second inverse transform converting the second coefficient block from a frequency domain to a pixel domain;
• and
• means for forming a decoded video block, wherein the means for forming the decoded video block comprises means for summing the residual video block with one or more predictive blocks.

A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device for decoding video data to:

• inverse quantize a first coefficient block;
• formulate a subset of a set of non-separable transforms, wherein to formulate the subset, the instructions further cause the one or more processors to determine the subset based on decoded information, the decoded information comprising one or more of an intra prediction mode, a Coding Unit (CU) level Enhanced Multiple Transforms (EMT) index, a Transform Unit (TU) level EMT index, a residual quadtree depth, quantized coefficients, a reference prediction block, and a relative location of a current TU inside a prediction unit (PU);
• select one candidate from the subset of the set of non-separable transforms as a first inverse transform;
• apply the first inverse transform to at least part of the inverse quantized first coefficient block to generate a second coefficient block, the first inverse transform being a non-separable transform;
• apply a second inverse transform to the second coefficient block to generate a residual video block, the second inverse transform converting the second coefficient block from a frequency domain to a pixel domain;
• sum the residual video block with one or more predictive blocks;
• and
• form a decoded video block based on the sum of the residual video block with one or more predictive blocks.