Interaction between matrix-based intra prediction and other coding tools

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

• determine, for a first conversion between a first video block of a video and a bitstream of the video, that a first intra mode is applied on the first video block;
• invoke a reference sample filtering operation to derive reference samples of the first video block;
• generate prediction samples for the first video block based on the reference samples;
• perform the first conversion based on the prediction samples of the first video block;
• determine, for a second conversion between a second video block of the video and the bitstream of the video, that a second intra mode is applied on the second video block of the video, wherein the second intra mode is different from the first intra mode;
• derive reference samples of the second video block without invoking the reference sample filtering operation;
• perform a boundary downsampling operation on the reference samples of the second video block based on a size of the second video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the second video block;
• and
• perform the second conversion based on the prediction samples of the second video block,
• wherein a first syntax element indicates usage of a third mode, and wherein due to the second intra mode being applied to the second video block, the first syntax element for the second video block is omitted from the bitstream, wherein in the third mode, a difference between a quantized residual and a prediction of the quantized residual is represented in the bitstream using a pulse coding modulation representation, and prediction samples in the third mode are derived by invoking the reference sample filtering operation.

A non-transitory computer-readable storage medium storing instructions that cause a processor to:

• determine, for a first conversion between a first video block of a video and a bitstream of the video, that a first intra mode is applied on the first video block;
• invoke a reference sample filtering operation to derive reference samples of the first video block;
• generate prediction samples for the first video block based on the reference samples;
• perform the first conversion based on the prediction samples of the first video block;
• determine, for a second conversion between a second video block of the video and the bitstream of the video, that a second intra mode is applied on the second video block of the video, wherein the second intra mode is different from the first intra mode;
• derive reference samples of the second video block without invoking the reference sample filtering operation;
• perform a boundary downsampling operation on the reference samples of the second video block based on a size of the second video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples tor the second video block;
• and
• perform the second conversion based on the prediction samples of the second video block,
• wherein a first syntax element indicates usage of a third mode, and wherein due to the second intra mode being applied to the second video block, the first syntax element for the second video block is omitted from the bitstream, wherein in the third mode, a difference between a quantized residual and a prediction of the quantized residual is represented in the bitstream using a pulse coding modulation representation, and prediction samples in the third mode are derived by invoking the reference sample filtering operation.

A method performed by a video processing apparatus, wherein the method comprises:

• determining that a first intra mode is applied on a first video block of the video;
• invoking a reference sample filtering operation to derive reference samples of the first video block;
• generating prediction samples for the first video block based on the reference samples;
• determining that a second intra mode is applied on a second video block of the video, wherein the second intra mode is different from the first intra mode;
• deriving reference samples of the second video block without invoking the reference sample filtering operation;
• performing a boundary downsampling operation on the reference samples of the second video block based on a size of the second video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the second video block;
• generating the bitstream based on the prediction samples of the first video block and the second video block, and
• storing the bitstream on a non-transitory computer readable recording medium;
• wherein a first syntax element indicates usage of a third mode, and wherein due to the second intra mode being applied to the second video block, the first syntax element for the second video block is omitted from the bitstream, wherein in the third mode, a difference between a quantized residual and a prediction of the quantized residual is represented in the bitstream using a pulse coding modulation representation, and prediction samples in the third mode are derived by invoking the reference sample filtering operation.