Modification to block size for transform mode in display stream compression

A device for selecting a coding mode from a plurality of coding modes for coding video data, comprising:

• a memory configured to store video data including a previous block and a current block;
• and
• at least one processor circuit coupled to the memory and configured to:
• select a current block and a next block of the video data, wherein the current and next blocks each have a size of P×Q;
• determine whether to evaluate larger block size transform (LBT) mode coding performance based on one or more conditions of a plurality of conditions being satisfied for either of the current or next blocks, wherein the plurality of conditions comprises (i) pattern mode is determined to be an optimal coding mode, (ii) a flat block is detected, (iii) a transition from complex to flat region is detected, and (iv) a transition from flat to complex region is detected;
• in response to determining to evaluate LBT mode coding performance, calculate a first rate-distortion (RD) cost for coding samples in the current and next blocks together with a LBT mode, wherein the LBT mode operates on a LBT size of P×M, wherein M>Q;
• calculate second RD costs for coding each of the current and next blocks individually with a second coding mode different from the LBT mode, wherein to calculate the second RD costs, the processor circuit is configured to calculate, for each of the current and next blocks, a respective second RD cost for coding a respective block individually with the second coding mode;
• determine whether the first RD cost is less than a sum of the second RD costs for the current and next blocks;
• and
• code the samples in the current and next blocks with the LBT mode in response to the first RD cost being less than the sum of the second RD costs for the current and next blocks.

An apparatus, comprising:

• means for selecting a current block and a next block of the video data, wherein the current and next blocks each have a size of P×Q;
• means for determining whether to evaluate larger block size transform (LBT) mode coding performance based on one or more conditions of a plurality of conditions being satisfied for either of the current or next blocks, wherein the plurality of conditions comprises (i) pattern mode is determined to be an optimal coding mode, (ii) a flat block is detected, (iii) a transition from complex to flat region is detected, and (iv) a transition from flat to complex region is detected;
• means for calculating a first rate-distortion (RD) cost for coding samples in the current and next blocks together with a LBT mode in response to determining to evaluate LBT mode coding performance, wherein the LBT mode operates on a LBT size of P×M, wherein M>Q;
• means for calculating second RD costs for coding each of the current and next blocks individually with a second coding mode different from the LBT mode, wherein the means for calculating the second RD costs comprises means for calculating, for each of the current and next blocks, a respective second RD cost for coding a respective block individually with the second coding mode;
• means for determining whether the first RD cost is less than a sum of the second RD costs for the current and next blocks;
• and
• means for coding the samples in the current and next blocks with the LBT mode in response to the first RD cost being less than the sum of the second RD costs for the current and next blocks.

A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause a processor of a device to:

• select a current block and a next block from video data, wherein the current and next blocks each have a size of P×Q;
• determine whether to evaluate larger block size transform (LBT) mode coding performance based on one or more conditions of a plurality of conditions being satisfied for either of the current or next blocks, wherein the plurality of conditions comprises (i) pattern mode is determined to be an optimal coding mode, (ii) a flat block is detected, (iii) a transition from complex to flat region is detected, and (iv) a transition from flat to complex region is detected;
• in response to determining to evaluate LBT mode coding performance, calculate a first rate-distortion (RD) cost for coding samples in the current and next blocks together with a LBT mode, wherein the LBT mode operates on a LBT size of P×M, wherein M>Q;
• calculate second RD costs for coding each of the current and next blocks individually with a second coding mode different from the LBT mode, wherein the instructions to calculate the second RD costs comprises instructions to calculate, for each of the current and next blocks, a respective second RD cost for coding a respective block individually with the second coding mode;
• determine whether the first RD cost is less than a sum of the second RD costs for the current and next blocks;
• and
• code the samples in the current and next blocks with the LBT mode in response to the first RD cost being less than the sum of the second RD costs for the current and next blocks.