Method and apparatus for selecting a coding mode used for encoding/decoding a residual block

A device comprising at least one processor configured to:

• obtain a first coding mode indicating a first 2D transform for coding a residual block when the first coding mode is enabled, wherein the first 2D transform is from a first set of 2D transforms;
• obtain a second coding mode indicating a second 2D transform for coding the residual block, when the second coding mode is enabled, wherein the second 2D transform is from a second set of 2D transforms;
• and
• encode the residual block according to the first coding mode, the second coding mode, or both the first coding mode and the second coding mode;
• wherein the coding of the residual block according to the second coding mode is enabled when a skip mode is disabled and when coding the residual block according to the first coding mode is disabled;
• and
• wherein one of the first set of 2D transforms and the second set of 2D transforms is a set of separable 2D transforms and the other of the first set of 2D transforms and the second set of 2D transforms is a set of non-separable 2D transforms.

A device comprising at least one processor configured to:

• decode a residual block according to a first coding mode, a second coding mode, or both the first coding mode and the second coding mode;
• enable, responsive to determining that a skip mode is disabled and that the first coding mode is disabled, decoding of the residual block according to the second coding mode;
• wherein the first coding mode indicates a first 2D transform for coding the residual block when the first coding mode is enabled, wherein the first 2D transform is from a first set of 2D transforms;
• wherein the second coding mode indicates a second 2D transform for coding the residual block when the second coding mode is enabled, wherein the second 2D transform is from a second set of 2D transforms;
• and
• wherein one of the first set of 2D transforms and the second set of 2D transforms is a set of separable 2D transforms and the other of the first set of 2D transforms and the second set of 2D transforms is a set of non-separable 2D transforms.

A method comprising:

• decoding a residual block according to a first coding mode, a second coding mode, or both the first coding mode and the second coding mode;
• enabling, responsive to determining that a skip mode is disabled and that the first coding mode is disabled, decoding the residual block according to the second coding mode;
• wherein the first coding mode indicates a first 2D transform for coding the residual block when the first coding mode is enabled, wherein the first 2D transform is from a first set of 2D transforms;
• wherein the second coding mode indicates a second 2D transform for coding the residual block when the second coding mode is enabled, wherein the second 2D transform is from a second set of 2D transforms;
• and
• wherein one of the first set of 2D transforms and the second set of 2D transforms is a set of separable 2D transforms and the other of the first set of 2D transforms and the second set of 2D transforms is a set of non-separable 2D transforms.

A non-transitory storage computer-readable storage medium comprising instructions that cause a processor to:

• decode a residual block according to a first coding mode, a second coding mode, or both the first coding mode and the second coding mode;
• enable, responsive to determining that a skip mode is disabled and that the first coding mode is disabled, decoding of the residual block according to the second coding mode;
• wherein the first coding mode indicates a first 2D transform for coding the residual block when the first coding mode is enabled, wherein the first 2D transform is from a first set of 2D transforms;
• wherein the second coding mode indicates a second 2D transform for coding the residual block when the second coding mode is enabled, wherein the second 2D transform is from a second set of 2D transforms;
• and
• wherein one of the first set of 2D transforms and the second set of 2D transforms is a set of separable 2D transforms and the other of the first set of 2D transforms and the second set of 2D transforms is a set of non-separable 2D transforms.