US 11,146,814 B2 · Skipping refinement based on patch similarity in bilinear interpolation based decoder-side motion vector refinement

Abstract

Methods and apparatuses of determining an alignment level between motion compensated reference patches for reducing motion vector refinement steps are provided. According to one method, obtaining, by a decoder, motion compensated interpolated samples based on sub-pixel accurate merge motion vectors from a bilinear motion compensated interpolation; computing, by the decoder, a sum of absolute differences (SAD) between two motion compensated reference patches using a subset of the motion compensated interpolated samples; determining, by the decoder, whether the SAD is less than a coding unit (CU) size-dependent threshold value; when the SAD is less than the CU size-dependent threshold value: skipping remaining decoder-side motion vector refinement (DMVR) process steps; and performing final motion compensation; and when the SAD is not less than the CU size-dependent threshold value: performing the remaining DMVR process steps; and performing the final motion compensation.

Claims20

Dependent claims

A method of motion compensation for a block of a frame in a video with a decoder-side motion vector refinement (DMVR) process comprising:

obtaining, by a decoder, motion compensated interpolated samples in two reference frames of the block based on sub-pixel accurate merge motion vectors of the block for the two reference frames via a bilinear motion compensated interpolation;
computing, by the decoder, a sum of absolute differences (SAD) between two motion compensated reference patches using a subset of the motion compensated interpolated samples;
determining, by the decoder, whether the SAD is less than a coding unit (CU) size-dependent threshold value, the CU threshold value depending on a size of the CU;
when the SAD is less than the CU size-dependent threshold value:
skipping remaining steps of the DMVR process for the block;
and
performing final motion compensation using the sub-pixel accurate merge motion vectors of the block to obtain reconstructed samples for the block;
and
when the SAD is not less than the CU size-dependent threshold value:
performing the remaining steps of the DMVR process to determine refined motion vectors for the block;
and
performing the final motion compensation using the refined motion vectors to obtain the reconstructed samples for the block.

9 more elements

A decoder, performing motion compensation for a block of a frame in a video with a decoder-side motion vector refinement (DMVR) process, the decoder comprising:

one or more processors;
and
a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the processors, the programming, when executed by the processors, configuring the decoder to:
obtain motion compensated interpolated samples in two reference frames of the block based on sub-pixel accurate merge motion vectors of the block for the two reference frames via a bilinear motion compensated interpolation;
compute a sum of absolute differences (SAD) between two motion compensated reference patches using a subset of the motion compensated interpolated samples;
determine whether the SAD is less than a coding unit (CU) size-dependent threshold value, the CU threshold value depending on a size of the CU;
when the SAD is less than the CU size-dependent threshold value:
skip remaining steps of the DMVR process for the block;
and
perform final motion compensation using the sub-pixel accurate merge motion vectors of the block to obtain reconstructed samples for the block;
and
when the SAD is not less than the CU size-dependent threshold value:
perform the remaining steps of the DMVR process to determine refined motion vectors for the block;
and
perform the final motion compensation using the refined motion vectors to obtain the reconstructed samples for the block.

12 more elements

A non-transitory computer-readable storage medium comprising instructions which, when the instructions are executed by a computer, cause the computer to carry out motion compensation for a block of a frame in a video with a decoder-side motion vector refinement (DMVR) process comprising:

obtaining motion compensated interpolated samples in two reference frames of the block based on sub-pixel accurate merge motion vectors of the block for the two reference frames via a bilinear motion compensated interpolation;
computing a sum of absolute differences (SAD) between two motion compensated reference patches using a subset of the motion compensated interpolated samples;
determining whether the SAD is less than a coding unit (CU) size-dependent threshold value, the CU threshold value depending on a size of the CU;
when the SAD is less than the CU size-dependent threshold value:
skipping remaining steps of the DMVR process for the block;
and
performing final motion compensation using the sub-pixel accurate merge motion vectors of the block to obtain reconstructed samples for the block;
and
when the SAD is not less than the CU size-dependent threshold value:
performing the remaining steps of the DMVR process to determine refined motion vectors for the block;
and
performing the final motion compensation using the refined motion vectors to obtain the reconstructed samples for the block.

9 more elements

Dependencies are inferred from the claim text.

Skipping refinement based on patch similarity in bilinear interpolation based decoder-side motion vector refinement

Life of this patent

Skipping refinement based on patch similarity in bilinear interpolation based decoder-side motion vector refinement

Life of this patent

Abstract

Claims20