Multi-pass decoder-side motion vector refinement

A device for decoding video data, the device comprising:

• memory configured to store the video data;
• and
• one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to:
• apply a multi-pass decoder-side motion vector refinement (DMVR) to a motion vector for a block of the video data to determine at least one refined motion vector;
• and
• decode the block based on the at least one refined motion vector,
• wherein the multi-pass DMVR comprises:
• a first pass, the first pass being block-based and applied to the block of the video data, wherein applying the first pass comprises applying bi-lateral matching to the block and deriving at least one first refined motion vector for the block of the video data;
• a second pass, the second pass being sub-block-based and applied to at least one second pass sub-block of the block of the video data, wherein a second pass sub-block width is equal to or less than a width of the block of the video data, and a second pass sub-block height is equal to or less than a height of the block of the video data, wherein the second pass uses the at least one first refined motion vector as an initial motion vector for the second pass, and wherein applying the second pass comprises applying bi-lateral matching to the at least one second pass sub-block and deriving at least one second refined motion vector for at least one respective second pass sub-block;
• and
• a third pass, the third pass being sub-block-based and applied to at least one third pass sub-block of the block of the video data, the third pass sub-block being a sub-block of the at least one respective second pass sub-block, wherein a third pass sub-block width is equal to or less than the second pass sub-block width, and a third pass sub-block height is equal to or less than the second pass sub-block height, and wherein the third pass uses the at least one second refined motion vector as an initial motion vector for the at least one third pass sub-block, wherein applying the third pass comprises applying bi-directional optical flow (BDOF) to the at least one third pass sub-block and deriving at least one third refined motion vector for at least one respective third pass sub-block, and wherein the at least one refined motion vector is based on the at least one third refined motion vector.

A non-transitory computer-readable storage medium storing instructions, which, when executed, cause one or more processors to:

• apply a multi-pass decoder-side motion vector refinement (DMVR) to a motion vector for a block of video data to determine at least one refined motion vector;
• and
• decode the block based on the at least one refined motion vector,
• wherein the multi-pass DMVR comprises:
• a first pass, the first pass being block-based and applied to the block of the video data, wherein applying the first pass comprises applying bi-lateral matching to the block and deriving at least one first refined motion vector for the block of the video data;
• a second pass, the second pass being sub-block-based and applied to at least one second pass sub-block of the block of the video data, wherein a second pass sub-block width is equal to or less than a width of the block of the video data, and a second pass sub-block height is equal to or less than a height of the block of the video data, wherein the second pass uses the at least one first refined motion vector as an initial motion vector for the second pass, and wherein applying the second pass comprises applying bi-lateral matching to the at least one second pass sub-block and deriving at least one second refined motion vector for at least one respective second pass sub-block;
• and
• a third pass, the third pass being sub-block-based and applied to at least one third pass sub-block of the block of the video data, the third pass sub-block being a sub-block of the at least one respective second pass sub-block, wherein a third pass sub-block width is equal to or less than the second pass sub-block width, and a third pass sub-block height is equal to or less than the second pass sub-block height, and wherein the third pass uses the at least one second refined motion vector as an initial motion vector for the at least one third pass sub-block, wherein applying the third pass comprises applying bi-directional optical flow (BDOF) to the at least one third pass sub-block and deriving at least one third refined motion vector for at least one respective third pass sub-block, and wherein the at least one refined motion vector is based on the at least one third refined motion vector.

A device for coding video data, the device comprising:

• means for applying a multi-pass decoder-side motion vector refinement (DMVR) to a motion vector for a block of the video data to determine at least one refined motion vector;
• and
• means for decoding the block based on the at least one refined motion vector,
• wherein the multi-pass DMVR comprises:
• a first pass, the first pass being block-based and applied to the block of the video data, wherein applying the first pass comprises applying bi-lateral matching to the block and deriving at least one first refined motion vector for the block of the video data;
• a second pass, the second pass being sub-block-based and applied to at least one second pass sub-block of the block of the video data, wherein a second pass sub-block width is equal to or less than a width of the block of the video data, and a second pass sub-block height is equal to or less than a height of the block of the video data, wherein the second pass uses the at least one first refined motion vector as an initial motion vector for the second pass, and wherein applying the second pass comprises applying bi-lateral matching to the at least one second pass sub-block and deriving at least one second refined motion vector for at least one respective second pass sub-block;
• and
• a third pass, the third pass being sub-block-based and applied to at least one third pass sub-block of the block of the video data, the third pass sub-block being a sub-block of the at least one respective second pass sub-block, wherein a third pass sub-block width is equal to or less than the second pass sub-block width, and a third pass sub-block height is equal to or less than the second pass sub-block height, and wherein the third pass uses the at least one second refined motion vector as an initial motion vector for the at least one third pass sub-block, wherein applying the third pass comprises applying bi-directional optical flow (BDOF) to the at least one third pass sub-block and deriving at least one third refined motion vector for at least one respective third pass sub-block, and wherein the at least one refined motion vector is based on the at least one third refined motion vector.