US 2022/0191502 A1 · METHODS AND APPARATUS FOR PREDICTION REFINEMENT FOR DECODER SIDE MOTION VECTOR REFINEMENT WITH OPTICAL FLOW

Abstract

Methods, devices, apparatus, systems, architectures and interfaces to improve motion vector (MV) refinement based sub-block (SB) level motion compensated prediction are provided. A decoding method includes receiving a bitstream of encoded video data, the bitstream including at least one block of video data including a plurality of SBs; performing a MV derivation, including a decoder based MV (DMVR) process, for at least one SB in the block to generate a refined MV for each SB; performing SB based motion compensation on the at least one sub-block to generate a SB based prediction within each SB; obtaining a spatial gradient for the prediction within each SB; determining a MV offset for each pixel in each SB; obtaining an intensity change in each SB based on the spatial gradients and MV offsets via an optical flow equation; and refining the prediction within each SB based on the obtained intensity changes.

Claims20

Dependent claims

A method of decoding video data, the method comprising:

receiving a bit stream of encoded video data, the bitstream including at least one block of video data, the block including a plurality of sub-blocks;
performing a motion vector derivation, including a decoder based motion vector (DMVR) process, for at least one sub-block in the block to generate a refined motion vector for each sub-block;
performing sub-block based motion compensation on the at least one sub-block to generate a sub-block based prediction within each sub-block;
obtaining a spatial gradient for the prediction within each sub-block;
determining a motion vector offset for each pixel in each sub-block;
obtaining an intensity change in each sub-block based on the spatial gradients and motion vector offsets via an optical flow equation;
and
refining the prediction within each sub-block based on the obtained intensity changes.

5 more elements

A decoder of a device including any of a transmitter, a receiver, a processor, and a memory, the decoder configured to:

receive a bit stream of encoded video data, the bitstream including at least one block of video data, the block including a plurality of sub-blocks;
perform a motion vector derivation, including a decoder based motion vector (DMVR) process, for each sub-block in the block to generate a refined motion vector for each sub-block;
perform sub-block based motion compensation on each sub-block to generate a sub-block based prediction within each sub-block;
obtain a spatial gradient for each prediction within each sub-block;
determine a motion vector offset for each pixel in each sub-block;
obtain an intensity change in each sub-block based on the spatial gradients and motion vector offsets via an optical flow equation;
and
refine the prediction within each sub-block based on the obtained intensity changes.

5 more elements

Dependencies are inferred from the claim text.

Methods and apparatus for prediction refinement for decoder side motion vector refinement with optical flow

Life of this patent

Methods and apparatus for prediction refinement for decoder side motion vector refinement with optical flow

Life of this patent

Abstract

Claims20