Methods and apparatuses of video processing for bi-directional prediction with motion refinement in video coding systems

An apparatus of processing video data by bi-directional prediction in a video coding system, the apparatus comprising one or more electronic circuits configured for:

• receiving input data associated with a current block coded or to be coded by bi-directional prediction in a current picture;
• determining a List 0 reference block for the current block by performing motion compensation from a reference picture in List 0 and determining a List 1 reference block for the current block by performing motion compensation from a reference picture in List 1;
• performing a motion vector refinement using a motion estimation search to refine the List 0 and List 1 reference blocks by finding a location corresponding to a minimum motion estimation cost around the List 0 reference block and a location corresponding to a minimum motion estimation cost around the List 1 reference block, wherein cost calculation for finding the minimum most estimation costs in the motion vector refinement is performed on each processing unit in the current block;
• determining whether a Bi-Directional Optical Flow (BDOF) operation is early terminated according to a cost of the refined List 0 and List 1 reference blocks of each processing unit in the current block, wherein the processing unit for BDOF early termination is aligned with the processing unit for the motion vector refinement;
• skipping the BDOF operation if the BDOF operation is early terminated and deriving a final inter predictor for the current block according to the refined List 0 and List 1 reference blocks, or performing the BDOF operation by deriving BDOF parameters to further derive the final inter predictor for the current block according to the refined List 0 and List 1 reference blocks if the BDOF operation is not early terminated, wherein the BDOF operation derives sample-based refinement through optical flow;
• and
• encoding or decoding the current block according to the final inter predictor of the current block.

A non-transitory computer readable medium storing program instruction causing a processing circuit of an apparatus to perform a bi-directional prediction processing method for video data, and the method comprising:

• receiving input data associated with a current block coded or to be coded by bi-directional prediction in a current picture;
• determining a List 0 reference block for the current block by performing motion compensation from a reference picture in List 0 and determining a List 1 reference block for the current block by performing motion compensation from a reference picture in List 1;
• performing a motion vector refinement using a motion estimation search to refine the List 0 and List 1 reference blocks by finding a location corresponding to a minimum motion estimation cost around the List 0 reference block and a location corresponding to a minimum motion estimation cost around the List 1 reference block, wherein cost calculation for finding the minimum most estimation costs in the motion vector refinement is performed on each processing unit in the current block;
• determining whether a Bi-Directional Optical Flow (BDOF) operation is early terminated according to a cost of the refined List 0 and List 1 reference blocks of each processing unit in the current block, wherein the processing unit for BDOF early termination is aligned with the processing unit for the motion vector refinement;
• skipping the BDOF operation if the BDOF operation is early terminated and deriving a final inter predictor for the current block according to the refined List 0 and List 1 reference blocks, or performing the BDOF operation by deriving BDOF parameters to further derive the final inter predictor for the current block according to the refined List 0 and List 1 reference blocks if the BDOF operation is not early terminated, wherein the BDOF operation derives sample-based refinement through optical flow;
• and
• encoding or decoding the current block according to the final inter predictor of the current block.