Inter prediction with refinement in video processing

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

• determine two corresponding regions for a sub-block of a current video block of a current picture of a video, wherein the two corresponding regions are in two reference pictures of the current video block, respectively;
• derive a sum of absolute difference (SAD) based on partial samples of the two corresponding regions;
• determine that a first prediction mode is applied to the sub-block based on the SAD, wherein, in the first prediction mode, prediction samples of the sub-block are refined based on at least gradient values in different directions;
• and
• perform, based on the first prediction mode, a conversion between the sub-block and a bitstream of the video;
• wherein, in the first prediction mode, τ0×τ1>0, wherein τ0 is a difference between a picture order count of the current picture and a picture order count of a first reference picture (R0) of the two reference pictures (τ0=POC(current)−POC(R0)), and wherein τ1 is a difference between a picture order count of a second reference picture (R1) of the two reference pictures and the picture order count of the current picture (τ1=POC(R1)−POC(current)).

A non-transitory computer-readable storage medium storing instructions that cause a processor to:

• determine two corresponding regions for a sub-block of a current video block of a current picture of a video, wherein the two corresponding regions are in two reference pictures of the current video block, respectively;
• derive a sum of absolute difference (SAD) based on partial samples of the two corresponding regions;
• determine that a first prediction mode is applied to the sub-block based on the SAD, wherein, in the first prediction mode, prediction samples of the sub-block are refined based on at least gradient values in different directions;
• and
• perform, based on the first prediction mode, a conversion between the sub-block and a bitstream of the video;
• wherein, in the first prediction mode, τ0×τ1>0, wherein τ0 is a difference between a picture order count of the current picture and a picture order count of a first reference picture (R0) of the two reference pictures (τ0=POC(current)−POC(R0)), and wherein τ1 is a difference between a picture order count of a second reference picture (R1) of the two reference pictures and the picture order count of the current picture (τ1=POC(R1)−POC(current)).

A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, the method comprising:

• determining two corresponding regions for a sub-block of a current video block of a current picture of a video, wherein the two corresponding regions are in two reference pictures of the current video block, respectively;
• deriving a sum of absolute difference (SAD) based on partial samples of the two corresponding regions;
• determining that a first prediction mode is applied to the sub-block based on the SAD, wherein, in the first prediction mode, prediction samples of the sub-block are refined based on at least gradient values in different directions;
• and
• generating, based on the first prediction mode, the bitstream;
• wherein, in the first prediction mode, τ0×τ1>0, wherein τ0 is a difference between a picture order count of the current picture and a picture order count of a first reference picture (R0) of the two reference pictures (τ0=POC(current)−POC(R0)), and wherein τ1 is a difference between a picture order count of a second reference picture (R1) of the two reference pictures and the picture order count of the current picture (τ1=POC(R1)−POC(current)).