Optical flow based video inter prediction

An encoding apparatus, comprising:

• one or more memories configured to store programming instructions;
• and
• at least one processor coupled to the one or more memories and configured to execute the instructions to cause the encoding apparatus to:
• perform bi-directional optical flow (BDOF) based inter prediction for a current block of a video signal, by:
• obtaining a pair of motion vectors for the current block with respect to a first reference frame and a second reference frame;
• obtaining prediction sample values corresponding to the first reference frame and prediction sample values corresponding to the second reference frame from the first reference frame and the second reference frame, respectively, using the pair of motion vectors for the current block with respect to the first reference frame and the second reference frame;
• determining a horizontal motion offset vx and a vertical motion offset vy of the current block, and wherein the vertical motion offset is determined based on the horizontal motion offset and a fifth variable s5, wherein:
• the fifth variable s5 indicates a sum of a plurality of first terms, wherein each of the plurality of first terms is obtained from a sign of an element of a second matrix and an element of a first matrix, and the element of the first matrix corresponds to the element of the second matrix;
• each element of the first matrix is obtained from a sum of a first horizontal predicted sample gradient that corresponds to a first reference frame of the current block and a second horizontal predicted sample gradient that corresponds to a second reference frame of the current block, wherein the first horizontal predicted sample gradient and the second horizontal predicted sample gradient correspond to the element of the first matrix;
• and
• each element of the second matrix is obtained from a sum of a first vertical predicted sample gradient that corresponds to the first reference frame of the current block and a second vertical predicted sample gradient that corresponds to the second reference frame of the current block, wherein the first vertical predicted sample gradient and the second vertical predicted sample gradient correspond to the element of the second matrix;
• and
• determining prediction sample values of the current block using the prediction sample values corresponding to the first reference frame, the prediction sample values corresponding to the second reference frame, and the horizontal motion offset and the vertical motion offset;
• and
• obtain residual information of the current block based on sample values of the current block and the prediction sample values of the current block;
• and
• encode information associated with the current block into a bitstream for transmission, wherein the information associated with the current block comprises the residual information of the current block and indication information, and wherein the indication information indicates the pair of motion vectors for the current block.

A computer-readable medium storing computer instructions that, when executed by one or more processors, cause an encoding apparatus to:

• perform bi-directional optical flow (BDOF) based inter prediction for a current block of a video signal, by:
• obtaining a pair of motion vectors for the current block with respect to a first reference frame and a second reference frame;
• obtaining prediction sample values corresponding to the first reference frame and prediction sample values corresponding to the second reference frame from the first reference frame and the second reference frame, respectively, using the pair of motion vectors for the current block with respect to the first reference frame and the second reference frame;
• determining a horizontal motion offset vx and a vertical motion offset vy of the current block, and wherein the vertical motion offset is determined based on the horizontal motion offset and a fifth variable s5, wherein:
• the fifth variable s5 indicates a sum of a plurality of first terms, wherein each of the plurality of first terms is obtained from a sign of an element of a second matrix and an element of a first matrix, and the element of the first matrix corresponds to the element of the second matrix;
• each element of the first matrix is obtained from a sum of a first horizontal predicted sample gradient that corresponds to a first reference frame of the current block and a second horizontal predicted sample gradient that corresponds to a second reference frame of the current block, wherein the first horizontal predicted sample gradient and the second horizontal predicted sample gradient correspond to the element of the first matrix;
• and
• each element of the second matrix is obtained from a sum of a first vertical predicted sample gradient that corresponds to the first reference frame of the current block and a second vertical predicted sample gradient that corresponds to the second reference frame of the current block, wherein the first vertical predicted sample gradient and the second vertical predicted sample gradient correspond to the element of the second matrix;
• and
• determining prediction sample values of the current block using the prediction sample values corresponding to the first reference frame, the prediction sample values corresponding to the second reference frame, and the horizontal motion offset and the vertical motion offset;
• and
• the instructions, when executed by the one or more processors, causes the encoding apparatus to further:
• obtain residual information of the current block based on sample values of the current block and the prediction sample values of the current block;
• and
• encode information associated with the current block into a bitstream for transmission, wherein the information associated with the current block comprises the residual information of the current block and indication information, and wherein the indication information indicates the pair of motion vectors for the current block.

A non-transitory computer-readable medium storing a bitstream and one or more instructions executable by at least one processor to perform operations of encoding or decoding of the bitstream,

• the operations comprising:
• performing bi-directional optical flow, BDOF, based inter prediction for a current block of a video, by:
• obtaining a pair of motion vectors for the current block with respect to a first reference frame and a second reference frame;
• obtaining prediction sample values corresponding to the first reference frame and prediction sample values corresponding to the second reference frame from the first reference frame and the second reference frame, respectively, using the pair of motion vectors for the current block with respect to the first reference frame and the second reference frame;
• determining a horizontal motion offset vx and a vertical motion offset vy of the current block, and wherein the vertical motion offset is determined based on the horizontal motion offset and a fifth variable s5, wherein:
• the fifth variable s5 indicates a sum of a plurality of first terms, wherein each of the plurality of first terms is obtained from a sign of an element of a second matrix and an element of a first matrix, and the element of the first matrix corresponds to the element of the second matrix;
• each element of the first matrix is obtained from a sum of a first horizontal predicted sample gradient that corresponds to a first reference frame of the current block and a second horizontal predicted sample gradient that corresponds to a second reference frame of the current block, wherein the first horizontal predicted sample gradient and the second horizontal predicted sample gradient correspond to the element of the first matrix;
• and
• each element of the second matrix is obtained from a sum of a first vertical predicted sample gradient that corresponds to the first reference frame of the current block and a second vertical predicted sample gradient that corresponds to the second reference frame of the current block, wherein the first vertical predicted sample gradient and the second vertical predicted sample gradient correspond to the element of the second matrix;
• and
• determining prediction sample values of the current block using the prediction sample values corresponding to the first reference frame, the prediction sample values corresponding to the second reference frame, and the horizontal motion offset and the vertical motion offset;
• obtaining residual information of the current block based on sample values of the current block and the prediction sample values of the current block;
• and
• encoding information associated with the current block into the bitstream, wherein the information associated with the current block comprises the residual information of the current block and indication information, and wherein the indication information indicates the pair of motion vectors for the current block.