Memory-bandwidth-efficient design for bi-directional optical flow (BIO)

A device for decoding video data, the device comprising:

• a memory configured to store the video data;
• and
• one or more processors configured to:
• determine that a first block of video data is encoded using an inter prediction mode;
• perform interpolation filtering using an N-tap filter to generate an interpolated search space, wherein N is an integer and corresponds to a number of taps in the N-tap filter;
• obtain a first predictive block for the first block of video data in the interpolated search space;
• determine that a second block of video data is encoded using a bi-directional inter prediction mode;
• determine that the second block of video data is encoded using a bi-directional optical flow (BIO) process;
• perform an inter prediction process for the second block of video data using the bi-directional inter prediction mode to determine a second predictive block;
• perform the BIO process on the second predictive block to determine a BIO-refined version of the second predictive block, wherein a number of reference samples used for calculating intermediate values for BIO offsets is limited to a region of (W+N−1)×(H+N−1) integer samples, wherein W corresponds to a width of the second block in integer samples, and H corresponds to a height of the second block in integer samples, wherein to perform the BIO process for the second block to determine the BIO-refined version of the second predictive block, the one or more processors are further configured to:
• fetch a block of reference samples corresponding to the region of (W+N−1)×(H+N−1) integer samples;
• apply sample repetition at boundaries of the block of reference samples corresponding to the region of (W+N−1)×(H+N−1) to generate an extended reference block with a size of (W+N−1+2S)×(H+N−1+2S), wherein S corresponds to a positive integer value;
• use sample values in the extended reference block, determining one or more BIO offsets;
• and
• add the one or more BIO offsets to the second predictive block to determine the BIO-refined version of the second predictive block;
• and
• output the BIO-refined version of the second predictive block.

A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to:

• determine that a first block of video data is encoded using an inter prediction mode;
• perform interpolation filtering using an N-tap filter to generate an interpolated search space, wherein N is an integer and corresponds to a number of taps in the N-tap filter;
• obtain a first predictive block for the first block of video data in the interpolated search space;
• determine that a second block of video data is encoded using a bi-directional inter prediction mode;
• determine that the second block of video data is encoded using a bi-directional optical flow (BIO) process;
• perform an inter prediction process for the second block of video data using the bi-directional inter prediction mode to determine a second predictive block;
• perform the BIO process on the second predictive block to determine a BIO-refined version of the second predictive block, wherein a number of reference samples used for calculating intermediate values for BIO offsets is limited to a region of (W+N−1)×(H+N−1) integer samples, wherein W corresponds to a width of the second block in integer samples, and H corresponds to a height of the second block in integer samples, wherein to perform the BIO process for the second block to determine the BIO-refined version of the second predictive block, the instructions cause the one or more processors to:
• fetch a block of reference samples corresponding to the region of (W+N−1)×(H+N−1) integer samples;
• apply sample repetition at boundaries of the block of reference samples corresponding to the region of (W+N−1)×(H+N−1) to generate an extended reference block with a size of (W+N−1+2S)×(H+N−1+2S), wherein S corresponds to a positive integer value;
• use sample values in the extended reference block, determining one or more BIO offsets;
• and
• add the one or more BIO offsets to the second predictive block to determine the BIO-refined version of the second predictive block;
• and
• output the BIO-refined version of the second predictive block.