Bidirectional optical flow based video coding and decoding

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

• determine, for a first block of a video, a first prediction sample;
• refine the first prediction sample, based on a first optical flow refinement procedure, with a first prediction sample offset by adding the first prediction sample offset to the first prediction sample to acquire a first final prediction sample, wherein the first prediction sample offset is determined based on at least one first gradient of the first prediction sample, wherein the first gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the first prediction sample, and wherein before being added to the first prediction sample to acquire the first final prediction sample, the first prediction sample offset is clipped to a first range;
• and
• perform a conversion between the first block and a bitstream of the video based on the first final prediction sample,
• wherein the first range is determined based on an input bit depth of the first block;
• wherein the instructions upon execution by the processor, further cause the processor to:
• determine, for a second block of the video, a second prediction sample;
• refine the second prediction sample, based on a second optical flow refinement procedure, with a second prediction sample offset to acquire a second final prediction sample;
• wherein the second prediction sample offset is determined based on at least one second gradient of the second prediction sample, wherein the second gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the second prediction sample;
• clip the second final prediction sample to a second range;
• and
• perform a conversion between the second block and the bitstream based on the second final prediction sample.

A non-transitory computer-readable storage medium storing instructions that cause a processor to:

• determine, for a first block of a video, a first prediction sample;
• refine the first prediction sample, based on a first optical flow refinement procedure, with a first prediction sample offset by adding the first prediction sample offset to the first prediction sample to acquire a first final prediction sample, wherein the first prediction sample offset is determined based on at least one first gradient of the first prediction sample, wherein the first gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the first prediction sample, and wherein before being added to the first prediction sample to acquire the first final prediction sample, the first prediction sample offset is clipped to a first range;
• and
• perform a conversion between the first block and a bitstream of the video based on the first final prediction sample,
• wherein the first range is determined based on an input bit depth of the first block;
• wherein the instructions further cause the processor to:
• determine, for a second block of the video, a second prediction sample;
• refine the second prediction sample, based on a second optical flow refinement procedure, with a second prediction sample offset to acquire a second final prediction sample;
• wherein the second prediction sample offset is determined based on at least one second gradient of the second prediction sample, wherein the second gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the second prediction sample;
• clip the second final prediction sample to a second range;
• and
• perform a conversion between the second block and the bitstream based on the second final prediction sample.

A method performed by a video processing apparatus, wherein the method comprises:

• determining, for a first block of a video, a first prediction sample;
• refining the first prediction sample, based on a first optical flow refinement procedure, with a first prediction sample offset by adding the first prediction sample offset to the first prediction sample to acquire a first final prediction sample, wherein the first prediction sample offset is determined based on at least one first gradient of the first prediction sample, wherein the first gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the first prediction sample, and wherein before being added to the first prediction sample to acquire the first final prediction sample, the first prediction sample offset is clipped to a first range;
• generating the bitstream based on the first final prediction sample, and
• storing the bitstream to a non-transitory computer-readable medium;
• wherein the first range is determined based on an input bit depth of the first block;
• wherein the method further comprises:
• determining, for a second block of the video, a second prediction sample;
• refining the second prediction sample, based on a second optical flow refinement procedure, with a second prediction sample offset to acquire a second final prediction sample;
• wherein the second prediction sample offset is determined based on at least one second gradient of the second prediction sample, wherein the second gradient is calculated based on at least a difference between two prediction samples acquired based on a location of the second prediction sample;
• clipping the second final prediction sample to a second range;
• and
• performing a conversion between the second block and the bitstream based on the second final prediction sample.