Back to results
US 11,695,950 B2Granted
Systems, apparatus and methods for inter prediction refinement with optical flow
AssigneeVid Scale, Inc.
InventorsJiancong Luo, Yuwen He
Life of this patent
Estimating the term…
Abstract
Method, apparatus and systems are disclosed. In one embodiment, a method of decoding includes obtaining a sub-block based motion prediction signal for a current block of the video; obtaining one or more spatial gradients of the sub-block based motion prediction signal or one or more motion vector difference values; obtaining a refinement signal for the current block based on the one or more obtained spatial gradients or the one or more obtained motion vector difference values; obtaining a refined motion prediction signal for the current block based on the sub-block based motion prediction signal and the refinement signal; and decoding the current block based on the refined motion prediction signal.
Claims16
A method of decoding a video, the method comprising:
- for a current block of the video, the block comprising a plurality of sub-blocks:
- generating a motion vector of a sub-block of the current block,
- generating a sub-block based motion prediction signal using the motion vector of the sub-block,
- determining a set of pixel-level motion vector difference values associated with the sub-block of the current block, wherein a pixel-level motion vector difference indicates a difference between a pixel-level motion vector obtained for a sample location in the sub-block and the motion vector of the sub-block,
- determining spatial gradients of the sub-block based motion prediction signal at each sample location of the sub-block,
- determining a motion prediction refinement signal for the current block based on the determined set of pixel-level motion vector difference values and the determined spatial gradients, and
- combining the sub-block based motion prediction signal and the motion prediction refinement signal to produce a refined motion prediction signal for the current block;
- and
- decoding the video using the refined motion prediction signal, as the prediction for the current block,
- wherein the motion vector of the sub-block is generated and the set of pixel-level motion vector difference values is determined using an affine motion model for the current block.
7 more elements
A method of encoding a video, the method comprising:
- for a current block of the video, the block comprising a plurality of sub-blocks:
- generating a motion vector of a sub-block,
- generating a sub-block based motion prediction signal using the motion vector of the sub-block,
- determining a set of pixel-level motion vector difference values associated with the sub-block of the current block, wherein a pixel-level motion vector difference indicates a difference between a pixel-level motion vector obtained for a sample location in the sub-block and the motion vector of the sub-block,
- determining spatial gradients of the sub-block based motion prediction signal at each sample location of the sub-block,
- determining a motion prediction refinement signal for the current block based on the determined set of pixel-level motion vector difference values and the determined spatial gradients, and
- combining the sub-block based motion prediction signal and the motion prediction refinement signal to produce a refined motion prediction signal for the current block;
- and
- encoding the video using the refined motion prediction signal, as the prediction for the current block,
- wherein the motion vector of the sub-block is generated and the set of pixel-level motion vector difference values is determined using an affine motion model for the current block.
7 more elements
A decoder configured to decode a video, comprising:
- a processor configured to:
- for a current block of the video, the block comprising a plurality of sub-blocks:
- generate a motion vector of a sub-block,
- generate a sub-block based motion prediction signal using the motion vector of the sub-block,
- determine a set of pixel-level motion vector difference values associated with the sub-block of the current block, wherein a pixel-level motion vector difference indicates a difference between a pixel-level motion vector obtained for a sample location in the sub-block and the motion vector of the sub-block,
- determine spatial gradients of the sub-block based motion prediction signal at each sample location of the sub-block,
- determine a motion prediction refinement signal for the current block based on the set of pixel-level motion vector difference values and the determined spatial gradients, and
- combine the sub-block based motion prediction signal and the motion prediction refinement signal to produce a refined motion prediction signal for the current block;
- and
- decode the video using the refined motion prediction signal, as the prediction for the current block,
- wherein the processor is configured to use an affine motion model for the current block to generate the motion vector of the sub-block and to determine the set of pixel-level motion vector difference values.
8 more elements
An encoder configured to encode a video, comprising:
- a processor configured to:
- for a current block of the video, the block comprising a plurality of sub-blocks:
- generate a motion vector of a sub-block,
- generate a sub-block based motion prediction signal,
- determine a set of pixel-level motion vector difference values associated with the sub-block of the current block, wherein a pixel-level motion vector difference indicates a difference between a pixel-level motion vector obtained for a sample location in the sub-block and the motion vector of the sub-block,
- determine spatial gradients of the sub-block based motion prediction signal at each sample location of the sub-block,
- determine a motion prediction refinement signal for the current block based on the set of pixel-level motion vector difference values and the determined spatial gradients, and
- combine the sub-block based motion prediction signal and the motion prediction refinement signal to produce a refined motion prediction signal for the current block;
- and
- encode the video using the refined motion prediction signal, as the prediction for the current block,
- wherein the processor is configured to use an affine motion model for the current block to generate the motion vector of the sub-block and to determine the set of pixel-level motion vector difference values.
8 more elements
Dependencies are inferred from the claim text.