# US 2019/0320199 A1 · Method and Apparatus of Motion Refinement for Video Coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320199 A1
- Document ID: US20190320199A1
- Status: Published application
- Kind: A1
- Application number: 16/471,689
- Filed: 2017-12-19
- Published: 2019-10-17
- Earliest claimed: 2016-12-22
- Assignees: Mediatek Inc.
- Applicants: Mediatek Inc.
- Inventors: Ching-Yeh CHEN; Tzu-Der CHUANG; Yu-Wen HUANG
- Web page: https://patentgrep.com/patent/US20190320199A1

## CPC

- H04N 19/523: with sub-pixel accuracy
- H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
- H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/513: Processing of motion vectors
- H04N 19/537: Motion estimation other than block-based
- H04N 19/563: Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
- H04N 19/57: Motion estimation characterised by a search window with variable size or shape
- H04N 19/80: Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/82: involving filtering within a prediction loop
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Method and apparatus of using Pattern-based MV derivation (PMVD), Bi-directional optical flow (BIO) or Decoder-side MV refinement (DMVR) to refine motion for a bi-direction predicted block are disclosed. According to one method of the present invention, a first and a second motion-compensated reference blocks associated with the current block in a first and a second reference pictures from reference picture list 0 and list 1 are determined respectively. The PMVD process, BIO process or DMVR process is applied to generate motion refinement for the current block using reference data corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block so as to reduce required system memory bandwidth. Also an adaptive block size method for block-based BIO is disclosed.

## Claims

### Claim 1

A method of video coding using Pattern-based MV derivation (PMVD), Bi-directional optical flow (BIO) or Decoder-side MV refinement (DMVR) to refine motion for a bi-direction predicted block, the method comprising:
receiving input data associated with a current block in a current picture, wherein the current block is coded using bi-direction prediction;
determining a first motion-compensated reference block associated with the current block in a first reference picture from reference picture list 0, wherein the first motion-compensated reference block includes additional surrounding first pixels around a corresponding block of the current block in the first reference picture for performing interpolation filter required for any fractional motion vector of the current block;
determining a second motion-compensated reference block associated with the current block in a second reference picture from reference picture list 1, wherein the second motion-compensated reference block includes additional surrounding second pixels around the corresponding block of the current block in the second reference picture for performing interpolation filter required for any fractional motion vector of the current block;
applying PMVD process, BIO process or DMVR process to generate motion refinement for the current block using reference data corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block; and
encoding or decoding the current block based on motion-compensated prediction according to the motion refinement.

### Claim 2 (depends on 1)

The method of claim 1, wherein the BIO process comprises calculating first gradients for first pixels in the first motion-compensated reference block and second gradients for second pixels in the second motion-compensated reference block, deriving displacement in x-direction and y-direction using the first gradients and the second gradients, and generating the motion refinement based on the displacement in the x-direction and the y-direction.

### Claim 3 (depends on 2)

The method of claim 2, wherein said calculating the first gradients and the second gradients uses a 3-tap filter corresponding to {−1, 0, 1} for non-boundary pixels of the first motion-compensated reference block and the second motion-compensated reference block; and said calculating the first gradients and the second gradients avoids a need for reference data from outside the first motion-compensated reference block and the second motion-compensated reference block by using a 2-tap filter corresponding to {−1, 1} for boundary pixels of the first motion-compensated reference block and the second motion-compensated reference block; and wherein the boundary pixels are in one-pixel boundaries of the first motion-compensated reference block or the second motion-compensated reference block.

### Claim 4 (depends on 1)

The method of claim 1, wherein the PMVD process, the BIO process or the DMVR process to generate motion refinement for the current block is skipped for boundary pixels of the current block if the PMVD process, the BIO process or the DMVR process for the boundary pixels of the current block requires any reference data outside the first motion-compensated reference block and the second motion-compensated reference block.

### Claim 5 (depends on 1)

The method of claim 1, wherein if the PMVD process, the BIO process or the DMVR process for the current block requires one or more motion-compensated reference pixels outside the first motion-compensated reference block and the second motion-compensated reference block, said one or more motion-compensated reference pixels are generated by padding.

### Claim 6

An apparatus of video coding using Pattern-based MV derivation (PMVD), Bi-directional optical flow (BIO) or Decoder-side MV refinement (DMVR) to refine motion for a bi-direction predicted block, the apparatus of video coding comprising one or more electronic circuits or processors arranged to:
receive input data associated with a current block in a current picture, wherein the current block is coded using bi-direction prediction;
determine a first motion-compensated reference block associated with the current block in a first reference picture from reference picture list 0, wherein the first motion-compensated reference block includes additional surrounding first pixels around a corresponding block of the current block in the first reference picture for performing interpolation filter required for any fractional motion vector of the current block;
determine a second motion-compensated reference block associated with the current block in a second reference picture from reference picture list 1, wherein the second motion-compensated reference block includes additional surrounding second pixels around the corresponding block of the current block in the second reference picture for performing interpolation filter required for any fractional motion vector of the current block;
apply PMVD process, BIO process or DMVR process to generate motion refinement for the current block using reference data corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block; and
encode or decode the current block based on motion-compensated prediction according to the motion refinement.

### Claim 7

A non-transitory computer readable medium storing program instructions causing a processing circuit of an apparatus to perform a video coding method, and the method comprising:
receiving input data associated with a current block in a current picture, wherein the current block is coded using bi-direction prediction;
determining a first motion-compensated reference block associated with the current block in a first reference picture from reference picture list 0, wherein the first motion-compensated reference block includes additional surrounding first pixels around a corresponding block of the current block in the first reference picture for performing interpolation filter required for any fractional motion vector of the current block;
determining a second motion-compensated reference block associated with the current block in a second reference picture from reference picture list 1, wherein the second motion-compensated reference block includes additional surrounding second pixels around the corresponding block of the current block in the second reference picture for performing interpolation filter required for any fractional motion vector of the current block;
applying PMVD process, BIO process or DMVR process to generate motion refinement for the current block using reference data corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block; and
encoding or decoding the current block based on motion-compensated prediction according to the motion refinement.

### Claim at position 8

8-17. (canceled)