# US 10,771,809 B2 · Picture prediction method and picture prediction apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,771,809 B2
- Document ID: US10771809B2
- Status: Granted
- Kind: B2
- Application number: 16/734,586
- Filed: 2020-01-06
- Published: 2020-05-07
- Granted: 2020-09-08
- Earliest claimed: 2015-07-03
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Sixin Lin; Ruopu Huang; Haitao Yang
- Web page: https://patentgrep.com/patent/US10771809B2

## CPC

- H04N 19/52: by predictive encoding
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/513: Processing of motion vectors
- H04N 19/523: with sub-pixel accuracy
- H04N 19/537: Motion estimation other than block-based
- H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A picture prediction method includes: determining motion vectors of W control points in a current picture block; obtaining motion vectors of P pixel units of the current picture block by using a motion model and the motion vectors of the W control points, where precision of the determined motion vectors of the W control points is 1/n of pixel precision, precision of the motion vector of each of the P pixel units is 1/N of the pixel precision, the motion vector of each of the P pixel units is used to determine a corresponding reference pixel unit in a reference picture of a corresponding pixel unit; and performing interpolation filtering on a pixel of the corresponding reference pixel unit by using an interpolation filter with a phase of Q, to obtain a predicted pixel value of each of the P pixel units.

## Claims

### Claim 1

A picture prediction method, comprising:
determining motion vectors of W control points in a current picture block, wherein the motion vectors of the W control points is determined based on a predictor with precision of 1/n of a pixel precision and a difference of motion vector with precision of 1/n of the pixel precision;
obtaining, by calculation, motion vectors of P pixel units of the current picture block by using a motion model and the motion vectors of the W control points, wherein, precision of the motion vector of each of the P pixel units is 1/N of the pixel precision, the motion vector of each of the P pixel units is used to determine a corresponding reference pixel unit in a reference picture of a corresponding pixel unit, W, n, and N are integers greater than 1, N is greater than n, and P is a positive integer; and
performing interpolation filtering on a pixel of the corresponding reference pixel unit by using an interpolation filter with a phase of Q, to obtain a predicted pixel value of each of the P pixel units, wherein Q is an integer greater than n.

### Claim 2 (depends on 1)

The method according to claim 1, wherein a value of N is a preset fixed value, and Q is less than or equal to N.

### Claim 3 (depends on 1)

The method according to claim 1, wherein a horizontal component or a vertical component of one of the motion vectors of the W control points is amplified N times in the motion model, or a component difference between motion vectors of any two of the W control points is amplified N times in the motion model.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the performing interpolation filtering comprises:
obtaining, by calculation, a phase of each of the P pixel units by using the motion vector of each of the P pixel units;
determining, based on the phase of each of the P pixel units, the interpolation filter with the phase of Q that is corresponding to the pixel unit, wherein a filter coefficient used by the interpolation filter is corresponding to the phase; and
performing interpolation filtering on the pixel of the corresponding pixel unit, in the reference picture, of each pixel unit by using the determined interpolation filter with the phase of Q that is corresponding to the pixel unit.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the obtaining the phase of each of the P pixel units comprises: obtaining the phase of each of the P pixel units according to the following formula by using the motion vector of each of the P pixel units:
X′=abs(vNx) % N, or X′=vNx & ((1≤M)−1);
Y′=abs(vNy) % N, or Y′=vNy & ((1≤M)−1); wherein
M is equal to log2 N, N is an integral power of 2, X′ represents a horizontal phase of a pixel unit with coordinates of (x, y) in the current picture block, Y′ represents a vertical phase of the pixel unit with the coordinates of (x, y) in the current picture block, vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block, and vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block.

### Claim 6 (depends on 4)

The method according to claim 4, wherein the phase comprises a horizontal phase and a vertical phase; and the determining the interpolation filter with the phase of Q comprises:
determining, based on the horizontal phase of each pixel unit, a horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit; and
determining, based on the vertical phase of each pixel unit, a vertical interpolation filter with a phase of Q that is corresponding to the pixel unit, wherein a filter coefficient used by the horizontal interpolation filter is corresponding to the horizontal phase, and a filter coefficient used by the vertical interpolation filter is corresponding to the vertical phase.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the performing interpolation filtering comprises:
performing horizontal interpolation filtering on a pixel of a corresponding reference pixel unit, in the reference picture, of a pixel unit i by using a determined horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit i, to obtain a horizontal interpolation filtering result; and
performing vertical interpolation filtering on the horizontal interpolation filtering result by using a determined vertical interpolation filter with a phase of Q that is corresponding to the pixel unit i, to obtain a predicted pixel value of the pixel unit i, wherein the pixel unit i is any one of the P pixel units.

### Claim 8 (depends on 6)

The method according to claim 6, wherein the performing interpolation filtering comprises:
performing vertical interpolation filtering on a pixel of a corresponding pixel unit, in the reference picture, of a pixel unit j by using a determined vertical interpolation filter with a phase of Q that is corresponding to the pixel unit j, to obtain a vertical interpolation filtering result; and
performing horizontal interpolation filtering on the vertical interpolation filtering result by using a determined horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit j, to obtain a predicted pixel value of the pixel unit j, wherein the pixel unit j is any one of the P pixel units.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the motion model is a translational motion model, an affine motion model, a rotational motion model, a parabolic motion model, a shearing motion model, a zooming motion model, a perspective motion model, or a bilinear motion model.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the motion model is represented as follows, wherein W is equal to 2:
{
v
Nx
=
(
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
L
⁢
x
-
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
L
⁢
y
+
Nv
0
⁢
x
)
/
n
v
Ny
=
(
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
L
⁢
x
+
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
L
⁢
y
+
Nv
0
⁢
y
)
/
n
;
or
⁢
⁢
{
v
Nx
=
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
L
⁢
x
-
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
L
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
x
v
Ny
=
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
L
⁢
x
+
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
L
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
y
,
wherein
L represents a width or a height of the current picture block, (v0x, v0y) and (v1x, v1y) represent motion vectors, whose precision is 1/n of the pixel precision, of two control points, vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of a pixel unit with coordinates of (x, y) in the current picture block, and vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the motion model is represented as follows, wherein W is equal to 3:
{
v
Nx
=
(
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
w
⁢
x
+
(
v
2
⁢
x
-
v
0
⁢
x
)
×
N
h
⁢
y
+
Nv
0
⁢
x
)
/
n
v
Ny
=
(
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
w
⁢
x
+
(
v
2
⁢
y
-
v
0
⁢
y
)
×
N
h
⁢
y
+
Nv
0
⁢
y
)
/
n
;
or
⁢
⁢
{
v
Nx
=
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
w
⁢
x
+
(
v
2
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
h
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
x
v
Ny
=
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
w
⁢
x
+
(
v
2
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
h
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
y
,
where
vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of a pixel unit with coordinates of (x, y) in the current picture block, vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block, (v0x, v0y), (v1x, v1y), and (v2x, v2y) represent motion vectors, whose precision is 1/n of the pixel precision, of three control points, w represents a width of the current picture block, and h represents a height of the current picture block.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the motion vectors of the W control points are predicted based on a motion vector, whose precision is 1/n of the pixel precision, of an encoded picture block or a decoded picture block that surrounds the current picture block.

### Claim 13 (depends on 1)

The method according to claim 1, wherein the picture prediction method is applied to a video coding process or applied to a video decoding process.

### Claim 14

A picture prediction apparatus, comprising:
a non-transitory memory storage comprising instructions; and
one or more hardware processors in communication with the memory storage, wherein the one or more hardware processors execute the instructions to:
determine motion vectors of W control points in a current picture block, wherein the motion vectors of the W control points is determined based on a predictor with precision of 1/n of a pixel precision and a difference of motion vector with precision of 1/n of the pixel precision;
obtain, by calculation, motion vectors of P pixel units of the current picture block by using a motion model and the motion vectors of the W control points, wherein precision of the motion vector of each of the P pixel units is 1/N of the pixel precision, the motion vector of each of the P pixel units is used to determine a corresponding reference pixel unit in a reference picture of a corresponding pixel unit, W, n, and N are integers greater than 1, N is greater than n, and P is a positive integer; and
perform interpolation filtering on a pixel of the corresponding reference pixel unit by using an interpolation filter with a phase of Q, to obtain a predicted pixel value of each of the P pixel units, wherein Q is an integer greater than n.

### Claim 15 (depends on 14)

The apparatus according to claim 14, wherein a value of N is a preset fixed value, and Q is less than or equal to N.

### Claim 16 (depends on 14)

The apparatus according to claim 14, wherein a horizontal component or a vertical component of one of the motion vectors of the W control points is amplified N times in the motion model, or a component difference between motion vectors of any two of the W control points is amplified N times in the motion model.

### Claim 17 (depends on 14)

The apparatus according to claim 14, wherein the one or more hardware processors is configured to:
obtain, by calculation, a phase of each of the P pixel units by using the motion vector of each of the P pixel units;
determine, based on the phase of each of the P pixel units, the interpolation filter with the phase of Q that is corresponding to the pixel unit, wherein a filter coefficient used by the interpolation filter is corresponding to the phase; and
perform interpolation filtering on the pixel of the corresponding pixel unit, in the reference picture, of each pixel unit by using the determined interpolation filter with the phase of Q that is corresponding to the pixel unit.

### Claim 18 (depends on 17)

The apparatus according to claim 17, wherein the one or more hardware processors is configured to: obtain the phase of each of the P pixel units according to the following formula by using the motion vector of each of the P pixel units:
X′=abs(vNx) % N, or X′=vNx & ((1≤M)−1);
Y′=abs(vNy) % N, or Y′=vNy & ((1≤M)−1); wherein
M is equal to log2 N, N is an integral power of 2, X′ represents a horizontal phase of a pixel unit with coordinates of (x, y) in the current picture block, Y′ represents a vertical phase of the pixel unit with the coordinates of (x, y) in the current picture block, vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block, and vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block.

### Claim 19 (depends on 17)

The apparatus according to claim 17, wherein the phase comprises a horizontal phase and a vertical phase; and the one or more hardware processors is configured to:
determine, based on the horizontal phase of each pixel unit, a horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit; and
determine, based on the vertical phase of each pixel unit, a vertical interpolation filter with a phase of Q that is corresponding to the pixel unit, wherein a filter coefficient used by the horizontal interpolation filter is corresponding to the horizontal phase, and a filter coefficient used by the vertical interpolation filter is corresponding to the vertical phase.

### Claim 20 (depends on 19)

The apparatus according to claim 19, wherein the one or more hardware processors is configured to:
perform horizontal interpolation filtering on a pixel of a corresponding reference pixel unit, in the reference picture, of a pixel unit i by using a determined horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit i, to obtain a horizontal interpolation filtering result; and
perform vertical interpolation filtering on the horizontal interpolation filtering result by using a determined vertical interpolation filter with a phase of Q that is corresponding to the pixel unit i, to obtain a predicted pixel value of the pixel unit i, wherein the pixel unit i is any one of the P pixel units.

### Claim 21 (depends on 19)

The apparatus according to claim 19, wherein the one or more hardware processors is configured to:
perform vertical interpolation filtering on a pixel of a corresponding pixel unit, in the reference picture, of a pixel unit j by using a determined vertical interpolation filter with a phase of Q that is corresponding to the pixel unit j, to obtain a vertical interpolation filtering result; and
perform horizontal interpolation filtering on the vertical interpolation filtering result by using a determined horizontal interpolation filter with a phase of Q that is corresponding to the pixel unit j, to obtain a predicted pixel value of the pixel unit j, wherein the pixel unit j is any one of the P pixel units.

### Claim 22 (depends on 14)

The apparatus according to claim 14, wherein the motion model is a translational motion model, an affine motion model, a rotational motion model, a parabolic motion model, a shearing motion model, a zooming motion model, a perspective motion model, or a bilinear motion model.

### Claim 23 (depends on 14)

The apparatus according to claim 14, wherein the motion model is represented as follows, wherein W is equal to 2:
{
v
Nx
=
(
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
L
⁢
x
-
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
L
⁢
y
+
Nv
0
⁢
x
)
/
n
v
Ny
=
(
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
L
⁢
x
+
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
L
⁢
y
+
Nv
0
⁢
y
)
/
n
;
or
⁢
⁢
{
v
Nx
=
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
L
⁢
x
-
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
L
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
x
v
Ny
=
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
L
⁢
x
+
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
L
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
y
,
wherein
L represents a width or a height of the current picture block, (v0x, v0y) and (v1x, v1y) represent motion vectors, whose precision is 1/n of the pixel precision, of two control points, vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of a pixel unit with coordinates of (x, y) in the current picture block, and vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block.

### Claim 24 (depends on 14)

The apparatus according to claim 14, wherein the motion model is represented as follows, wherein W is equal to 3:
{
v
Nx
=
(
(
v
1
⁢
x
-
v
0
⁢
x
)
×
N
w
⁢
x
+
(
v
2
⁢
x
-
v
0
⁢
x
)
×
N
h
⁢
y
+
Nv
0
⁢
x
)
/
n
v
Ny
=
(
(
v
1
⁢
y
-
v
0
⁢
y
)
×
N
w
⁢
x
+
(
v
2
⁢
y
-
v
0
⁢
y
)
×
N
h
⁢
y
+
Nv
0
⁢
y
)
/
n
;
or
⁢
⁢
{
v
Nx
=
(
v
1
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
w
⁢
x
+
(
v
2
⁢
x
-
v
0
⁢
x
)
×
(
N
/
n
)
h
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
x
v
Ny
=
(
v
1
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
w
⁢
x
+
(
v
2
⁢
y
-
v
0
⁢
y
)
×
(
N
/
n
)
h
⁢
y
+
(
N
/
n
)
⁢
v
0
⁢
y
,
wherein
vNx represents a horizontal component of a motion vector, whose precision is 1/N of the pixel precision, of a pixel unit with coordinates of (x, y) in the current picture block, vNy represents a vertical component of the motion vector, whose precision is 1/N of the pixel precision, of the pixel unit with the coordinates of (x, y) in the current picture block, (v0x, v0y), (v1x, v1y), and (v2x, v2y) represent motion vectors, whose precision is 1/n of the pixel precision, of three control points, w represents a width of the current picture block, and h represents a height of the current picture block.

### Claim 25 (depends on 14)

The apparatus according to claim 14, wherein the motion vectors of the W control points are predicted based on a motion vector, whose precision is 1/n of the pixel precision, of an encoded picture block or a decoded picture block that surrounds the current picture block.

### Claim 26 (depends on 14)

The apparatus according to claim 14, wherein the current picture block is generated in a video coding process or applied to a video decoding process.

### Claim 27

A non-transitory computer-readable medium storing computer instructions, that when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
determining motion vectors of W control points in a current picture block, wherein the motion vectors of the W control points is determined based on a predictor with precision of 1/n of a pixel precision and a difference of motion vector with precision of 1/n of the pixel precision;
obtaining, by calculation, motion vectors of P pixel units of the current picture block by using a motion model and the motion vectors of the W control points, wherein precision of the motion vector of each of the P pixel units is 1/N of the pixel precision, the motion vector of each of the P pixel units is used to determine a corresponding reference pixel unit in a reference picture of a corresponding pixel unit, W, n, and N are integers greater than 1, N is greater than n, and P is a positive integer; and
performing interpolation filtering on a pixel of the corresponding reference pixel unit by using an interpolation filter with a phase of Q, to obtain a predicted pixel value of each of the P pixel units, wherein Q is an integer greater than n.