# US 2019/0320205 A1 · Template matching-based prediction method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320205 A1
- Document ID: US20190320205A1
- Status: Published application
- Kind: A1
- Application number: 16/452,082
- Filed: 2019-06-25
- Published: 2019-10-17
- Earliest claimed: 2016-12-26
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yongbing LIN
- Web page: https://patentgrep.com/patent/US20190320205A1

## CPC

- H04N 19/61: in combination with predictive coding
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/182: the unit being a pixel
- H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
- H04N 19/90: using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A template matching-based prediction method includes: performing matching search separately in at least two reference images of a to-be-processed unit based on a current template to obtain, at each time of the matching search, one set of motion information, one unidirectional matching template, and one unidirectional template distortion value that are corresponding to the reference image, where the current template includes a plurality of reconstructed pixels with a preset quantity at preset positions in a neighboring domain of the to-be-processed unit, and the unidirectional template distortion value represents a difference between the current template and the unidirectional matching template; determining, as target motion information of the to-be-processed unit, motion information corresponding to a smallest one of the obtained unidirectional template distortion values; and constructing a predicted value of the to-be-processed unit based on the target motion information.

## Claims

### Claim 1

A method for prediction based on template matching, the method comprising:
performing matching search separately in at least two reference images of a to-be-processed unit based on a current template to obtain, at each time of the matching search for a given reference image, one set of motion information, one unidirectional matching template, and one unidirectional template distortion value that are corresponding to the given reference image, wherein the current template comprises a plurality of reconstructed pixels with a preset quantity at preset positions in a neighboring domain of the to-be-processed unit, and the unidirectional template distortion value represents a difference between the current template and the unidirectional matching template;
determining, as target motion information of the to-be-processed unit, motion information corresponding to a smallest one of the obtained unidirectional template distortion values; and
constructing a predicted value of the to-be-processed unit based on the target motion information.

### Claim 2 (depends on 1)

The method according to claim 1, wherein after the performing matching search separately in the at least two reference images of the to-be-processed unit, the method further comprises:
obtaining a weighted template distortion value based on pixel values of two of the unidirectional matching templates, wherein the weighted template distortion value represents a weighted difference between the current template and the two unidirectional matching templates, and the weighted template distortion value is corresponding to two sets of motion information corresponding to the two unidirectional matching templates; and
wherein the determining, as the target motion information of the to-be-processed unit, the motion information corresponding to the smallest one of the obtained unidirectional template distortion values comprises:
determining, as the target motion information, motion information corresponding to a smallest one of the obtained unidirectional template distortion values and the weighted template distortion value.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the constructing the predicted value of the to-be-processed unit based on the target motion information comprises:
in response to determining that the target motion information comprises a first set of motion information,
obtaining, based on the first set of motion information, the predicted value of the to-be-processed unit from a first reference image corresponding to the first set of motion information; or
in response to determining that the target motion information comprises a second set of motion information and a third set of motion information,
obtaining, based on the second set of motion information, a second predicted value from a second reference image corresponding to the second set of motion information;
obtaining, based on the third set of motion information, a third predicted value from a third reference image corresponding to the third set of motion information; and
determining a weighted value of the second predicted value and the third predicted value as the predicted value of the to-be-processed unit.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the performing matching search comprises:
traversing, in reconstructed pixels within a preset range of the reference image, a plurality of candidate reconstructed-pixel combinations that have a same size and a same shape as the current template, and calculating a plurality of pixel differences between the plurality of candidate reconstructed-pixel combinations and the current template;
determining the unidirectional matching template and the unidirectional template distortion value based on a smallest one of the pixel differences, wherein the unidirectional matching template comprises a candidate reconstructed-pixel combination corresponding to the unidirectional template distortion value; and
determining the motion information based on the reference image and a position vector difference between the unidirectional matching template and the current template.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the performing matching search separately in the at least two reference images of the to-be-processed unit based on the current template comprises:
performing the matching search in at least one reference image of a forward reference image list of the to-be-processed unit based on the current template to obtain one set of forward motion information, one forward matching template, and one forward template distortion value; and
performing the matching search in at least one reference image of a backward reference image list of the to-be-processed unit based on the current template to obtain one set of backward motion information, one backward matching template, and one backward template distortion value.

### Claim 6 (depends on 2)

The method according to claim 2, wherein the obtaining the weighted template distortion value based on the pixel values of two of the unidirectional matching templates is expressed by using the following formula:
Tw=|ω0×T1+(1−ω0)×T2−Tc|,
wherein Tw represents the weighted template distortion value, T1 and T2 represent the pixel values of the two unidirectional matching templates, Tc represents a pixel value of the current template, ω0 represents a weighting coefficient, ω0≥0, and ω0≤1.

### Claim 7 (depends on 1)

The method according to claim 1, before the performing matching search separately in the at least two reference images of the to-be-processed unit based on the current template, the method further comprises:
determining that a prediction mode of the to-be-processed unit is a merge mode.

### Claim 8 (depends on 2)

The method according to claim 2, wherein before the determining, as the target motion information, the motion information corresponding to the smallest one of the obtained unidirectional template distortion values and the weighted template distortion value, the method further comprises:
adjusting the weighted template distortion value to obtain an adjusted weighted template distortion value; and
wherein the determining, as the target motion information, the motion information corresponding to the smallest one of the obtained unidirectional template distortion values and the weighted template distortion value comprises:
determining, as the target motion information, motion information corresponding to a smallest one of the obtained unidirectional template distortion values and the adjusted weighted template distortion value.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the adjusting the weighted template distortion value to obtain the adjusted weighted template distortion value comprises:
multiplying the weighted template distortion value by an adjustment coefficient to obtain the adjusted weighted template distortion value, wherein the adjustment coefficient is greater than 0 and less than or equal to 1.

### Claim 10

An apparatus for prediction based on template matching, the apparatus comprising:
a non-transitory memory having processor-executable instructions stored thereon; and
a processor, coupled to the non-transitory memory, configured to execute the processor-executable instructions to facilitate:
performing matching search separately in at least two reference images of a to-be-processed unit based on a current template to obtain, at each time of the matching search for a given reference image, one set of motion information, one unidirectional matching template, and one unidirectional template distortion value that are corresponding to the given reference image, wherein the current template comprises a plurality of reconstructed pixels with a preset quantity at preset positions in a neighboring domain of the to-be-processed unit, and the unidirectional template distortion value represents a difference between the current template and the unidirectional matching template;
determining, as target motion information of the to-be-processed unit, motion information corresponding to a smallest one of the obtained unidirectional template distortion values; and
constructing a predicted value of the to-be-processed unit based on the target motion information.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein after the matching search is performed separately in the at least two reference images of the to-be-processed unit, the processor is configured to execute the processor-executable instructions to further facilitate:
obtaining a weighted template distortion value based on pixel values of two of the unidirectional matching templates, wherein the weighted template distortion value represents a weighted difference between the current template and the two unidirectional matching templates, and the weighted template distortion value is corresponding to two sets of motion information corresponding to the two unidirectional matching templates; and
determining, as the target motion information, motion information corresponding to a smallest one of the obtained unidirectional template distortion values and the weighted template distortion value.

### Claim 12 (depends on 10)

The apparatus according to claim 10, wherein the processor is configured to execute the processor-executable instructions to further facilitate:
in response to determining that the target motion information comprises a first set of motion information,
obtaining, based on the first set of motion information, the predicted value of the to-be-processed unit from a first reference image corresponding to the first set of motion information; or
in response to determining that the target motion information comprises a second set of motion information and a third set of motion information,
obtaining, based on the second set of motion information, a second predicted value from a second reference image corresponding to the second set of motion information;
obtaining, based on the third set of motion information, a third predicted value from a third reference image corresponding to the third set of motion information; and
determining a calculated weighted value of the second predicted value and the third predicted value as the predicted value of the to-be-processed unit.

### Claim 13 (depends on 10)

The apparatus according to claim 10, wherein the processor is configured to execute the processor-executable instructions to further facilitate:
traversing, in reconstructed pixels within a preset range of the reference image, a plurality of candidate reconstructed-pixel combinations that have a same size and a same shape as the current template, and calculate a plurality of pixel differences between the plurality of candidate reconstructed-pixel combinations and the current template;
determining the unidirectional matching template and the unidirectional template distortion value based on a smallest one of the pixel differences, wherein the unidirectional matching template comprises a candidate reconstructed-pixel combination corresponding to the unidirectional template distortion value; and
determining the motion information based on the reference image and a position vector difference between the unidirectional matching template and the current template.

### Claim 14 (depends on 10)

The apparatus according to claim 10, wherein the processor is configured to execute the processor-executable instructions to further facilitate:
performing the matching search in at least one reference image of a forward reference image list of the to-be-processed unit based on the current template to obtain one set of forward motion information, one forward matching template, and one forward template distortion value; and
performing the matching search in at least one reference image of a backward reference image list of the to-be-processed unit based on the current template to obtain one set of backward motion information, one backward matching template, and one backward template distortion value.

### Claim 15 (depends on 11)

The apparatus according to claim 11, wherein the obtaining the weighted template distortion value based on the pixel values of two of the unidirectional matching templates is expressed by using the following formula:
Tw=|ω0×T1+(1−ω0)×T2−Tc|,
wherein Tw represents the weighted template distortion value, T1 and T2 represent the pixel values of the two unidirectional matching templates, Tc represents a pixel value of the current template, ω0 represents a weighting coefficient, ω0≥0, and ω0 ≤1.

### Claim 16 (depends on 10)

The apparatus according to claim 10, wherein before the performing matching search separately in the at least two reference images of the to-be-processed unit, the processor is configured to execute the processor-executable instructions to further facilitate:
determining that a prediction mode of the to-be-processed unit is a merge mode.

### Claim 17 (depends on 11)

The apparatus according to claim 11, wherein the processor is configured to execute the processor-executable instructions to further facilitate:
adjusting the weighted template distortion value to obtain an adjusted weighted template distortion value; and
determining, as the target motion information, motion information corresponding to a smallest one of the obtained unidirectional template distortion values and the adjusted weighted template distortion value.

### Claim 18 (depends on 17)

The apparatus according to claim 17, wherein the processor is configured to execute the processor-executable instructions to further facilitate:
multiplying the weighted template distortion value by an adjustment coefficient to obtain the adjusted weighted template distortion value, wherein the adjustment coefficient is greater than 0 and less than or equal to 1.

### Claim 19

A method for prediction based on template matching, the method comprising:
performing matching search in a first reference image of a to-be-processed unit based on a current template to obtain first motion information, a first matching template, and a first template distortion value that are of the to-be-processed unit, wherein the current template comprises a plurality of reconstructed pixels with a preset quantity at preset positions in a neighboring domain of the to-be-processed unit, and the first template distortion value represents a difference between the first matching template and the current template;
updating a pixel value of the current template based on a pixel value of the first matching template;
performing matching search in a second reference image of the to-be-processed unit based on the updated current template to obtain second motion information, a second matching template, and a second template distortion value that are of the to-be-processed unit, wherein the second template distortion value represents a difference between the second matching template and the updated current template; and
determining a weighted predicted value of the to-be-processed unit based on the first motion information and the second motion information when the second template distortion value is less than the first template distortion value.

### Claim 20 (depends on 19)

The method according to claim 19, wherein the updating the pixel value of the current template based on the pixel value of the first matching template comprises updating the pixel value of the current template in the following manner:
T1=(T0−ω0×P1)/(1−ω0),
wherein T1 represents a pixel value of the updated current template, T0 represents the pixel value of the current template, P1 represents the pixel value of the first matching template, ω0 represents a weighting coefficient corresponding to the first matching template, and ω0 is a positive number less than 1.