# US 11,451,824 B2 · Motion vector refinement search with integer pixel resolution > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,451,824 B2 - Document ID: US11451824B2 - Status: Granted - Kind: B2 - Application number: 17/135,624 - Filed: 2020-12-28 - Published: 2021-06-24 - Granted: 2022-09-20 - Earliest claimed: 2018-06-28 - Assignees: Huawei Technologies Co., Ltd. - Applicants: Huawei Technologies Co., Ltd. - Inventors: Semih Esenlik; Zhijie Zhao; Ivan Krasnov; Jianle Chen - Web page: https://patentgrep.com/patent/US11451824B2 ## CPC - H04N 19/563: Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes - H04N 19/103: Selection of coding mode or of prediction mode - H04N 19/172: the region being a picture, frame or field - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/521: {for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors} - H04N 19/523: with sub-pixel accuracy - H04N 19/55: Motion estimation with spatial constraints, e.g. at image or region borders - H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search - H04N 19/57: Motion estimation characterised by a search window with variable size or shape - H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures - H04N 19/80: Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An apparatus and method determining a motion vector for a block, which relates to motion vector determination employing template matching or bilateral matching, are provided. At first, candidate motion vectors are obtained and are mapped to respective representatives (representative motion vectors). In particular, if the obtained candidate motion vectors point to fractional positions and thus, interpolation from the integer sample positions is required, the initial motion vector is rounded to a closest integer sample position. The rounded position is then used to define the search space for the template or bilateral matching to determine the best matching motion vector for the block (in other words, a refinement of an initial motion vector around which the candidate space is spanned). Finally, the mapping operation can be reverted in order to obtain a result in the original search space. ## Claims ### Claim 1 An apparatus for determining a motion vector for a block of a video picture, wherein the apparatus comprises a memory coupled to a processor, the memory comprising processor-executable instructions which when executed by the processor cause the processor to: obtain at least two candidate motion vectors pointing to positions in a reference picture and forming a search space; individually modify each of the at least two candidate motion vectors by shift operations to the positions in the reference picture so as to obtain for each of the candidate motion vectors a respective representative motion vector in a modified search space formed by the at least two representative motion vectors, wherein a candidate motion vector of the at least two candidate motion vectors points to a corresponding fractional sample position in the search space, wherein the shift operation applied to the candidate motion vector results in a position pointed to by the respective representative motion vector being given by rounding the fractional sample position pointed to by the candidate motion vector to an integer sample position, and wherein the integer sample position resulting from rounding the fractional sample position is one of the sample positions of the modified search space, select a first motion vector from the modified search space, by evaluating a criterion for each sample position in the modified search space, and obtain a second motion vector by applying an inverse shift operation of the shift operation by which the first motion vector was obtained to the first motion vector. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the processor is further configured to decide, on the basis of a judgment of a further criterion, whether to output the first motion vector as the motion vector for the block or to obtain and output the second motion vector as the motion vector for the block, wherein the further criterion is whether the first motion vector is equal to a particular representative motion vector of the modified search space and the first motion vector is output if the judgment is negative and the second motion vector is obtained and output if the judgment is affirmative. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein the processor is further configured to output the second motion vector as the motion vector for the block. ### Claim 4 (depends on 3) The apparatus according to claim 3, wherein the modified search space is formed by K rows and L columns of samples located in rows and columns in integer distance from each other, wherein the integer sample position resulting from rounding the fractional position is one of the samples of the modified search space. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein the rounded fractional position is located substantially in the center of the modified search space. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein the rounding is obtained as an integer sample position closest to the fractional position to which the candidate motion vector points. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein the rounding is obtained as an integer sample position in a predefined direction, the predefined direction being left, right, top or bottom. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the modified search space consists of sample positions spaced by integer sample steps from each other. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein there are obtained at least two pairs of candidate motion vectors such that the candidate motion vectors of each pair point to positions in different reference pictures, accordingly forming search spaces in each of the reference pictures, and the processing circuitry is adapted to perform the processing defined in claim 1 for the candidate motion vectors of the pair so as to obtain a pair of first motion vectors and a pair of second motion vectors. ### Claim 10 (depends on 9) The apparatus according to claim 9, wherein the processor is configured to select the first pair of motion vectors by performing bilateral matching between the pairs of representative motion vectors in the modified search spaces in the reference pictures. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the search space on which the bilateral or template matching is performed includes one or more fractional sample positions, each of the fractional sample positions belonging to the search space is obtained by interpolation filtering with a filter of a predefined tap-size assessing integer samples only within a window, the window is formed by integer samples accessible for the template or bilateral matching in the search space. ### Claim 12 (depends on 1) The apparatus according to claim 1, wherein the processor is further configured to obtain a template for the block and select the first motion vector by template matching with the template in the modified search space. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein the processor is configured to perform template or bilateral matching for a search space of which the sample positions are determined in a plurality of operations including: determining an integer search space consisting of sample positions spaced by integer sample steps from each other, performing template or bilateral matching in the integer search space to obtain a best matching integer position, based on the best matching integer position, determining a fractional search space with at least one sample spaced by less than integer sample step from the closest integer search space sample positions, and performing template or bilateral matching in the fractional search space to obtain a best matching position. ### Claim 14 A method for determining a motion vector for a block, wherein the method is applied to a processor of an apparatus and comprises: obtaining at least two candidate motion vectors pointing to positions in a reference picture and forming a search space; individually modifying each of the at least two candidate motion vectors by predetermined shift operations to the positions in the reference picture so as to obtain a respective representative motion vector in a modified search space formed by at least two representative motion vectors, wherein a candidate motion vector of the at least two candidate motion vectors points to a corresponding fractional sample position in the search space, wherein the shift operation applied to the candidate motion vector results in a position pointed to by the respective representative motion vector being given by rounding the fractional sample position pointed to by the candidate motion vector to an integer sample position, and wherein the integer sample position resulting from rounding the fractional sample position is one of the sample positions of the modified search space, selecting a first motion vector from the modified search space, by evaluating a predetermined criterion in the modified search space, and obtaining a second motion vector by applying an inverse shift operation of the predetermined shift operation by which the first motion vector was obtained to the first motion vector. ### Claim 15 (depends on 14) The method according to claim 14, further comprising: deciding, on the basis of a judgment of a further criterion, whether to output the first motion vector as the motion vector for the block or to obtain and output the second motion vector as the motion vector for the block, wherein the further criterion is whether the first motion vector is equal to a particular representative motion vector of the modified search space and the first motion vector is output in response to the judgment being negative and the second motion vector is obtained and output in response to the judgment being affirmative. ### Claim 16 (depends on 14) The method according to claim 14, further comprising: outputting the second motion vector as the motion vector for the block. ### Claim 17 A non-transitory computer readable medium comprising processor-executable instructions which when executed by a processor cause the processor to perform a method for determining a motion vector for a block, the method including: obtaining at least two candidate motion vectors pointing to positions in a reference picture and forming a search space; individually modifying each of the at least two candidate motion vectors by predetermined shift operations to the positions in the reference picture so as to obtain a respective representative motion vector in a modified search space formed by at least two representative motion vectors, wherein a candidate motion vector of the at least two candidate motion vectors points to a corresponding fractional sample position in the search space, wherein the shift operation applied to the candidate motion vector results in a position pointed to by the respective representative motion vector being given by rounding the fractional sample position pointed to by the candidate motion vector to an integer sample position, and wherein the integer sample position resulting from rounding the fractional sample position is one of the sample positions of the modified search space, selecting a first motion vector from the modified search space, by evaluating a predetermined criterion in the modified search space, and obtaining a second motion vector by applying an inverse shift operation of the predetermined shift operation by which the first motion vector was obtained to the first motion vector. ### Claim 18 (depends on 17) The non-transitory computer readable medium according to claim 17, wherein the method further comprises: deciding, on the basis of a judgment of a further criterion, whether to output the first motion vector as the motion vector for the block or to obtain and output the second motion vector as the motion vector for the block, wherein the further criterion is whether the first motion vector is equal to a particular representative motion vector of the modified search space and the first motion vector is output in response to the judgment being negative and the second motion vector is obtained and output in response to the judgment being affirmative. ### Claim 19 (depends on 17) The non-transitory computer readable medium according to claim 17, wherein the method further comprises: outputting the second motion vector as the motion vector for the block.