# US 12,192,507 B2 · Delta motion vector in prediction refinement with optical flow process > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,507 B2 - Document ID: US12192507B2 - Status: Granted - Kind: B2 - Application number: 18/085,979 - Filed: 2022-12-21 - Published: 2023-04-27 - Granted: 2025-01-07 - Earliest claimed: 2019-04-19 - Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Inventors: Kai Zhang; Li Zhang; Hongbin Liu; Yue Wang - Web page: https://patentgrep.com/patent/US12192507B2 ## CPC - H04N 19/513: Processing of motion vectors - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures - H04N 19/593: involving spatial prediction techniques - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video processing method is disclosed to include: generating one or more initial predictions for a sub-block of a current video block of the video; refining the one or more initial predictions to generate a final prediction block using a prediction refinement with optical flow (PROF) procedure by calculating motion vector differences based on variables having a bit width not exceeding a predetermined maximum bit width; and performing a conversion between the current video block and a coded representation of the video using the variables having the bit width not exceeding the predetermined maximum bit width. ## Claims ### Claim 1 A video processing method, comprising: determining, for a current video block, an applicability of an optical flow operation with an affine mode according to a first rule that specifies a condition for applying the optical flow operation; generating initial prediction samples of a sub-block of the current video block coded with the affine mode; applying the optical flow operation to generate final prediction samples for the sub-block by deriving a prediction refinement based on motion vector differences dMvH and/or dMvV, wherein dMvH and dMvV indicate motion vector differences along a horizontal direction and a vertical direction; and performing a conversion between the current video block and a bitstream of a video based on the final prediction samples, wherein a clipping operation is performed on the dMvH and/or dMvV before the deriving of the prediction refinement according to a second rule, wherein the second rule specifies to perform the clipping operation before the prediction refinement being acquired and wherein the clipping operation is performed based on a precision of the dMvH and/or dMvV, and wherein the dMvH and/or dMvV is clipped as follows: dMvH[posX][posY]=Clip3(−2K−1,2K−1−1,dMvH[posX][posY]); dMvV[posX][posY]=Clip3(−2K−1,2K−1−1,dMvV[posX][posY]). ### Claim 2 (depends on 1) The method of claim 1, wherein K=6. ### Claim 3 (depends on 1) The method of claim 1, wherein the initial prediction samples are derived based on an initial motion vector, and wherein the initial motion vector is derived based on a dimension of the current video block and a position of the sub-block. ### Claim 4 (depends on 1) The method of claim 1, wherein the motion vector differences are calculated based on variables having a bit width not exceeding a predetermined maximum bit width. ### Claim 5 (depends on 1) The method of claim 1, wherein the prediction refinement is derived based on the motion vector differences and gradient values. ### Claim 6 (depends on 5) The method of claim 5, wherein the gradient values are calculated based on a difference between multiple initial prediction samples along the horizontal direction and the vertical direction. ### Claim 7 (depends on 1) The method of claim 1, wherein the performing of the conversion includes decoding the current video block from the bitstream. ### Claim 8 (depends on 1) The method of claim 1, wherein the performing of the conversion includes encoding the current video block into the bitstream. ### Claim 9 An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: determine, for a current video block, an applicability of an optical flow operation with an affine mode according to a first rule that specifies a condition for applying the optical flow operation; generate initial prediction samples of a sub-block of the current video block coded with the affine mode; apply the optical flow operation to generate final prediction samples for the sub-block by deriving a prediction refinement based on motion vector differences dMvH and/or dMvV, wherein dMvH and dMvV indicate motion vector differences along a horizontal direction and a vertical direction; and perform a conversion between the current video block and a bitstream of a video based on the final prediction samples, wherein a clipping operation is performed on the dMvH and/or dMvV before the deriving of the prediction refinement according to a second rule, wherein the second rule specifies to perform the clipping operation before the prediction refinement being acquired and wherein the clipping operation is performed based on a precision of the dMvH and/or dMvV, and wherein the dMvH and/or dMvV is clipped as follows: dMvH[posX][posY]=Clip3(−2K−1,2K−1−1,dMvH[posX][posY]); dMvV[posX][posY]=Clip3(−2K−1,2K−1−1,dMvV[posX][posY]). ### Claim 10 (depends on 9) The apparatus of claim 9, wherein K=6. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein the initial prediction samples are derived based on an initial motion vector, and wherein the initial motion vector is derived based on a dimension of the current video block and a position of the sub-block. ### Claim 12 (depends on 9) The apparatus of claim 9, wherein the motion vector differences are calculated based on variables having a bit width not exceeding a predetermined maximum bit width. ### Claim 13 (depends on 9) The apparatus of claim 9, wherein the prediction refinement is derived based on the motion vector differences and gradient values. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the gradient values are calculated based on a difference between multiple initial prediction samples along the horizontal direction and the vertical direction. ### Claim 15 A non-transitory computer-readable storage medium storing instructions that cause a processor to: determine, for a current video block, an applicability of an optical flow operation with an affine mode according to a first rule that specifies a condition for applying the optical flow operation; generate initial prediction samples of a sub-block of the current video block coded with the affine mode; apply the optical flow operation to generate final prediction samples for the sub-block by deriving a prediction refinement based on motion vector differences dMvH and/or dMvV, wherein dMvH and dMvV indicate motion vector differences along a horizontal direction and a vertical direction; and perform a conversion between the current video block and a bitstream of a video based on the final prediction samples, wherein a clipping operation is performed on the dMvH and/or dMvV before the deriving of the prediction refinement according to a second rule, wherein the second rule specifies to perform the clipping operation before the prediction refinement being acquired and wherein the clipping operation is performed based on a precision of the dMvH and/or dMvV, and wherein the dMvH and/or dMvV is clipped as follows: dMvH[posX][posY]=Clip3(−2K−1,2K−1−1,dMvH[posX][posY]); dMvV[posX][posY]=Clip3(−2K−1,2K−1−1,dMvV[posX][posY]). ### Claim 16 A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: determining, for a current video block, an applicability of an optical flow operation with an affine mode according to a first rule that specifies a condition for applying the optical flow operation; generating initial prediction samples of a sub-block of the current video block coded with the affine mode; applying the optical flow operation to generate final prediction samples for the sub-block by deriving a prediction refinement based on motion vector differences dMvH and/or dMvV, wherein dMvH and dMvV indicate motion vector differences along a horizontal direction and a vertical direction; and generating the bitstream of the video based on the final prediction samples, wherein a clipping operation is performed on the dMvH and/or dMvV before the deriving of the prediction refinement according to a second rule, wherein the second rule specifies to perform the clipping operation before the prediction refinement being acquired and wherein the clipping operation is performed based on a precision of the dMvH and/or dMvV, and wherein the dMvH and/or dMvV is clipped as follows: dMvH[posX][posY]=Clip3(−2K−1,2K−1−1,dMvH[posX][posY]); dMvV[posX][posY]=Clip3(−2K−1,2K−1−1,dMvV[posX][posY]).