# US 12,120,340 B2 · Constraints for usage of updated motion information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,340 B2 - Document ID: US12120340B2 - Status: Granted - Kind: B2 - Application number: 17/739,598 - Filed: 2022-05-09 - Published: 2022-08-25 - Granted: 2024-10-15 - Earliest claimed: 2018-08-04 - Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Inventors: Hongbin Liu; Li Zhang; Kai Zhang; Yue Wang - Web page: https://patentgrep.com/patent/US12120340B2 ## CPC - H04N 19/109: among a plurality of temporal predictive coding modes - H04N 19/52: by predictive encoding - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/184: the unit being bits, e.g. of the compressed video stream - H04N 19/513: Processing of motion vectors - H04N 19/523: with sub-pixel accuracy - H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search - H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/96: Tree coding, e.g. quad-tree coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Embodiments of the present disclosure relates to constraints for usage of updated motion information. A method for video processing is provided, including: determining original motion information associated with a current block; generating updated motion information based on specific prediction mode; and performing, based on the updated motion information, a conversion between the current block and a bitstream representation of a video data including the current block, wherein the specific prediction mode includes one or more of bi-directional optical flow (BIO) refinement, a decoder-side motion vector refinement (DMVR), frame-rate up conversion (FRUC) techniques or a template matching technique. ## Claims ### Claim 1 A method for video processing, comprising: determining, for a conversion between a current block of a video and a bitstream of the video, motion information associated with the current block; refining, based on a first coding tool, the motion information to acquire first motion information that is refined motion vector, wherein the first coding tool is a bi-directional optical flow coding tool and is used to acquire a first offset for refining the motion information; generating, based on the first motion information from the current block, a first prediction for the current block; updating, based on a second coding tool, the first prediction to acquire a second prediction, wherein the second coding tool is a bi-directional optical flow coding tool and is used to acquire second offset for updating the first prediction; and performing, based on the second prediction, the conversion. ### Claim 2 (depends on 1) The method of claim 1, wherein different interpolation filters are used in the refining based on the first coding tool and the updating based on the second coding tool. ### Claim 3 (depends on 1) The method of claim 1, wherein only integer motion vector is used in the refining based on the first coding tool. ### Claim 4 (depends on 1) The method of claim 1, wherein the motion information is signaled in Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice header, coding tree units (CTUs) or CUs. ### Claim 5 (depends on 1) The method of claim 1, wherein whether to use the first coding tool and/or the second coding tool depends on the motion information. ### Claim 6 (depends on 5) The method of claim 5, wherein the motion information comprises one or more of size, shape, slice type and prediction direction of the current block. ### Claim 7 (depends on 1) The method of claim 1, wherein the conversion includes encoding the current block into the bitstream. ### Claim 8 (depends on 1) The method of claim 1, wherein the conversion includes decoding the current block from the bitstream. ### Claim 9 An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: determine, for a conversion between a current block of a video and a bitstream of the video, motion information associated with the current block; refine, based on a first coding tool, the motion information to acquire first motion information that is refined motion vector, wherein the first coding tool is a bi-directional optical flow coding tool and is used to acquire a first offset for refining the motion information; generate, based on the first motion information from the current block, a first prediction for the current block; updating, based on a second coding tool, the first prediction to acquire a second prediction, wherein the second coding tool is a bi-directional optical flow coding tool and is used to acquire second offset for updating the first prediction; and perform, based on the second prediction, the conversion. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein different interpolation filters are used in the refining based on the first coding tool and the updating based on the second coding tool. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein only integer motion vector is used in the refining based on the first coding tool. ### Claim 12 (depends on 9) The apparatus of claim 9, wherein the motion information is signaled in Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice header, coding tree units (CTUs) or CUs. ### Claim 13 (depends on 9) The apparatus of claim 9, wherein whether to use the first coding tool and/or the second coding tool depends on the motion information. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the motion information comprises one or more of size, shape, slice type and prediction direction of the current block. ### Claim 15 A non-transitory computer-readable storage medium storing instructions that cause a processor to: determine, for a conversion between a current block of a video and a bitstream of the video, motion information associated with the current block; refine, based on a first coding tool, the motion information to acquire first motion information that is refined motion vector, wherein the first coding tool is a bi-directional optical flow coding tool and is used to acquire a first offset for refining the motion information; generate, based on the first motion information from the current block, a first prediction for the current block; update, based on a second coding tool, the first prediction to acquire a second prediction, wherein the second coding tool is a bi-directional optical flow coding tool and is used to acquire second offset for updating the first prediction; and perform, based on the second prediction, the conversion. ### Claim 16 A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: determining, for a current block of a video, motion information associated with the current block; refining, based on a first coding tool, the motion information to acquire first motion information that is refined motion vector, wherein the first coding tool is a bi-directional optical flow coding tool and is used to acquire a first offset for refining the motion information; generating, based on the first motion information from the current block, a first prediction for the current block; updating, based on a second coding tool, the first prediction to acquire a second prediction, wherein the second coding tool is a bi-directional optical flow coding tool and is used to acquire second offset for updating the first prediction; and generating, based on the second prediction, the bitstream. ### Claim 17 (depends on 1) The method of claim 1, wherein the first offset is calculated using a spatial gradient between at least two pixels at a same picture, and/or a temporal gradient between at least two pixels at different pictures. ### Claim 18 (depends on 1) The method of claim 1, wherein the second offset is calculated using spatial gradient between at least two pixels at a same picture, and/or a temporal gradient between at least two pixels at different pictures. ### Claim 19 (depends on 9) The apparatus of claim 9, wherein the first offset is calculated using a spatial gradient between at least two pixels at a same picture, and/or a temporal gradient between at least two pixels at different pictures. ### Claim 20 (depends on 9) The apparatus of claim 9, wherein the second offset is calculated using spatial gradient between at least two pixels at a same picture, and/or a temporal gradient between at least two pixels at different pictures.