# US 2021/0243435 A1 · Sub-block motion candidate list in video coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243435 A1 - Document ID: US20210243435A1 - Status: Published application - Kind: A1 - Application number: 17/238,548 - Filed: 2021-04-23 - Published: 2021-08-05 - Earliest claimed: 2018-10-24 - Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc. - Inventors: Li ZHANG; Kai ZHANG; Hongbin LIU; Yue WANG - Web page: https://patentgrep.com/patent/US20210243435A1 ## CPC - H04N 19/52: by predictive encoding - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/184: the unit being bits, e.g. of the compressed video stream - H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) - H04N 19/513: Processing of motion vectors - H04N 19/54: using feature points or meshes - H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search - H04N 19/61: in combination with predictive coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Devices, systems and methods for the simplification of sub-block motion candidate lists for video coding are described. In a representative aspect, a method for video processing includes determining, for a conversion between a current block of a video and a bitstream representation of the video, a sub-block level merge candidate list that includes at least one sub-block coding type. The method also includes performing the conversion based on the sub-block level merge candidate list. ## Claims ### Claim 1 A method for processing video data, comprising: determining, for a conversion between a current video block of a video and a bitstream of the video, a sub-block level merge candidate list that includes at least one sub-block coding type; and performing the conversion based on the sub-block level merge candidate list, wherein motion candidates of the at least one sub-block coding type are added to the sub-block level merge candidate list based on an adaptive ordering. ### Claim 2 (depends on 1) The method of claim 1, wherein the at least one sub-block coding type comprises a sub-block based temporal motion vector prediction coding type and an affine motion prediction coding type. ### Claim 3 (depends on 1) The method of claim 1, wherein each of at least one sub-block coding type is assigned with a range of merge indices, and wherein the merge index within a first range corresponds to the sub-block based temporal motion vector prediction coding type, and the first range includes a single value of 0. ### Claim 4 (depends on 3) The method of claim 3, wherein the merge index within a second range corresponds to the affine motion prediction coding type, and wherein the second range excludes a value of 0. ### Claim 5 (depends on 1) The method of claim 1, wherein a motion candidate of the sub-block based temporal motion vector prediction coding type is always available in the sub-block level merge candidate list, and wherein temporal information is only allowed to derive motion candidates of the sub-block based temporal motion vector prediction coding type. ### Claim 6 (depends on 1) The method of claim 1, wherein the range of merge indices for a coding type is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a tile group header, a coding tree unit (CTU), a tile, a coding unit (CU), a prediction unit (PU) or a CTU row, and the range of merge indices for a coding type is based on a width or a height of the current video block. ### Claim 7 (depends on 1) The method of claim 1, wherein the adaptive ordering indicates that a motion candidate of the sub-block based temporal motion vector prediction coding type is added to the sub-block level merge candidate list prior to a motion candidate of the affine motion prediction coding type, or the adaptive ordering indicates a motion candidate of the sub-block based temporal motion vector prediction coding type and a motion candidate of the affine motion prediction type are added to the sub-block level merge candidate list in an interleaved manner. ### Claim 8 (depends on 1) The method of claim 1, wherein the adaptive ordering is based on coded information of the current video block or neighboring blocks of the current video block, and wherein, in case a majority of the neighboring blocks of the current video block is affine coded, the adaptive ordering indicates that a motion candidate of the affine motion prediction coding type is added to the sub-block level merge candidate list prior to motion candidates of other types. ### Claim 9 (depends on 1) The method of claim 1, wherein the adaptive ordering is based on a ratio of affine motion candidates to non-affine motion candidates in the sub-block level merge candidate list, and wherein, in case the ratio is greater than a threshold, the adaptive ordering indicates that a motion candidate of the affine motion coding type is added to the sub-block level merge candidate list prior to motion candidates of other types. ### Claim 10 (depends on 1) The method of claim 1, wherein the adaptive ordering is applicable to first K affine motion candidates in the sub-block level merge candidate list, K being a positive integer. ### Claim 11 (depends on 1) The method of claim 1, wherein the adaptive ordering is signaled by in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a tile group header, a coding tree unit (CTU), a tile, a coding unit (CU), a prediction unit (PU) or a CTU row. ### Claim 12 (depends on 1) The method of claim 1, further comprising: determining, for the conversion, a sub-block level coding technique based on an indication that is signaled in a picture header, a picture parameter set (PPS), a slice header, or a tile group header; and performing the conversion at least based on the sub-block level coding technique. ### Claim 13 (depends on 12) The method of claim 12, wherein the sub-block level coding technique comprises a sub-block based temporal motion vector prediction coding technique and an affine coding technique. ### Claim 14 (depends on 12) The method of claim 12, wherein the indication indicates that the sub-block coding technique is disabled. ### Claim 15 (depends on 1) The method of claim 1, further comprising: determining, for the conversion, a sub-block level temporal motion candidate using a derivation process applicable to a block level temporal motion vector prediction candidate conversion between the current video block and the bitstream; and performing the conversion at least based on the sub-block level temporal motion candidate. ### Claim 16 (depends on 1) The method of claim 1, further comprising: determining, for the conversion, a block level temporal motion vector prediction candidate using a derivation process applicable to a sub-block level temporal motion candidate conversion between the current video block and the bitstream; and performing the conversion at least based on the block level temporal motion vector prediction candidate. ### Claim 17 (depends on 1) The method of claim 1, wherein the conversion generates the current video block from the bitstream. ### Claim 18 (depends on 1) The method claim 1, wherein the conversion generates the bitstream from the current video block. ### Claim 19 An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: determine, for a conversion between a current video block of a video and a bitstream of the video, a sub-block level merge candidate list that includes at least one sub-block coding type; and perform the conversion based on the sub-block level merge candidate list, wherein motion candidates of the at least one sub-block coding type are added to the sub-block level merge candidate list based on an adaptive ordering. ### Claim 20 A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: determining, for a conversion between a current video block of a video and a bitstream of the video, a sub-block level merge candidate list that includes at least one sub-block coding type; and generating the bitstream based on the sub-block level merge candidate list, wherein motion candidates of the at least one sub-block coding type are added to the sub-block level merge candidate list based on an adaptive ordering.