# US 2025/0220179 A1 · Sub-block based prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0220179 A1
- Document ID: US20250220179A1
- Status: Published application
- Kind: A1
- Application number: 19/085,308
- Filed: 2025-03-20
- Published: 2025-07-03
- Earliest claimed: 2018-10-22
- Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Inventors: Hongbin Liu; Li Zhang; Kai Zhang; Yue Wang
- Web page: https://patentgrep.com/patent/US20250220179A1

## CPC

- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/513: Processing of motion vectors
- H04N 19/521: {for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for video processing includes: determining, that the video block is split into a plurality of sub-blocks based on a condition associated with the video block, refining motion information of each sub-block using a motion refinement process, the motion refinement process being a one-step or an iterative multi-step refinement process; and performing a video processing on the video block based on the refined motion information of each sub-block.

## Claims

### Claim 1

A method for video processing, comprising:
selectively using a motion refinement approach to refine motion information of a video block based on at least one of the video block and a reference picture for coding the video block; and
performing a conversion between the video block of a video and a bitstream of the video based on refined motion information of the video block.

### Claim 2 (depends on 1)

The method of claim 1, wherein the motion refinement approach comprises at least one of bi-directional optical flow (BIO) and decoder side motion vector refinement (DMVR).

### Claim 3 (depends on 1)

The method of claim 1, wherein the motion refinement approach is not used if the reference picture is a current coding picture.

### Claim 4 (depends on 1)

The method of claim 1, wherein the motion refinement approach comprises a multi-step refinement process, and the method further comprises:
using refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, to generate at least one motion compensated reference block of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and using original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to generate at least one motion compensated reference block of 1st step; or
using refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, as a start searching point to derive refined MVs of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and using original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to derive refined MVs of 1st step;
wherein LX=L0 or L1, L0 and L1 representing reference list 0 and list 1 respectively.

### Claim 5 (depends on 4)

The method of claim 4, wherein short-tap filters are used in at least one of a 1st to (N−1)th steps of the multi-step refinement process.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
using the refined MVs of an Nth step of the multi-step refinement process to perform motion compensation for the video block, and generating a final prediction of the video block based on the motion compensation.

### Claim 7 (depends on 4)

The method of claim 4, wherein the multi-step refinement process is not used if the video block has a minimum size of a width and height which is not larger than 8.

### Claim 8 (depends on 4)

The method of claim 4, wherein the multi-step refinement process is not used if at least one of a width and height of the video block is not less than 64.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
using partial pixels of the video block or sub-blocks within the video block to calculate partial temporal and/or spatial gradients of the video block or the sub-blocks within the video block; and
deriving, refined motion vectors of the video block or the sub-blocks within the video block, using the partial temporal and/or spatial gradients.

### Claim 10 (depends on 9)

The method of claim 9, wherein pixels from every N rows and/or columns of top-left/top-right/bottom-left/bottom-right quarter of the video block or the sub-blocks within the video block are used to calculate the partial temporal and/or the spatial gradients.

### Claim 11 (depends on 1)

The method of claim 1, wherein the conversion includes encoding the video block into the bitstream.

### Claim 12 (depends on 1)

The method of claim 1, wherein the conversion includes decoding the video block from the bitstream.

### Claim 13

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
selectively use a motion refinement approach to refine motion information of a video block based on at least one of the video block and a reference picture for coding the video block; and
perform a conversion between the video block of a video and a bitstream of the video based on refined motion information of the video block.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the motion refinement approach comprises at least one of Bi-directional optical flow (BIO) and decoder side motion vector refinement (DMVR).

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the motion refinement approach is not used if the reference picture is a current coding picture.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the motion refinement approach comprises a multi-step refinement process, and the processor is further caused to:
use refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, to generate at least one motion compensated reference block of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and use original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to generate at least one motion compensated reference block of 1st step; or
use refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, as a start searching point to derive refined MVs of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and use original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to derive refined MVs of 1st step;
wherein LX=L0 or L1, L0 and L1 representing reference list 0 and list 1 respectively.

### Claim 17

A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
selectively using a motion refinement approach to refine motion information of a video block based on at least one of the video block and a reference picture for coding the video block; and
generating the bitstream based on refined motion information of the video block.

### Claim 18 (depends on 17)

The non-transitory computer-readable recording medium of claim 17, wherein the motion refinement approach comprises at least one of Bi-directional optical flow (BIO) and decoder side motion vector refinement (DMVR).

### Claim 19 (depends on 17)

The non-transitory computer-readable recording medium of claim 17, wherein the motion refinement approach is not used if the reference picture is a current coding picture.

### Claim 20 (depends on 17)

The non-transitory computer-readable recording medium of claim 17, wherein the motion refinement approach comprises a multi-step refinement process, and the method further comprises:
using refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, to generate at least one motion compensated reference block of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and using original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to generate at least one motion compensated reference block of 1st step; or
using refined MVs (MVLXi_x, MVLXi_y) of the video block, derived in ith step of the multi-step refinement process, as a start searching point to derive refined MVs of (i+1)th step, for i=1 to N−1, N being a total number of refinement steps performed during the multi-step refinement process and being greater than 1, and using original MVs (MVLX0_x, MVLX0_y), which are not subjected to refinement, to derive refined MVs of 1st step;
wherein LX=L0 or L1, L0 and L1 representing reference list 0 and list 1 respectively.
