# US 10,455,254 B2 · Method and apparatus of video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,455,254 B2
- Document ID: US10455254B2
- Status: Granted
- Kind: B2
- Application number: 15/806,632
- Filed: 2017-11-08
- Published: 2018-05-10
- Granted: 2019-10-22
- Earliest claimed: 2016-11-10
- Assignees: Mediatek Inc.
- Applicants: Mediatek Inc.
- Inventors: Han Huang; Shan Liu; Chih-Wei Hsu
- Web page: https://patentgrep.com/patent/US10455254B2

## CPC

- H04N 19/865: {with detection of the former encoding block subdivision in decompressed video}
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513)
- H04N 19/82: involving filtering within a prediction loop
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Aspects of the disclosure provide a method of video coding includes receiving input data associated with a first block and a second block of an image frame. The method further includes identifying a reference size and performing a deblocking process if it is determined that the deblocking process is to be performed. The preforming the deblocking process may include processing pixels adjacent to the block boundary using a first set of deblocking filter settings if a first block size of the first block and a second block size of the second block are greater than the reference size, and processing the pixels using a second set of deblocking filter settings if the first block size or the second block size is not greater than the reference size.

## Claims

### Claim 1

A method of video coding, comprising:
receiving input data associated with a first block and a second block of an image frame, the first block and the second block defining a block boundary therebetween, the block boundary extending along a first direction, the first block having a first block size along a second direction different from the first direction, and the second block having a second block size along the second direction;
identifying a reference size along the second direction for coding the image frame;
determining, by a processing circuit of a video coding apparatus, whether to perform a deblocking process on pixels adjacent to the block boundary; and
performing, by the processing circuit of the video coding apparatus, the deblocking process if it is determined that the deblocking process is to be performed, including
processing the pixels adjacent to the block boundary using a first set of deblocking filter settings if the first block size and the second block size are greater than the reference size, the first set of deblocking filter settings using at most a first number of pixels on each side of the block boundary; and
processing the pixels adjacent to the block boundary using a second set of deblocking filter settings if the first block size or the second block size is not greater than the reference size, the second set of deblocking filter settings using at most a second number of pixels on each side of the block boundary, and the second number being less than the first number,
wherein
the reference size is N pixels, N being a positive integer,
the second number is equal to or less than N/2,
when the second set of deblocking filter settings includes weak filter settings and strong filter settings, the processing the pixels if the first block size or the second block size is not greater the reference size comprises:
applying the strong filter settings if a first value representing first spatial activity corresponding to the pixels adjacent to the block boundary is less than a first threshold; and
applying the weak filter settings if the first value representing the first spatial activity corresponding to the pixels adjacent to the block boundary is equal to or greater than the first threshold, and
when the second set of deblocking filters includes unified filter settings that uses at most N/2 pixels on each side of the block boundary, the processing the pixels if the first block size or the second block size is not greater the reference size comprises:
applying the unified filter settings.

### Claim 2 (depends on 1)

The method of video coding according to claim 1, wherein the applying the weak filter settings is performed in a line-by-line basis and comprises:
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the weak filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the weak filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.

### Claim 3 (depends on 1)

The method of video coding according to claim 1, wherein
the weak filter settings of the second set of deblocking filters uses at most N/2 pixels on each side of the block boundary, and
the strong filter settings of the second set of deblocking filters uses at most N/2 pixels on each side of the block boundary.

### Claim 4 (depends on 1)

The method of video coding according to claim 1, wherein
a line of pixels along the second direction includes two pixels, p0 and p1, of the first block and two pixels, q0 and q1, of the second block,
p0 and q0 are adjacent to the block boundary,
p1 is next to p0,
q1 is next to q0,
the weak filter settings include generating a new value p0′ to replace p0 according to
p0′=(3*p1+7*p0 +9*q0−3*q1 +8)>>4, and
the strong filter settings include generating a new value p0′ to replace p0 according to
p0′=(p1+2*p0 +2*q0−q1+2)>>2.

### Claim 5 (depends on 1)

The method of video coding according to claim 1, wherein
the weak filter settings of the second set of deblocking filters uses at most N/4 sample on each side of the block boundary, and
the strong filter settings of the second set of deblocking filters uses at most N/2 pixels on each side of the block boundary.

### Claim 6 (depends on 1)

The method of video coding according to claim 1, wherein the applying the unified filter settings is performed in a line-by-line basis and comprises:
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the unified filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the unified filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.

### Claim 7 (depends on 1)

The method of video coding according to claim 1, wherein the reference size along the second direction for coding the image frame corresponds to a minimum block size along the second direction.

### Claim 8 (depends on 1)

The method of video coding according to claim 1, wherein
the input data include encoded video data of the image frame, and
the method further comprises extracting the reference size from the encoded video data.

### Claim 9 (depends on 1)

The method of video coding according to claim 1, further comprising:
generating encoded video data of the image frame, and
including in the encoded video data information regarding the reference size.

### Claim 10

A video coding apparatus, comprising:
a processing circuit configured to:
receive input data associated with a first block and a second block of an image frame, the first block and the second block defining a block boundary therebetween, the block boundary extending along a first direction, the first block having a first block size along a second direction different from the first direction, and the second block having a second block size along the second direction;
identify a reference size along the second direction for coding the image frame;
determine whether to perform a deblocking process on pixels adjacent to the block boundary; and
perform the deblocking process if it is determined that the deblocking process is to be performed, including
processing the pixels adjacent to the block boundary using a first set of deblocking filter settings if the first block size and the second block size are greater than the reference size, the first set of deblocking filter settings using at most a first number of pixels on each side of the block boundary; and
processing the pixels adjacent to the block boundary using a second set of deblocking filter settings if the first block size or the second block size is not greater than the reference size, the second set of deblocking filter settings using at most a second number of pixels on each side of the block boundary, and the second number being less than the first number,
wherein
the reference size is N pixels, N being a positive integer,
the second number is equal to or less than N/2,
the processing circuit is further configured to, when the second set of deblocking filter settings includes weak filter settings and strong filter settings, and the first block size or the second block size is not greater the reference size,
apply the strong filter settings if a first value representing first spatial activity corresponding to the block boundary is less than a first threshold, and
apply the weak filter settings if the first value representing the first spatial activity corresponding to the block boundary is equal to or greater than the first threshold, and
the processing circuit is further configured to, when the second set of deblocking filter settings includes unified filter settings that uses at most N/2 pixels on each side of the block boundary, and the first block size or the second block size is not greater the reference size,
apply the unified filter settings.

### Claim 11 (depends on 10)

The video coding apparatus according to claim 10, wherein the processing circuit is configured to apply the weak filter settings in a line-by-line basis by
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the weak filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the weak filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.

### Claim 12 (depends on 10)

The video coding apparatus according to claim 10, wherein the processing circuit is configured to apply the unified filter settings in a line-by-line basis by
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the unified filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the unified filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.

### Claim 13

A non-transitory computer readable medium storing program instructions for causing a processing circuit of an apparatus to perform a video coding method, and the method comprising:
receiving input data associated with a first block and a second block of an image frame, the first block and the second block defining a block boundary therebetween, the block boundary extending along a first direction, the first block having a first block size along a second direction different from the first direction, and the second block having a second block size along the second direction;
identifying a reference size along the second direction for coding the image frame;
determining whether to perform a deblocking process on pixels adjacent to the block boundary; and
performing the deblocking process if it is determined that the deblocking process is to be performed, including
processing the pixels adjacent to the block boundary using a first set of deblocking filter settings if the first block size and the second block size are greater than the reference size, the first set of deblocking filter settings using at most a first number of pixels on each side of the block boundary; and
processing the pixels adjacent to the block boundary using a second set of deblocking filter settings if the first block size or the second block size is not greater than the reference size, the second set of deblocking filter settings using at most a second number of pixels on each side of the block boundary, and the second number being less than the first number,
wherein
the reference size is N pixels, N being a positive integer,
the second number is equal to or less than N/2,
when the second set of deblocking filter settings includes weak filter settings and strong filter settings, the processing the pixels if the first block size or the second block size is not greater the reference size comprises:
applying the strong filter settings if a first value representing first spatial activity corresponding to the block boundary is less than a first threshold; and
applying the weak filter settings if the first value representing the first spatial activity corresponding to the block boundary is equal to or greater than the first threshold, and
when the second set of deblocking filters includes unified filter settings that uses at most N/2 pixels on each side of the block boundary, the processing the pixels if the first block size or the second block size is not greater the reference size comprises:
applying the unified filter settings.

### Claim 14 (depends on 13)

The non-transitory computer readable medium according to claim 13, wherein the applying the weak filter settings is performed in a line-by-line basis and comprises:
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the weak filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the weak filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.

### Claim 15 (depends on 13)

The non-transitory computer readable medium according to claim 13, wherein the applying the unified filter settings is performed in a line-by-line basis and comprises:
determining a second value representing second spatial activity corresponding to a line of pixels along the second direction;
processing a first pixel of the first block in the line that is adjacent to the block boundary using the unified filter settings;
processing a second pixel of the first block in the line that is next to the first pixel using the unified filter settings if the second value representing the second spatial activity is less than a second threshold; and
skipping processing the second pixel of the first block if the second value representing the second spatial activity is equal to or greater than the second threshold.