# US 12,003,716 B2 · Deblocking filter selection in video or image coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,716 B2
- Document ID: US12003716B2
- Status: Granted
- Kind: B2
- Application number: 18/082,100
- Filed: 2022-12-15
- Published: 2023-06-22
- Granted: 2024-06-04
- Earliest claimed: 2019-09-06
- Assignees: Telefonaktiebolaget LM Ericsson (publ)
- Applicants: Telefonaktiebolaget LM Ericsson (publ)
- Inventors: Kenneth Andersson; Jack Enhorn
- Web page: https://patentgrep.com/patent/US12003716B2

## CPC

- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/182: the unit being a pixel
- H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513)
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The long filter decision in VVC is modified by adding at least one gradient check that at least includes sample p6 or q6. This makes it possible to avoid using the long filters when there is some natural structure at sample p6 or q6. In one specific embodiment at least two gradient checks including both q6 and p6 are added. In another embodiment, the dpq threshold is modified from beta>>2 to beta>>4. This threshold change embodiment may be used in conjunction with or instead of the embodiment in which a gradient check that includes p6 and/or q6 is added to the long filter decision.

## Claims

### Claim 1

A method for deblocking filtering of vertical and/or horizontal block boundaries, the method being performed by an image or video encoder or an image or video decoder, the method comprising:
obtaining a first set of sample values comprising a first block of sample values and a second block of sample values, wherein the first block of sample values comprises a set of sample values B1_i,k for i=0-7 and k=0-3, the second block of sample values comprises a set of sample values B2_j,k for j=0-7 and k=0-3 and the second block has a maximum filter length, maxFilterLengthQ;
deciding whether or not to use a long filter to filter at least a subset of the first set of sample values, wherein deciding whether or not to use a long filter to filter at least a subset of the first set of sample values comprises:
calculating a first gradient value, maxGradB1_6,0 as Abs(B1_7,0−B1_6,0−B1_5,0+B1_4,0);
calculating a first pq value, pq1_0, using maxGradB1_6,0, wherein pq1_0=(sp+sq), wherein sp=(Abs(B1_0,0−B1_3,0)+Abs(B1_3,0−B1_7,0)+maxGradB1_6,0+1)>>1 and wherein:
if maxFilterLengthQ is equal to 7, then sq=(Abs(B2_0,0−B2_3,0)+Abs(B2_3,0−B2_7,0)+maxGradB2_6,0+1)>>1, where maxGradB2_6,0 is a third gradient value,
if maxFilterLengthQ is equal to 5, then sq=(Abs(B2_0,0−B2_3,0)+Abs(B2_3,0−B2_5,0)+1)>>1, and
if maxFilterLengthQ is not equal to 7 and is not equal to 5, then sq=Abs(B2_0,0−B2_3,0);
comparing pq1_0 to a threshold, wherein the threshold is based on a value β that is selected from a set of predetermined beta values based on a quantization parameter value, Q;
calculating a second gradient value, maxGradB1_6,3, using sample value B1_6,3;
calculating a second pq value, pq1_3, using maxGradB1_6,3; and
comparing pq1_3 to the threshold; and
applying the long filter to filter at least the subset of the first set of sample values, as a result of deciding to use the long filter.

### Claim 2 (depends on 1)

The method of claim 1, wherein maxGradB2_6,0 equals Abs(B2_7,0−B2_6,0−B2_5,0+B2_4,0).

### Claim 3 (depends on 1)

The method of claim 1, wherein pq1_3=(sp3+sq3), and wherein sp3=(Abs(B1_0,3−B1_3,3)+Abs(B1_3,3−B1_7,3)+maxGradB1_6,3+1)>>1.

### Claim 4 (depends on 3)

The method of claim 3, wherein maxGradB1_6,3 equals Abs(B1_7,3−B1_6,3−B1_5,3+B1_4,3).

### Claim 5 (depends on 1)

The method of claim 1, wherein
the step of calculating pq1_0 using maxGradB1_6,0 is performed as a result of determining that a maximum filter length for the first block, maxFilterLengthP, is equal to 7.

### Claim 6 (depends on 1)

The method of claim 1, wherein
the step of calculating pq1_3 using maxGradB1_6,3 is performed as a result of determining that a max filter length for the first block, maxFilterLengthP, is equal to 7.

### Claim 7 (depends on 1)

The method of claim 1, wherein
when deblocking a vertical boundary, the i value identifies a column of a block and the k value identifies a row of the block, and
when deblocking a horizontal boundary, the k value identifies a column of the block and the i value identifies a row of the block.

### Claim 8 (depends on 1)

The method of claim 1, wherein the samples to filter by the long filter when maxFilterLengthP is equal to 7 are B1_0,0 to B1_6,0, B1_0,1 to B1_6,1, B1_0,2 to B1_6,2 and B1_0,3 to B1_6,3.

### Claim 9 (depends on 1)

The method of claim 1, wherein the samples to filter by the long filter when maxFilterLengthQ is equal to 7 are B2_0,0 to B2_6,0, B2_0,1 to B2_6,1, B2_0,2 to B2_6,2 and B2_0,3 to B2_6,3.

### Claim 10 (depends on 1)

A non-transitory computer readable medium storing a computer program comprising instructions which when executed by processing circuitry causes the processing circuitry to perform the method of claim 1.

### Claim 11

An apparatus for deblocking filtering of vertical and/or horizontal block boundaries, the apparatus comprising:
memory; and
processing circuitry coupled to the memory, wherein the apparatus is configured to perform a method comprising:
obtaining a first set of sample values comprising a first block of sample values and a second block of sample values, wherein the first block of sample values comprises a set of sample values B1_i,k for i=0-7 and k=0-3, the second block of sample values comprises a set of sample values B2_j,k for j=0-7 and k=0-3 and the second block has a maximum filter length, maxFilterLengthQ;
deciding whether or not to use a long filter to filter at least a subset of the first set of sample values, wherein deciding whether or not to use a long filter to filter at least a subset of the first set of sample values comprises:
calculating a first gradient value, maxGradB1_6,0 as Abs(B1_7,0−B1_6,0−B1_5,0+B1_4,0);
calculating a first pq value, pq1_0, using maxGradB1_6,0, wherein pq1_0=(sp+sq), wherein sp=(Abs(B1_0,0−B1_3,0)+Abs(B1_3,0−B1_7,0)+maxGradB1_6,0+1)>>1 and wherein:
if maxFilterLengthQ is equal to 7, then sq=(Abs(B2_0,0−B2_3,0)+Abs(B2_3,0−B2_7,0)+maxGradB2_6,0+1)>>1, where maxGradB2_6,0 is a third gradient value,
if maxFilterLengthQ is equal to 5, then sq=(Abs(B2_0,0−B2_3,0)+Abs(B2_3,0−B2_5,0)+1)>>1, and
if maxFilterLengthQ is not equal to 7 and is not equal to 5, then sq=Abs(B2_0,0−B2_3,0);
comparing pq1_0 to a threshold, wherein the threshold is based on a value β that is selected from a set of predetermined beta values based on a quantization parameter value, Q;
calculating a second gradient value, maxGradB1_6,3, using sample value B1_6,3;
calculating a second pq value, pq1_3, using maxGradB1_6,3; and
comparing pq1_3 to the threshold; and
applying the long filter to filter at least the subset of the first set of sample values, as a result of deciding to use the long filter.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein maxGradB2_6,0 equals Abs(B2_7,0−B2_6,0−B2_5,0+B2_4,0).

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein pq1_3=(sp3+sq3), and wherein sp3=(Abs(B1_0,3−B1_3,3)+Abs(B1_3,3−B1_7,3)+maxGradB1_6,3+1)>>1.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein maxGradB1_6,3 equals Abs(B1_7,3−B1_6,3−B1_5,3+B1_4,3).

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein
the step of calculating pq1_0 using maxGradB1_6,0 is performed as a result of determining that a maximum filter length for the first block, maxFilterLengthP, is equal to 7.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein
the step of calculating pq1_3 using maxGradB1_6,3 is performed as a result of determining that a max filter length for the first block, maxFilterLengthP, is equal to 7.

### Claim 17 (depends on 11)

The apparatus of claim 11, wherein
when deblocking a vertical boundary, the i value identifies a column of a block and the k value identifies a row of the block, and
when deblocking a horizontal boundary, the k value identifies a column of the block and the i value identifies a row of the block.

### Claim 18 (depends on 11)

The apparatus of claim 11, wherein the samples to filter by the long filter when maxFilterLengthP is equal to 7 are B1_0,0 to B1_6,0, B1_0,1 to B1_6,1, B1_0,2 to B1_6,2 and B1_0,3 to B1_6,3.

### Claim 19 (depends on 11)

The apparatus of claim 11, wherein the samples to filter by the long filter when maxFilterLengthQ is equal to 7 are B2_0,0 to B2_6,0, B2_0,1 to B2_6,1, B2_0,2 to B2_6,2 and B2_0,3 to B2_6,3.