# US 2024/0179301 A1 · Improved Prediction with Local Illumination Compensation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0179301 A1
- Document ID: US20240179301A1
- Status: Published application
- Kind: A1
- Application number: 18/513,101
- Filed: 2023-11-17
- Published: 2024-05-30
- Earliest claimed: 2022-11-18
- Assignees: Comcast Cable Communications, LLC
- Applicants: Comcast Cable Communications, LLC
- Inventors: Alexey Konstantinovich Filippov; Vasily Alexeevich Rufitskiy; Esmael Hejazi Dinan
- Web page: https://patentgrep.com/patent/US20240179301A1

## CPC

- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- G06T 5/20: using local operators
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/593: involving spatial prediction techniques
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
- G06T 2207/10016: Video; Image sequence
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A current block to be encoded and candidate reference block templates may be determined from the same picture. A bitstream may include an indication of whether to use local illumination compensation (LIC) on the current block. Based on the indication, differences between the current block and the candidate reference blocks may be determined. Based on the differences, a reference block may be determined, and a prediction of the current block may be made based on the indication and the difference between the current block and the reference block.

## Claims

### Claim 1

A method comprising:
receiving, in a bitstream, an indication of whether to use a local illumination compensation;
based on the indication, determining differences between a corresponding template of each candidate reference block of a plurality of candidate reference blocks and a template of a current block; and
based on the determined differences, predicting the current block by applying the local illumination compensation to a reference block, from the plurality of candidate reference blocks, in accordance with the indication.

### Claim 2 (depends on 1)

The method of claim 1, wherein the determining the differences comprises:
based on the indication indicating to use the local illumination compensation, determining the differences using a first difference calculation that is different from a second difference calculation for calculating the differences based on the indication indicating not to use the local illumination compensation.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first difference calculation is less sensitive to illumination differences than the second difference calculation.

### Claim 4 (depends on 2)

The method of claim 2, wherein the first difference calculation comprises using a formula for a sum of absolute differences (SAD), and wherein the second difference calculation comprises using one of: a mean-removed SAD (MR-SAD), Hadamard absolute differences (HAD), or a sum of absolute transformed differences (SATD).

### Claim 5 (depends on 1)

The method of claim 1, wherein the determining the differences comprises:
based on the indication indicating to use the local illumination compensation process and for each candidate reference block of the plurality of candidate reference blocks, determining local illumination compensation parameters based on:
the template of each candidate reference block; and
the template of the current block;
for each candidate reference block of the plurality of candidate reference blocks:
applying, based on the determined local illumination compensation parameters, the local illumination compensation to the template of the candidate reference block; and
after the applying, determining the differences using a sum of absolute differences (SAD).

### Claim 6 (depends on 1)

The method of claim 1, wherein the current block and the plurality of candidate reference blocks are within a picture, and wherein the determining the differences comprises:
determining, within a predefined search region of the picture, each candidate reference block of the plurality of candidate reference blocks by comparing, based on the indication, the template of each candidate reference block with the template of the current block.

### Claim 7 (depends on 1)

The method of claim 1, wherein the determining the differences comprises:
based on block vectors of a predetermined set of neighboring blocks of the current block, determining the plurality of candidate reference blocks; and
determining, based on the differences, an ordering sequence of the candidate reference blocks,
wherein the predicting the current block is further based on the determined sequence.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
determining a cost for each of a plurality of block vector difference (BVD) candidates comprising a first BVD candidate and a second BVD candidate, wherein each of the BVD candidates correspond to a respective one of the plurality of candidate reference blocks, and wherein a value of a magnitude symbol of the first BVD candidate is different from a value of a magnitude symbol of the second BVD candidate;
based on the determined costs, selecting one of the plurality of BVD candidates as a BVD predictor;
decoding a second indication indicating whether a value of a magnitude symbol of a BVD matches a value of a magnitude symbol of the BVD predictor; and
based on the value of the magnitude symbol of the BVD predictor and the second indication, determining a value of the magnitude symbol of the BVD.

### Claim 9 (depends on 1)

The method of claim 1, wherein the indication is a flag associated with the current block.

### Claim 10 (depends on 1)

The method of claim 1, wherein the bitstream includes a second indication, associated with the current block, indicating the use of an intra block copy (IBC) mode.

### Claim 11 (depends on 1)

The method of claim 1, wherein the bitstream includes a second indication, associated with the current block, indicating the use of a template matching prediction (intra TMP) mode.

### Claim 12 (depends on 1)

The method of claim 1, wherein the predicting the current block is further based on a prediction error received from the bitstream.

### Claim 13

A method comprising:
based on whether a local illumination compensation is to be applied, determining differences between a corresponding template of each candidate reference block of a plurality of candidate reference blocks and a template of a current block, wherein the current block and the plurality of candidate reference blocks are within a picture;
based on the determined differences, predicting the current block by applying the local illumination compensation to a reference block, from the plurality of candidate reference blocks, in accordance with an indication of whether the local illumination compensation process applies to the reference block; and
sending, in a bitstream, a prediction error associated with the reference block and the indication of whether the local illumination compensation process applies to the reference block.

### Claim 14 (depends on 13)

The method of claim 13, wherein the determining the differences comprises:
based on the indication indicating that the local illumination compensation applies to the reference block, determining the differences using a first difference calculation that is different from a second difference calculation for calculating the differences based on the indication indicating not to use the local illumination compensation.

### Claim 15 (depends on 13)

The method of claim 13, wherein the determining the differences comprises:
determining a block vector predictor (BVP) and a magnitude of a block vector difference (BVD) for the current block;
based on the BVP and the magnitude of the BVD, determining the plurality of candidate reference blocks; and
determining a sign of the BVD, and wherein the predicting the current block is further based on the determined sign.

### Claim 16 (depends on 13)

The method of claim 13, further comprising:
determining a block vector difference (BVD) based on a difference between a block vector (BV) and a block vector predictor (BVP);
determining a cost for each of a plurality of BVD candidates comprising a first BVD candidate and a second BVD candidate, wherein each of the BVD candidates correspond to a respective one of the plurality of candidate reference blocks, and wherein a value of a magnitude symbol of the first BVD candidate is different from a value of a magnitude symbol of the second BVD candidate;
based on the determined costs, selecting one of the plurality of BVD candidates as a BVD predictor; and
encoding an indication indicating whether a value of a magnitude symbol of the BVD matches a value of the magnitude symbol of the BVD predictor.

### Claim 17

A method comprising:
determining a plurality of candidate reference blocks in accordance with block vectors of a predetermined set of neighbor blocks of a current block;
based on an application of a local illumination compensation, determining differences between a corresponding template of each candidate reference block from the plurality of candidate reference blocks and a template of the current block, wherein the current block and the plurality of candidate reference blocks are within a picture;
determining an ordering sequence of the candidate reference blocks in accordance with the determined differences;
based on the determined differences, predicting the current block based on a reference block and the ordering sequence, wherein the reference block is one of the plurality of candidate reference blocks, and wherein the predicting comprises applying the local illumination compensation to the reference block based on an indication of whether the local illumination compensation process applies to the reference block; and
sending, in a bitstream, a prediction error associated with the reference block and the indication of whether the local illumination compensation process applies.

### Claim 18 (depends on 17)

The method of claim 17, wherein the indication is a flag associated with the current block.

### Claim 19 (depends on 17)

The method of claim 17, wherein the bitstream includes a second indication, associated with the current block, indicating use of an intra block copy (IBC) mode.

### Claim 20 (depends on 17)

The method of claim 17, wherein the bitstream includes a second indication, associated with the current block, indicating use of a template matching prediction (intra TMP) mode.
