# US 11,122,270 B2 · In-loop reshaping with local illumination compensation in image coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,122,270 B2
- Document ID: US11122270B2
- Status: Granted
- Kind: B2
- Application number: 16/892,976
- Filed: 2020-06-04
- Published: 2020-12-10
- Granted: 2021-09-14
- Earliest claimed: 2019-06-05
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Jay Nitin Shingala; Ashwin Natesan; Peng Yin
- Web page: https://patentgrep.com/patent/US11122270B2

## CPC

- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/186: the unit being a colour or a chrominance component
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, processes, and systems are presented for combining signal reshaping (also referred to as luma mapping chroma residuals scaling) with local illumination compensation (LIC) in video coding. Examples and trade-offs when the LIC model parameters are computed in the original domain, the reshaped domain, or a mixed domain, are presented.

## Claims

### Claim 1

A method to combine in-loop reshaping with local illumination compensation (LIC) with a processor, the method comprising:
accessing first neighboring samples of a current coded unit (CU) in a reshaped domain;
accessing neighboring samples of a reference block to the current CU in an original domain;
generating second neighboring samples of the current CU in the original domain by applying an inverse mapping function to the first neighboring samples of the current CU;
generating LIC parameters based on the second neighboring samples of the current CU and the neighboring samples of the reference block to the current CU;
accessing first inter-predicted samples for the current CU in the original domain;
generating second inter-predicted samples for the current CU based on the LIC parameters, a linear mapping using the LIC parameters, and the first inter-predicted samples for the current CU; and
generating output inter-predicted samples for the current CU in the reshaped domain by applying a forward mapping function to the second inter-predicted samples for the current CU.

### Claim 2

A method to combine in-loop reshaping with local illumination compensation (LIC) with a processor, the method comprising:
accessing neighboring samples of a current coded unit (CU) in a reshaped domain;
accessing first neighboring samples of a reference block to the current CU in an original domain;
generating second neighboring samples of the reference block to the current CU in the reshaped domain by applying a forward mapping function to the second neighboring samples of the reference block to the current CU;
generating LIC parameters based on the neighboring samples of the current CU and the second neighboring samples of the reference block to the current CU;
accessing first inter-predicted samples for the current CU in the original domain;
generating second inter-predicted samples for the current CU in the reshaped domain by applying the forward mapping function to the first inter-predicted samples for the current CU; and
generating output inter-predicted samples for the current CU in the reshaped domain based on the LIC parameters, a linear mapping using the LIC parameters, and the second inter-predicted samples for the current CU.

### Claim 3

A method to combine in-loop reshaping with local illumination compensation (LIC) with a processor, the method comprising:
accessing first neighboring samples of a current coded unit (CU) in a reshaped domain;
accessing first neighboring samples of a reference block to the current CU in an original domain;
generating second neighboring samples of the reference block to the current CU in the reshaped domain by applying a forward mapping function to the second neighboring samples of the reference block to the current CU;
generating LIC parameters based on the first neighboring samples of the current CU and the second neighboring samples of the reference block to the current CU;
accessing first inter-predicted samples for the current CU in the original domain;
generating second inter-predicted samples for the current CU in the original domain based on the LIC parameters, a linear mapping using the LIC parameters, and the first inter-predicted samples for the current CU; and
generating output inter-predicted samples for the current CU in the reshaped domain by applying the forward mapping function to the second inter-predicted samples for the current CU.

### Claim 4 (depends on 1)

The method of claim 1, further comprising, combining the output inter-predicted samples for the current CU in the reshaped domain with received residual luma samples for the current CU in the reshaped domain to generate first reconstructed luma samples for the current CU in the reshaped domain.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
generating second reconstructed luma samples for the current CU in the original domain by applying an inverse reshaping function to the first reconstructed luma samples for the current CU; and
generating decoded luma samples for the current CU based on the second reconstructed luma samples for the current CU.

### Claim 6 (depends on 3)

The method of claim 3, wherein the LIC parameters comprise two parameters and both parameters are derived based on the neighboring samples of the current CU and the second neighboring samples of the reference block to the current CU.

### Claim 7 (depends on 3)

The method of claim 3, wherein the LIC parameters comprise a first parameter and a second parameter, wherein the first parameter is derived based on the neighboring samples of the current CU and the second neighboring samples of the reference block to the current CU, and computing the second parameter comprises:
generating second neighboring samples of the current CU in the original domain by applying an inverse mapping function to the first neighboring samples of the current CU; and
generating the second parameter based on the second neighboring samples of the current CU and the first neighboring samples of the reference block to the current CU.

### Claim 8 (depends on 1)

An apparatus comprising a processor and configured to perform the method of claim 1.

### Claim 9 (depends on 1)

A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with claim 1.