# US 2019/0320191 A1 · Chroma Reshaping for High Dynamic Range Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320191 A1
- Document ID: US20190320191A1
- Status: Published application
- Kind: A1
- Application number: 16/303,028
- Filed: 2017-05-17
- Published: 2019-10-17
- Earliest claimed: 2016-05-19
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Qing SONG; Guan-Ming SU; Qian CHEN; Tao CHEN
- Web page: https://patentgrep.com/patent/US20190320191A1

## CPC

- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/124: Quantisation
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N 19/98: Adaptive-dynamic-range coding [ADRC]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods and systems for adaptive chroma reshaping are discussed. Given an input image, a luma-reshaped image is first generated based on its luma component. For each chroma component of the input image, the range of the pixel values in the luma reshaped image is divided into bins, and for each bin a maximal scale factor is generated based on the chroma pixel values in the input image corresponding to the pixels of the luma reshaped image in the bin. A forward reshaping function is generated based on a reference reshaped function and the maximal scale factors, and reshaped chroma pixel values for the chroma component are generated based on the forward reshaping function and the corresponding pixel values in the luma reshaped image. Implementations options using look-up tables for mobile platforms with limited computational resources are also described.

## Claims

### Claim 1

A method for adaptive chroma reshaping of a high dynamic range (HDR) image with a processor, the method comprising:
accessing with the processor an HDR input image comprising pixel values in a luma component and one or more chroma components;
for a chroma component of the input image:
partitioning the pixel values of either the luma component of the HDR input image or a reshaped luma component of the HDR input image into two or more bins;
generating for each bin a maximal scale factor based on the chroma pixel values corresponding to the respective luma or reshaped luma pixel values in each bin;
generating a forward reshaping function based on a reference reshaping function and the maximal scale factors, the forward reshaping function mapping either luma pixel values or reshaped luma pixel values to forward chroma scale factors and correspondingly the reference reshaping function mapping luma pixel values or reshaped luma pixel values to reference chroma scale factors; and
generating reshaped chroma pixel values for the chroma component based on the chroma pixel values, the forward reshaping function, and the corresponding luma pixel values or reshaped luma pixel values of the HDR input image.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating an inverse reshaping function based on the forward reshaping function;
generating parameters characterizing the inverse reshaping function; and
generating and outputting metadata based on the parameters characterizing the inverse reshaping function.

### Claim 3 (depends on 1)

The method of claim 1, wherein for each bin, for respective luma pixel values or reshaped luma pixel values in the bin, the maximal scale factor is computed based on a tth-percentile value of a function of the corresponding chroma values in the bin.

### Claim 4 (depends on 3)

The method of claim 3, wherein for pixel values normalized in [0 1), generating the maximal scale factor for the m-th bin comprises computing
Ψ
j
,
m
=
{
i

m
M
≤
s
^
ji
y
<
m
+
1
M
}
b
j
,
m
=
η
·
p
i

(
{

v
ji
u
-
0.5


(
i
∈
Ψ
j
,
m
)
}
)
q
~
j
,
m
=
min

{
0.5
b
j
,
m
,
q
max
}
where M is the number of bins, Ψj,m denotes the m-th bin for the HDR input image j, ŝjiy denotes the i-th pixel of the respective luma component or the reshaped luma component of the HDR input image j, vjiu denotes the corresponding chroma pixel for the chroma component of the HDR input image j, η is a weighting factor, qmax is a fixed value, and pt(x) returns the tth-percentile value of the values in vector x, wherein optionally: t=100 and η is between 0.85 and 0.95, or η=1 and t is between 95 and 99.

### Claim 5 (depends on 3)

The method of claim 3, wherein generating the forward reshaping function {tilde over (ƒ)}j( ) for the HDR input image j comprises computing
{
α
~
jk
*
}
=
arg
{
α
~
jk
}


min

(
∑
m
=
0
M
-
1



r
m
-
f
~
j

(
x
m
)

2
)
,
subject to
0<{tilde over (ƒ)}j(xm)≤{tilde over (q)}j,m for all m,
wherein M is the number of bins, {tilde over (q)}j,m denotes the maximal scale factor for the m-th bin,
r
m
=
f
r

(
x
m
)
=
f
r

(
2

m
+
1
2

M
)
,
for m=0, . . . , M−1, where ƒr( ) denotes the reference reshaping function, ŝjiy denotes the i-th pixel of the respective luma component or reshaped luma component of the HDR input image, and
f
~
j

(
s
^
ji
y
)
=
∑
k
=
0
K
F


α
~
jk
·
(
s
^
jk
y
)
k
,
where KF denotes the order of the polynomial of the forward reshaping function.

### Claim 6 (depends on 5)

The method of claim 5, wherein generating the forward reshaping function ({tilde over (ƒ)}j( )) comprises computing
{
α
~
jk
*
}
=
arg
{
α
~
jk
}


min

(
∑
m
=
0
M
-
1


w
j
,
m
·

r
m
-
f
~
j

(
x
m
)

2
)
,
subject to 0<{tilde over (ƒ)}j(xm)≤{tilde over (q)}j,m,
where wj,m are weights for the m-th bin for the HDR input image j.

### Claim 7 (depends on 6)

The method of claim 6, wherein for the m-th bin, weight wj,m is proportional to the number of pixel values of the respective luma component or reshaped luma component of the HDR input image j in the m-th bin.

### Claim 8 (depends on 6)

The method of claim 6, wherein the forward reshaping function {tilde over (ƒ)}j( ) is generated based on the starting point of the reference reshaping function and a second point to be crossed by the forward reshaping function, wherein the second point corresponds to the coordinates for which the value of all the maximal scale values that are below the reference reshaping function has the maximum distance from the reference reshaping function, and optionally
wherein computing the second point (xT,{tilde over (q)}j,T) comprises computing
T
=
arg


min
m


(
q
~
j
,
m
-
r
m
)
.

### Claim 9 (depends on 1)

The method of claim 1, wherein for pixel values normalized in [0 1), generating the reshaped chroma values ŝjiu for the color component (u) of the HDR input image (j) comprises computing:
ŝjiu=ƒj(ŝjiy)·(vjiu−0.5)+0.5,
where ƒj(ŝjiy) denotes the output of the forward reshaping function for pixel value ŝjiy of the respective luma component or reshaped luma component of the i-th pixel of the HDR input image, and vjiu denotes the corresponding chroma pixel value for the color component of the i-th pixel of the HDR input image.

### Claim 10 (depends on 2)

The method of claim 2, wherein the parameters characterizing the inverse reshaping function are generated to match the syntax of a third-order multivariate, multiple regression representation for a prediction function.

### Claim 11 (depends on 3)

The method of claim 3 wherein generating the forward reshaping function comprises:
computing (xT, yn*) coordinates characterizing a desired crossing point of the forward reshaping function, wherein the xT coordinate corresponds to a respective luma pixel value or reshaped luma pixel value for which the value of the maximal scale values that are below the reference reshaping function has the maximum distance from the reference reshaping function; and
generating the polynomial coefficients of the forward reshaping function based on the (xT, yn*) point coordinates and a first look-up table comprising polynomial coefficients for a plurality of forward reshaping functions.

### Claim 12 (depends on 11)

The method of claim 11, wherein computing the (xT, yn*) coordinates comprises computing
T
=
arg


min
m


(
q
~
j
,
m
-
r
m
)
,
where m denotes the m-th bin, {tilde over (q)}j,m denotes the maximal scale factor for the m-th bin,
r
m
=
f
r

(
x
m
)
=
f
r

(
2

m
+
1
2

M
)
,
for m=0, . . . , M−1, where ƒr( ) denotes the reference reshaping function, M denotes the total number of bins, and
n
*
=
arg


min
n



q
~
j
,
T
-
y
n

,
where yn, for n=1, 2, . . . , N−1, represents the center of the n-th interval, when the range [1.0, rmax] is subdivided into N intervals, where rmax denotes a maximum of the reference reshaping function.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
generating parameters of an inverse reshaping function based on the (xT, yn*) point coordinates and a second look-up table comprising polynomial coefficients for a plurality of inverse reshaping functions; and
generating and outputting metadata based on the parameters characterizing the inverse reshaping function.

### Claim 14 (depends on 1)

An apparatus comprising a processor and configured to perform the method according to claim 1.

### Claim 15 (depends on 1)

A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors according to claim 1.