# US 10,123,019 B2 · Piecewise inter-layer prediction for signals with enhanced dynamic range

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,123,019 B2
- Document ID: US10123019B2
- Status: Granted
- Kind: B2
- Application number: 15/118,064
- Filed: 2015-02-04
- Published: 2017-06-22
- Granted: 2018-11-06
- Earliest claimed: 2014-02-13
- Assignees: Dolby Laboratories Licensing Corporation; Dolby International AB
- Applicants: Dolby International AB; Dolby Laboratories Licensing Corporation
- Inventors: Guan-Ming Su; Qian Chen; Peng Yin; Ralf Willenbacher
- Web page: https://patentgrep.com/patent/US10123019B2

## CPC

- H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/182: the unit being a pixel
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/52: by predictive encoding
- H04N 19/521: {for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Pixel data of a video sequence with enhanced dynamic range (EDR) are predicted based on pixel data of a corresponding video sequence with standard dynamic range (SDR) and a piecewise inter-layer predictor. The output parameters of the piecewise predictor are computed based atleast on two sets of pre-computed values and a prediction cost criterion. The first set of pre-computed values applies to all input SDR frames and comprises a set of SDR pixel values raised to one or more integer power terms. The second set of pre-computed values is frame specific and is computed based on a histogram of an input SDR frame and pixel values of the corresponding EDR frame. The pre-computed values allow for a fast iterative algorithm to identify the best pivot points for the piecewise polynomials according to a prediction cost and to solve for the coefficients of the piecewise predictor.

## Claims

### Claim 1

A method for generating a piecewise inter-layer predictor,
the method comprising:
accessing a first video signal with a first dynamic range;
accessing a second video signal representing the first video signal in a second dynamic range, wherein the second dynamic range is lower than the first dynamic range; and
generating with a processor a piecewise inter-layer predictor to predict pixels of the first video signal based on pixels of the second video signal, wherein generating the piecewise inter-layer predictor comprises:
generating a first set of pre-computed values based on computing bn, wherein b denotes a pixel value in a frame of the second video signal in the range [0, K−1], n is an integer larger than 0, K=2M, and M denotes the bit depth of the second video signal;
for a frame of the second video signal, using the processor to:
generate a histogram for the frame of the second video signal, wherein generating a histogram value hb comprises computing
h
b
=
∑
i
=
0
P
-
1
⁢
(
s
ji
==
b
)
,
wherein for the j-th frame in the second video signal, P denotes the total number of pixels under consideration in the frame, and sji denotes pixel values of the second video signal in the frame;
generate a second set of pre-computed values comprising:
w
b
=
∑
i
=
0
P
-
1
⁢
⁢
(
(
s
ji
⩵
b
)
·
v
ji
)
and
w
b
2
=
∑
i
=
0
P
-
1
⁢
⁢
(
(
s
ji
⩵
b
)
·
v
ji
2
)
,
wherein vji denotes pixel values of the corresponding j-th frame in the first video signal; and
generating coefficients of at least one polynomial in the piecewise predictor using the first set and second set of pre-computed values and the histogram.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first dynamic range is a high or an enhanced dynamic range, and the second dynamic range is a standard dynamic range.

### Claim 3 (depends on 2)

The method of claim 2, wherein the bit depth M is 8 or 10 bits.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the first set of pre-computed values comprises computing
t
b
n
=
(
b
K
)
n

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the piecewise predictor further comprises computing an output pivot point of the predictor, wherein the pivot point separates the input range between a first and a second polynomial of the predictor, wherein computing the output pivot point comprises:
for two or more candidate pivot points within a range of possible pivot points:
compute the coefficients of the first and second polynomials using the first set and the second set of pre-computed values;
compute a distortion metric for a prediction cost function based on the computed coefficients; and
select as the output pivot point, the pivot point among the two or more candidate pivot points for which the computed distortion metric is minimum.

### Claim 6 (depends on 5)

The method of claim 5, wherein the computed distortion metric is based on the mean square error difference between predicted values generated using the predictor polynomials and corresponding pixel values of the first video signal.

### Claim 7 (depends on 5)

The method of claim 5, wherein the computed distortion metric is based on minimizing the difference between a maximum positive and a minimum negative prediction error generated using the polynomial predictor.

### Claim 8 (depends on 1)

The method of claim 1, wherein the piecewise predictor comprises two, four, six, or eight, second-order polynomials.

### Claim 9 (depends on 5)

The method of claim 5, wherein computing the coefficients of the first and second polynomials using the first and the second set of pre-computed values takes into consideration a continuity constrain between the first and second polynomials at the candidate pivot point.

### Claim 10 (depends on 5)

The method of claim 5, wherein computing the distortion metric is based on the computed histogram and the second set of pre-computed values.

### Claim 11 (depends on 5)

The method of claim 5, wherein computing the distortion metric comprises computing
D
j
L
⁡
(
s
jv
)
=
∑
b
=
0
s
jv
-
1
⁢
⁢
(
w
b
2
-
2
⁢
⁢
w
b
·
v
^
jb
+
h
b
·
(
v
^
jb
)
2
)
wherein sjv, denotes the candidate pivot point, wb denotes a value in the second set of pre-computed values, hb denotes a histogram value for a pixel with value b, and {circumflex over (v)}jb denotes predicted values.

### Claim 12 (depends on 5)

The method of claim 5, wherein the range of possible pivot points comprises the range [0, K−1].

### Claim 13 (depends on 5)

The method of claim 5, wherein the range of possible pivot points comprises the range [search_low, search_high], where for a given threshold delta,
search_low=max(sj-1,vopt−delta, 0),
search_high=min(sj-1,vopt+delta, K−1),
where sj-1,vopt denotes the output pivot point for a previous frame.

### Claim 14 (depends on 1)

The method of claim 1, wherein generating the histogram and values in the second set of pre-computed values for a current frame takes into consideration histogram values and pre-computed values for one or more previous frames to the current frame.

### Claim 15 (depends on 4)

The method of claim 4, wherein P, the total number of pixels under consideration in frame j, is smaller than the actual number of pixels in the frame j.

### Claim 16 (depends on 1)

The method of claim 1, wherein the first set and the second set of pre-computed values are stored into one or more look-up tables.

### Claim 17 (depends on 1)

The method of claim 1, wherein generating the piecewise predictor further comprises:
dividing the second dynamic range into a set of segments, wherein the lengths of each segment between the second segment and the second to last segment are equal.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
for at least one segment in the set of segments,
determining a training segment, wherein the training segment comprises the at least one segment and part of its neighbor segment; and generating coefficients of the polynomial predictor for the at least one segment using pixels in the training segment.

### Claim 19 (depends on 1)

An apparatus comprising a processor and configured to perform the method recited in claim 1.

### Claim 20 (depends on 1)

A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing a method with one or more processors in accordance with claim 1.