# US 10,123,018 B2 · Feature based bitrate allocation in non-backward compatible multi-layer codec via machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,123,018 B2
- Document ID: US10123018B2
- Status: Granted
- Kind: B2
- Application number: 15/270,135
- Filed: 2016-09-20
- Published: 2017-03-30
- Granted: 2018-11-06
- Earliest claimed: 2015-09-29
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Sankaranarayanan Piramanayagam; Hsuan-Ting Chou; Guan-Ming Su
- Web page: https://patentgrep.com/patent/US10123018B2

## CPC

- H04N 19/136: Incoming video signal characteristics or properties
- G09G 5/10: Intensity circuits
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/179: the unit being a scene or a shot
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/187: the unit being a scalable video layer
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
- G09G 2350/00: Solving problems of bandwidth in display systems
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Novel methods and systems for non-backward compatible video encoding are disclosed. The bitrates of the base layer and enhancement layer are dynamically assigned based on features found in scenes in the video compared to a machine learned quality classifier.

## Claims

### Claim 1

A method of assigning bitrates to base layers and enhancement layers in a dual codec, the method comprising:
scanning a scene of a video for bright and dark areas and accumulating feature data of the scene, and determining feature values of one or more features of the scene using the accumulated feature data of the scene;
calculating an alpha value based on the bright and dark areas, wherein the alpha value is the exponent of a formula for forward reshaping of a luminance channel of the video;
decomposing the scene into a base layer and at least one enhancement layer, wherein said decomposing involves determining the base layer and the at least one enhancement layer of the video based on the alpha value;
determining a bitrate class of the base layer and the at least one enhancement layer by processing the feature values of the one or more features of the scene against a machine learning model; and
setting the bitrates of the base layer and the at least one enhancement layer based on the determined bitrate class,
wherein processing of the feature values of the one or more features against the machine learning model comprises:
mapping the feature values of the one or more features into N dimensional space where N is the number of the one or more features; and
determining where the mapped feature values is located in the N dimensional space compared to predetermined hyperplanes of the N dimensional space.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the base layer and the at least one enhancement layer of the video is performed by clipping at a mid-range of the forward reshaped luminance channel.

### Claim 3 (depends on 1)

The method of claim 1, wherein the formula for forward reshaping si is of the form
s
i
=
round
⁡
(
(
C
H
-
C
L
)
⁢
(
v
i
-
v
L
v
H
-
v
L
)
α
+
C
L
)
where α is the alpha value, νL and νH are minimum and maximum luminance values in an individual shot of the scene of the video, and CL and CH are minimum and maximum values of a reshaped output signal.

### Claim 4 (depends on 1)

The method of claim 1, wherein the one or more features are selected from the list of: ratio of enhancement layer pixels of an enhancement layer to base layer pixels, ratio of enhancement layer pixels of an enhancement layer to total pixels, ratio of standard deviation in one of the at least one enhancement layer to a channel standard deviation in the base layer for a channel, the alpha value, and salient region detection.

### Claim 5 (depends on 4)

The method of claim 4, where the video is in IPT color space and the channel is any of I channel, P channel, or T channel.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more features comprises a ratio of standard deviation in the one of the at least one enhancement layer to a channel standard deviation in the base layer for the I channel and a ratio of standard deviation in the one of the at least one enhancement layer to a channel standard deviation in the base layer for the T channel.

### Claim 7 (depends on 4)

The method of claim 4, wherein the salient regions comprise images of fire, images of faces, or both images of fire and images of faces.

### Claim 8 (depends on 1)

The method of claim 1, wherein the calculating the alpha value is a value in the range of 1.0 to 1.7.

### Claim 9 (depends on 1)

The method of claim 1, wherein the accumulating of feature data comprises accumulating the feature values into a plurality of bins, and optionally,
wherein each bin of the plurality of bins corresponds to a range of luminance values.

### Claim 10 (depends on 1)

The method of claim 1, wherein the feature values comprise a ratio of standard deviation in one of the at least one enhancement layer to a channel standard deviation in the base layer, and optionally
wherein the standard deviations of each bin of the plurality of bins are multiplied by a power function.