# US 11,310,537 B2 · Linear encoder for image/video processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,537 B2
- Document ID: US11310537B2
- Status: Granted
- Kind: B2
- Application number: 16/975,096
- Filed: 2019-02-28
- Published: 2021-03-25
- Granted: 2022-04-19
- Earliest claimed: 2018-02-28
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Neeraj J. Gadgil; Guan-Ming Su
- Web page: https://patentgrep.com/patent/US11310537B2

## CPC

- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/98: Adaptive-dynamic-range coding [ADRC]
- G06F 1/03: working, at least partly, by table look-up (G06F1/025 takes precedence)
- G06T 5/90: Dynamic range modification of images or parts thereof
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/142: Detection of scene cut or scene change
- H04N 19/182: the unit being a pixel
- H04N 19/184: the unit being bits, e.g. of the compressed video stream
- H04N 19/186: the unit being a colour or a chrominance component
- G06T 2207/20208: High dynamic range [HDR] image processing
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In some embodiments, an encoder device is disclosed to receive an input video stream containing images in a first dynamic range including a first image. The device receives a second image representing the first image. The device obtains statistical data for the first and the second images. The device determines, at a first time delay, a scene cut data from the input video stream and storing the scene cut data in a first sliding window. The device determines, at a second time delay, a first smoothing mapping function based on a second sliding window and the determined scene cut data. The device determines, at a third time delay, a second smoothing mapping function based on a third sliding window and the determined scene cut data. The device generates, at the third time delay, a composer metadata for the first image based on the first and second smoothing mapping functions.

## Claims

### Claim 1

A method for luma and chroma reshaping metadata generation with a processing system, the method comprising:
receiving an input video stream containing images in a first dynamic range including a first image;
receiving a second image representing the first image of the input video stream, the second image having a second dynamic range, wherein the second dynamic range is higher than the first dynamic range;
obtaining statistical data for the first and the second images, wherein the obtained statistical data includes luminance and chroma statistics for the first and second images, and storing the statistical data in a buffer;
determining, at a first time delay, scene cut data based on the statistical data in a first sliding window (L3) including the first image, one or more look ahead images after the first image in time in the input video stream, and one or more look back images before the first image in time in the input video stream;
determining, at a second time delay, a first mapping function for mapping luminance values for the first image from the first dynamic range into the second dynamic range based on the statistical data and determined scene cut data in a second sliding window (L2), wherein data in the second sliding window are determined based on data in the first sliding window;
determining, at a third time delay, a second mapping function by performing a smoothing operation to the first mapping function based on the first mapping function and the determined scene cut data in a third sliding window (L1), wherein data in the third sliding window are determined based on data in the second sliding window;
determining, at the third time delay, a third mapping function for mapping chroma values for the first image from the first dynamic range into the second dynamic range based on the statistical data and the determined scene cut data in the third sliding window (L1); and
generating, at the third time delay, a composer metadata for the first image based on the second and the third mapping functions, wherein the generated composer metadata are used by a target display device to map the first image from the first dynamic range into the second dynamic range for display.

### Claim 2 (depends on 1)

The method of claim 1, wherein the processing system is a linear encoder, the linear encoder having a configurable time delay between an input frame and a corresponding composer metadata.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of the first, second, and third sliding windows is a buffer that, over time, receives successive portions of the input video stream, each of the portions storing data for a set of consecutive images such that the buffer slides over time over the input video stream.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first, second, and third sliding windows are implemented as ring memory buffers with configurable buffer lengths.

### Claim 5 (depends on 4)

The method of claim 4, wherein the statistical data are obtained in a statistics ring buffer with a configurable buffer length, wherein the buffer length is configured based on the buffer lengths of the first, second, and third sliding windows.

### Claim 6 (depends on 4)

The method of claim 4, wherein the ring memory buffer implementing the second sliding window stores histogram statistics of the first and second images, wherein the histogram statistics include one or more bins representing codewords for pixels of the first or the second images respectively.

### Claim 7 (depends on 6)

The method of claim 6, wherein the histogram statistics are down-sampled by dividing the codewords for pixels of the first or the second images respectively into a predetermined number of non-overlapping bins.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the first mapping function comprises determining a luminance mapping function based on a first and a second cumulative histograms, wherein the first and the second cumulative histograms are computed from luminance statistics of the first and second images respectively.

### Claim 9 (depends on 8)

The method of claim 8, wherein the second mapping function is a backward reshaping look-up table (BLUT).

### Claim 10 (depends on 9)

The method of claim 9, further comprising performing a smoothing operation on the first mapping function based on the detected scene cut data using one or more smoothing windows of different window lengths.

### Claim 11 (depends on 10)

The method of claim 10, further comprising converting the second mapping function to coefficients of piecewise polynomials to generate the composer metadata.

### Claim 12 (depends on 1)

The method of claim 1, wherein determining the third mapping function comprises solving a least square problem to compute multivariate multiple regression (MMR) matrix coefficients based on chroma statistics of pixels of the first and the second images.

### Claim 13 (depends on 1)

A non-transitory machine readable medium with stored instructions which when executed by a display management system cause the system to perform a method as in claim 1.

### Claim 14 (depends on 1)

A display management system which includes a memory and a processing system configured to perform a method as in claim 1.