# US 2019/0058886 A1 · Preserving texture/noise consistency in video codecs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0058886 A1
- Document ID: US20190058886A1
- Status: Published application
- Kind: A1
- Application number: 15/763,039
- Filed: 2016-09-22
- Published: 2019-02-21
- Earliest claimed: 2015-09-23
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Qian CHEN; Guan-Ming SU
- Web page: https://patentgrep.com/patent/US20190058886A1

## CPC

- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- H04N 19/124: Quantisation
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/172: the region being a picture, frame or field
- H04N 19/179: the unit being a scene or a shot
- H04N 19/1887: {the unit being a variable length codeword}
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
- H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness
- H04L 65/765: {intermediate}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

For each content-mapped frame of a scene, it is determined whether the content mapped frame is susceptible to object fragmentation with respect to texture in a homogeneous region based on statistical values derived from the content-mapped image and a source image mapped into the content-mapped image. The homogeneous region is a region of consistent texture in the source image. Based on a count of content-mapped frames susceptible to object fragmentation in homogeneous region, it is determined whether the scene is susceptible to object fragmentation in homogeneous region. If so, an upper limit for mapped codewords for a prediction function for predicting codewords of a predicted image from the mapped codewords in the content-mapped image is adjusted. Mapped codewords above the upper limit are clipped to the upper limit.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method of layered encoding, comprising:
for each content-mapped image in a group of content-mapped images among a series of consecutive content-mapped images in a video sequence that collectively represent a scene, determining whether the content-mapped image is of a frame category susceptible to object fragmentation with respect to texture in a homogeneous region based at least in part on one or more statistical values derived from the content-mapped image and a source image that has been mapped into the content-mapped image, the homogeneous region being a region of consistent texture in the source image, the source image being of a dynamic range higher than a relatively narrow dynamic range of the content-mapped image;
based on a count of content-mapped images of the frame category susceptible to object fragmentation in the homogeneous region among the group of content-mapped images, determining whether the scene is of a scene category susceptible to object fragmentation in the homogeneous region;
in response to determining that the scene is of the scene category susceptible to object fragmentation in the homogeneous region, adjusting an upper limit for mapped codewords for a prediction function for predicting codewords of a predicted image from the mapped codewords in the content-mapped image, mapped codewords above the upper limit being clipped to the upper limit, the upper limit being selected to prevent object fragmentation in the homogeneous region in reconstructed images generated by downstream devices.

### Claim 22 (depends on 21)

The method as recited in claim 21, wherein the prediction function is a piece-wise prediction function, segments of the piece-wise prediction function being represented by a plurality of polynomials.

### Claim 23 (depends on 21)

The method as recited in claim 21, wherein the upper limit is selected such that the entire homogeneous region is encoded in an enhancement layer of a multi-layer video signal to thereby prevent object fragmentation of the homogeneous region, the base layer having encoded therein the series of consecutive content-mapped images and the enhancement layer having encoded therein, for each content-mapped image among the series of consecutive content-mapped images, a difference image between the source image and the predicted image.

### Claim 24 (depends on 21)

The method as recited in claim 21, further comprising:
encoding the series of consecutive content-mapped images in the video sequence into a base layer of a multi-layer video signal;
for each content-mapped image in the series of content-mapped images, predicting a predicted image by applying the prediction function to the mapped codewords in the content-mapped image, and computing a difference image between the source image that has been mapped into the content-mapped image and the predicted image; and
encoding the computed difference images for the series of content-mapped images into an enhancement layer of the multi-layer video signal.

### Claim 25 (depends on 21)

The method as recited in claim 21, wherein the content-mapped image is of a frame category susceptible to object fragmentation in the homogeneous region if the homogeneous region in a corresponding reconstructed image generated by a downstream device is likely to have inconsistent texture.

### Claim 26 (depends on 21)

The method as recited in claim 21, wherein the one or more statistical values comprise one or more of: a count histogram with bins each of which contains a count of pre-mapped codewords for a mapped codeword, or a distribution of standard deviation values over mapped codewords.

### Claim 27 (depends on 21)

The method as recited in claim 21, wherein determining whether the content-mapped image is of the frame category susceptible to object fragmentation in the homogeneous region includes determining whether the content-mapped image is a special trim pass frame based at least in part on the one or more statistical values.

### Claim 28 (depends on 21)

The method as recited in claim 21, further comprising:
deriving a residual energy histogram from the content-mapped image and the source image, the residual energy histogram indicating, for each mapped codeword in the content-mapped image, the residual energy of all pre-mapped codewords for the mapped codeword;
locating one or more residual energy peaks in the residual energy histogram;
based on the one or more residual energy peaks located in the residual energy histogram, classifying the content-mapped image into a frame-level residual energy peak type in a plurality of frame-level residual energy peak types; and
adjusting the upper limit for mapped codewords based at least in part on the frame level residual energy peak type of the content-mapped image.

### Claim 29 (depends on 28)

The method as recited in claim 28, wherein the plurality of frame-level residual energy peak types comprises one or more of: a frame type of single residual energy peak in highlight, a frame type of multiple residual energy peaks in highlight, a frame type of one residual energy peak in highlight and one residual energy peak in dark, or a frame type of single residual energy peak in dark.

### Claim 30 (depends on 21)

The method as recited in claim 21, further comprising classifying the scene into a scene-level residual energy peak type in a plurality of scene-level residual energy peak types based on one or more frame-level residual energy peak types of content-mapped images in the group of content-mapped images.

### Claim 31 (depends on 30)

The method as recited in claim 30, wherein the scene-level residual energy peak type corresponds to a frame-level residual energy peak type of a majority of content-mapped images in the group of content-mapped images.

### Claim 32 (depends on 22)

The method as recited in claim 22, wherein the upper limit is represented by the last pivot point of all pivot points of the plurality of polynomials, the segments of the piece-wise prediction function being delimited by respective adjacent ones among the pivot points.

### Claim 33 (depends on 22)

The method as recited in claim 22, wherein the upper limit is adjusted from a preconfigured value for the plurality of polynomials to a value computed from one or more of: residual energy information as derived from the content-mapped image and the source image, or gradient information as derived from at least one of the content-mapped image or the source image.

### Claim 34 (depends on 33)

The method as recited in claim 33, wherein the upper limit is further adjusted from the computed value to a nearest preconfigured value below the computed value; and wherein the preconfigured value and the nearest preconfigured value below the computed value belong to a set of preconfigured values defined for the plurality of polynomials.

### Claim 35 (depends on 21)

The method as recited in claim 21, wherein adjusting an upper limit for mapped codewords in a plurality of polynomials representing a predictor includes:
determining whether the scene is of a scene type of single peak in highlight;
in response to determining that the scene is of the scene type of single peak in highlight, determining whether a noise estimation of the scene exceeds a noise threshold;
in response to determining that the noise estimation of the scene exceeds the noise threshold, adjusting the upper limit to the average value of mapped codewords for which a gradient peak is attained in the content-mapped images.

### Claim 36 (depends on 21)

The method as recited in claim 21, wherein adjusting an upper limit for mapped codewords in a plurality of polynomials representing a predictor includes:
determining whether the scene is of a scene type of multiple peaks in highlight;
in response to determining that the scene is of the scene type of multiple peaks in highlight,
determining whether the scene is a special trim pass scene, which is a scene for which all content-mapped images in the group of content-mapped images collectively representing the scene are special trim pass frames, and a special trim pass frame is a content-mapped image for which the peak count in a count histogram is in a specific sub-range of mapped codewords in the content-mapped image, each bin of the count histogram containing a count of pre-mapped codewords for a mapped codeword;
in response to determining that the scene is a special trim pass scene,
determining a scene-level minimum residual peak based on minimum residual energy peaks of content-mapped images of a frame type of multiple peaks in highlight in the scene, the minimum residual energy peaks being determined from a residual energy histogram that indicates, for each mapped codeword in the content-mapped image, the residual energy of all pre-mapped codewords for the mapped codeword;
adjusting the upper limit to a closest mapped codeword at which a residual energy equals a specific percentage of the scene-level minimum residual energy peak, wherein the closest mapped codeword is smaller than a mapped codeword represented by the scene-level minimum residual energy peak.

### Claim 37 (depends on 21)

The method as recited in claim 21, wherein adjusting an upper limit for mapped codewords in a plurality of polynomials representing a predictor includes:
determining whether the scene is of a scene type of multiple peaks in highlight;
in response to determining that the scene is of the scene type of multiple peaks in highlight,
determining whether the scene is a special trim pass scene, which is a scene for which all content-mapped images in the group of content-mapped images collectively representing the scene are special trim pass frames, and a special trim pass frame is a content-mapped image for which the peak count in a count histogram is in a specific sub-range of mapped codewords in the content-mapped image, each bin of the count histogram containing a count of pre-mapped codewords for a mapped codeword;
in response to determining that the scene is not a special trim pass scene, adjusting the upper limit to an average of minimum residual energy peaks of content-mapped images of multiple-peaks-in-highlight type in the scene, the minimum residual energy peaks being determined from a residual energy histogram that indicates, for each mapped codeword in the content-mapped image, the residual energy of all pre-mapped codewords for the mapped codeword.

### Claim 38 (depends on 21)

The method as recited in claim 21, wherein the upper limit is represented by a mapped codeword in a luminance-related channel of a color space.

### Claim 39

A method, comprising:
receiving a content-mapped image in a group of content-mapped images among a series of consecutive content-mapped images in a video sequence that collectively represent a scene and prediction metadata defining a plurality of polynomials that represent segments of a piece-wise predictor for predicting codewords of a predicted image from mapped codewords in the content-mapped image, the content-mapped image in the group of content-mapped images being generated by an upstream device based at least in part on a content-mapping function and a source image of a dynamic range higher than a relatively low dynamic range of the content-mapped image, the prediction metadata specifying an upper limit for mapped codewords for the piece-wise predictor that is adjusted in response to the upstream device determining that the scene is of a scene category susceptible to object fragmentation in a homogeneous region, the homogeneous region being a region of consistent texture in the source image;
generating, based at least in part on the piece-wise predictor with the upper limit, a predicted image, mapped codewords above the upper limit being clipped to the upper limit;
generating, based at least in part on the predicted image, a reconstructed image that represents a reconstructed version of the source image.

### Claim 40 (depends on 39)

The method as recited in claim 39, further comprising:
receiving a difference image representing a difference between the predicted image and the source image; and
generating the reconstructed image by combining the predicted image and the residual image.
