# US 2024/0007682 A1 · Wrapped reshaping for codeword augmentation with neighborhood consistency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0007682 A1
- Document ID: US20240007682A1
- Status: Published application
- Kind: A1
- Application number: 18/252,357
- Filed: 2021-11-10
- Published: 2024-01-04
- Earliest claimed: 2020-11-11
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Janos HORVATH; Harshad KADU; Guan-Ming SU
- Web page: https://patentgrep.com/patent/US20240007682A1

## CPC

- H04N 19/98: Adaptive-dynamic-range coding [ADRC]
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/192: the adaptation method, adaptation tool or adaptation type being iterative or recursive
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An input image of a first bit depth in an input domain is received. Forward reshaping operations are performed on the input image to generate a forward reshaped image of a second bit depth in a reshaping domain. An image container containing image data derived from the forward reshaped image is encoded into an output video signal of the second bit depth.

## Claims

### Claim 1

A method comprising:
receiving an input image of a first bit depth in an input domain from an input video signal of the first bit depth, the first bit depth being higher than a second bit depth in a reshaping domain;
performing forward reshaping operations on the input image to generate a forward reshaped image of the second bit depth in the reshaping domain, the forward reshaping operations including wrapping input codewords in the input image along a non-wrapped axis of the input domain into reshaped codewords in the forward reshaped image in a wrapped axis of the reshaping domain;
encoding, into an output video signal of the second bit depth, an image container containing image data derived from the forward reshaped image, the image data in the image container causing a recipient device of the output video signal to construct a backward reshaped image of a third bit depth for rendering on a display device, the third bit depth being higher than the second bit depth.

### Claim 2

A method comprising:
receiving an input image of a first bit depth in an input domain from an input video signal of the first bit depth, the first bit depth being higher than a second bit depth in a reshaping domain;
performing forward reshaping operations on the input image to generate a forward reshaped image of the second bit depth in the reshaping domain, the forward reshaping operations including wrapping input codewords in the input image along a non-wrapped axis of the input domain into reshaped codewords in the forward reshaped image in a wrapped axis of the reshaping domain, wherein the reshaping domain is a wrapped reshaping domain and wherein the wrapped axis refers to an axis in the wrapped reshaping domain mapped from the non-wrapped axis through one or more wrapping transformations, wherein the one or more wrapping transformations are one or more geometric transforms;
encoding, into an output video signal of the second bit depth, an image container containing image data derived from the forward reshaped image, the image data in the image container causing a recipient device of the output video signal to construct a backward reshaped image of a third bit depth for rendering on a display device, the third bit depth being higher than the second bit depth.

### Claim 3 (depends on 1)

The method of claim 1, wherein non-wrapped axis refers to a single channel or dimension or a combination of multiple channels or dimensions in the input image in the input domain.

### Claim 4 (depends on 1)

The method of claim 1, wherein the wrapped axis of the reshaping domain has a geometric shape different from the non-wrapped axis; wherein a total number of available codewords on the wrapped axis is more than a total number of available codewords on the non-wrapped axis.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
applying a forward truncated field transform to the forward reshaped image to generate an intermediate image in a truncated field domain, wherein the forward truncated field transform ensures that codewords stay in a designated codeword value range;
performing one or more image processing operations on the intermediate image to generate a processed intermediate image;
applying an inverse truncated field transform to the processed intermediate image to generate the image data in the image container.

### Claim 6 (depends on 5)

The method of claim 5, wherein the processed intermediate image includes one or more clipped codeword values generated from clipping operations to ensure all codeword values in the processed intermediate image to be within a target spatial shape representing the reshaping domain.

### Claim 7 (depends on 1)

The method of claim 1, wherein the second bit depth represents one of: 8 bits, 10 bits, 12 bits or another number of bits lower than the first bit depth.

### Claim 8 (depends on 1)

The method of claim 1, wherein the forward reshaping operations are based on a set of forward reshaping mappings; wherein a set of operational parameter values used in the set of forward reshaping mappings is selected, based at least in part on input codewords in the input image, from among a plurality of sets of operational parameter values; wherein each set of operational parameters in the plurality of sets of operational parameter values is optimized to minimize predictions errors for a respective training image cluster among a plurality of training image clusters.

### Claim 9 (depends on 8)

The method of claim 8, wherein the set of forward reshaping mappings includes one or more of: multivariate multiple regression (MMR) mappings, tensor product B-spline (TPB) mappings, forward reshaping lookup tables (FLUTs), or other types of forward reshaping mappings.

### Claim 10 (depends on 8)

The method of claim 8, wherein each set of operational parameters in the plurality of sets of operational parameter values is generated based on a backward error subtraction for signal adjustment (BESA) algorithm.

### Claim 11 (depends on 10)

The method of claim 10, wherein prediction errors propagated in the BESA algorithm are computed based at least in part on (a) differences between input values and reconstructed values and (b) spatial gradients derived as partial derivatives of a cross-channel forward reshaping function used for generating the reconstructed values.

### Claim 12 (depends on 1)

The method of claim 1, wherein input codewords in the input image are scaled with one or more scaling factors; wherein the one or more scaling factors are optimized at runtime using a search algorithm based on one or more of: a Golden section algorithm, a Nelder-Mead algorithm, or another search algorithm.

### Claim 13 (depends on 1)

The method of claim 1, wherein an input reshape representing the input domain is mapped, using a set of mapping functions, to a target reference shape representing the reshaping domain; wherein the target reference shape is selected, based at least in part on a distribution of input codewords in the input image, from among a plurality of different reference shapes corresponding to a plurality of different candidate reshaping domains.

### Claim 14 (depends on 13)

The method of claim 13, wherein the plurality of different reference shapes comprises: at least one of: a complete geometric shape, a complete geometric shape inserted with a single gap portion, a complete geometric shape inserted with multiple gap portions, a donut shape, a cylinder shape, a torus shape, another shape homomorphic to a cube, or another shape non-homomorphic to a cube.

### Claim 15 (depends on 1)

The method of claim 1, wherein the input domain represents one of: an RGB color space, a YCbCr color space, a perceptual quantized color space, a linear color space, or another color space.

### Claim 16 (depends on 1)

The method of claim 1, wherein the first bit depth represents one of: 12 bits, 16 or more bits, or another number of bits higher than the second bit depth.

### Claim 17 (depends on 1)

The method of claim 1, wherein the forward reshaping operations represent primary reshaping operations; the method further comprising: performing secondary reshaping operations on the input image to linearly scale a limited input data range to a full data range for each color channel, wherein the secondary reshaping operations include one or more of: per-channel pre-reshaping or scaling.

### Claim 18 (depends on 17)

The method of claim 17, wherein image metadata is generated based on operational parameters used in the primary reshaping operations and the secondary reshaping operations; wherein the image metadata is provided in the output video signal of the second bit depth to the recipient device.

### Claim 19 (depends on 17)

The method of claim 17, wherein the secondary reshaping operations are performed in a first stage preceding a second stage in which the primary reshaping operations are performed.

### Claim 20 (depends on 17)

The method of claim 17, wherein a combination of the secondary reshaping operations and the primary reshaping operations is performed at runtime in a single combined stage.

### Claim 21 (depends on 1)

The method of claim 1, wherein the reshaping domain is represented by a torus shape or a cylinder shape.

### Claim 22

(canceled)

### Claim 23

A method comprising:
decoding, from a video signal, an image container containing image data derived from a forward reshaped image in a reshaping domain, a source image of a first bit depth in a source domain having been forward reshaped to generate the forward reshaped image in a second bit depth lower than the first bit depth;
applying backward reshaping operations on the image data decoded from the video signal to generate a backward reshaped image in a target domain, the backward reshaped image being of a third bit depth higher than the second bit depth, the backward reshaping operations including unwrapping codewords in the image data along a wrapped axis of the reshaping domain into backward reshaped codewords in the backward reshaped image in a non-wrapped axis of the target domain;
rendering, on a display device, a display image generated from the backward reshaped image.

### Claim 24 (depends on 1)

An apparatus comprising a processor and configured to perform the method recited in claim 1.

### Claim 25 (depends on 1)

A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing a method with one or more processors in accordance with the method of claim 1.
