# US 2020/0394772 A1 · Systems and methods for tone mapping of high dynamic range images for high-quality deep learning based processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394772 A1
- Document ID: US20200394772A1
- Status: Published application
- Kind: A1
- Application number: 16/438,750
- Filed: 2019-06-12
- Published: 2020-12-17
- Earliest claimed: 2019-06-12
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Attila Tamas Afra
- Web page: https://patentgrep.com/patent/US20200394772A1

## CPC

- G06T 5/92: based on global image properties
- G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)}
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 3/0418: {using chaos or fractal principles}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/70: Denoising; Smoothing
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for tone mapping of high dynamic range (HDR) images for high-quality deep learning based processing are disclosed. In one embodiment, a graphics processor includes a media pipeline to generate media requests for processing images and an execution unit to receive media requests from the media pipeline. The execution unit is configured to compute an auto-exposure scale for an image to effectively tone map the image, to scale the image with the computed auto-exposure scale, and to apply a tone mapping operator including a log function to the image and scaling the log function to generate a tone mapped image.

## Claims

### Claim 1

A graphics processor, comprising:
a media pipeline to generate media requests for processing images; and
an execution unit to receive media requests from the media pipeline, the execution unit is configured to compute an auto-exposure scale for an image to effectively tone map the image, to scale the image with the computed auto-exposure scale, and to apply a tone mapping operator including a log function to the image and scaling the log function to generate a tone mapped image.

### Claim 2 (depends on 1)

The graphics processor of claim 1, wherein the log function compresses a range of values for the image.

### Claim 3 (depends on 2)

The graphics processor of claim 2, wherein the execution unit is configured to apply a gamma correction to the tone mapped image to generate a gamma corrected tone mapped image and to make a tone mapping curve more perceptually linear for the image.

### Claim 4 (depends on 3)

The graphics processor of claim 3, wherein the execution unit is configured to provide the tone mapped image as input for a neural network (e.g., CNN).

### Claim 5 (depends on 4)

The graphics processor of claim 4, wherein the neural network processes the tone mapped image to generate an output of the neural network.

### Claim 6 (depends on 5)

The graphics processor of claim 5, wherein the execution unit is configured to apply an inverse tone mapping operator and then inverse exposure scale to this output of the neural network to generate the final output image.

### Claim 7

A graphics processing unit, comprising:
memory to store graphics data for images; and
a core having a plurality of execution units for performing graphics operations, wherein at least one execution unit is configured to downsample the image in multiple dimensions, to compute an auto-exposure scale for an image to effectively tone map the image, to scale the image with the computed auto-exposure scale, and to apply a tone mapping operator including a log function to the image and scaling the log function to generate a tone mapped image.

### Claim 8 (depends on 7)

The graphics processing unit of claim 7, wherein the at least one execution unit is configured to apply a gamma correction to the tone mapped image to generate a gamma corrected tone mapped image.

### Claim 9 (depends on 8)

The graphics processing unit of claim 8, wherein the at least one execution unit is configured to provide the tone mapped image as input for a convolution neural network (CNN).

### Claim 10 (depends on 9)

The graphics processing unit of claim 9, wherein the neural network processes the tone mapped image with a denoising algorithm, anti-aliasing algorithm, super resolution, or demosaicing to generate an output of the neural network.

### Claim 11 (depends on 10)

The graphics processing unit of claim 10, wherein the at least one execution unit is configured to apply an inverse tone mapping operator and then inverse exposure scale to this output of the neural network to generate an output image having a high dynamic range (HDR) of the image.

### Claim 12 (depends on 7)

The graphics processing unit of claim 7, wherein the tone mapping operator comprises (log2(x+1)/16)1/2.2.

### Claim 13 (depends on 12)

The graphics processing unit of claim 12, wherein values of the log function are scaled by 1/16 such that floating-point values are compressed between 0 and 1.

### Claim 14

A computer implemented method for tone mapping images:
downsampling an image in multiple dimensions;
computing, with a graphics processing unit (GPU), an auto-exposure scale for the image to effectively tone map the image;
scaling the image with the computed auto-exposure scale; and
applying a tone mapping operator including a log function to the image and scaling the log function to generate a tone mapped image.

### Claim 15 (depends on 14)

The computer implemented method of claim 14, further comprising:
applying a gamma correction to the tone mapped image to generate a gamma corrected tone mapped image and generating a tone mapping curve that is perceptually linear.

### Claim 16 (depends on 15)

The computer implemented method of claim 15, further comprising:
providing the tone mapped image as input for a convolutional neural network (CNN); and
processing including one of denoising algorithm and anti-aliasing algorithm the tone mapped image to generate an output of the CNN.

### Claim 17 (depends on 16)

The computer implemented method of claim 16, wherein the processing includes applying tone mapping to the image and target images, and computing a loss directly in the tone mapped color space.

### Claim 18 (depends on 17)

The computer implemented method of claim 17, further comprising:
applying an inverse of the tone mapping operator to the output of the CNN.

### Claim 19 (depends on 18)

The computer implemented method of claim 18, further comprising:
applying an inverse of the exposure scale to the output of the CNN to generate the final output image.
