# US 2022/0284548 A1 · System and method of automatic image enhancement using system generated feedback mechanism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284548 A1
- Document ID: US20220284548A1
- Status: Published application
- Kind: A1
- Application number: 17/249,411
- Filed: 2021-03-02
- Published: 2022-09-08
- Earliest claimed: 2021-03-02
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Saigeetha Aswathnarayanan Jegannathan; Sridhar Jonnala; Harsha Deuri; Venkata Jami; Prakriti Pritmani; Aakash Praliya
- Web page: https://patentgrep.com/patent/US20220284548A1

## CPC

- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06T 5/10: using non-spatial domain filtering
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 5/92: based on global image properties
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- G06V 10/40: Extraction of image or video features
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
- G06T 2207/20201: Motion blur correction
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for automatically enhancing image data comprising receiving image data. Utilizing a neural network to analyze the image data to predict how each of a plurality of image enhancement processes will affect the image data. Utilizing the neural network to calculate a reward value for each of the plurality of enhancement processes that can be applied to the image data. Determining if the image data should be enhanced or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the image data. When it is determined that the image data should be enhanced, then determining which of the plurality of image enhancing process should be applied to the image data, wherein determination is based on the reward value for each of the plurality of enhancement processes. Applying the determined enhanced processes to the image data.

## Claims

### Claim 1

A method for automatically enhancing image data, the method comprising:
receiving, by a computer, image data;
utilizing, by the computer, a neural network to analyze the image data to predict how each of a plurality of image enhancement processes will affect the image data;
utilizing, by the computer, the neural network to calculate a reward value for each of the plurality of enhancement processes that can be applied to the image data;
determining, by the computer, if the image data should be enhanced or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the image data;
wherein when it is determined that the image data should be enhanced, then determining, by the computer, which of the plurality of image enhancing process should be applied to the image data, wherein determination is based on the reward value for each of the plurality of enhancement processes; and
applying, by the computer, the determined enhanced processes to the image data.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
analyzing, by the computer, the image data to identify features contained within the image data; and
counting, by the computer, the identified features contained in the image data.

### Claim 3 (depends on 2)

The method of claim 2, wherein the determination for determine if the image data should be enhanced or not is further based on the count of identified features.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of enhancement processes is comprised of deblurring, denoising, glare removal, brightness adjustment, and resolution enhancement.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
utilizing, by the computer, the neural network to analyze the enhanced image data and calculating, by the computer, a new reward value for each of the plurality of enhancement processes to be applied to the enhanced image data;
determining, by the computer, if the enhanced image data should be enhanced again or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the enhanced image data;
wherein when it is determined that the enhanced image data should be enhanced again, then determining, by the computer, which of the plurality of image enhancing process should be applied to the enhanced image data, wherein determination is based on the reward value for each of the plurality of enhancement processes; and
applying, by the computer, the determined enhanced processes to the enhanced image data.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
wherein when it is determined that the enhanced image data should not be enhanced again, outputting, by the computer, the enhanced image to a user computing device.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
generating, by the computer, an enhancement report, wherein the enhancement report includes each enhancement process applied to the image data and the enhancement values associated with each enhancement process; and
outputting, by the computer, the enhancement report with the outputted enhanced image data.

### Claim 8 (depends on 1)

The method of claim 1, wherein the neural network is a Dueling Double Deep Q Network (D3QN).

### Claim 9

A computer program product for automatically enhancing image data, the computer program product comprising:
one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media, the program instructions comprising:
program instructions to receive image data;
program instructions to utilize a neural network to analyze the image data to predict how each of a plurality of image enhancement processes will affect the image data;
program instructions to utilize the neural network to calculate a reward value for each of the plurality of enhancement processes that can be applied to the image data;
program instructions to determine if the image data should be enhanced or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the image data;
wherein when it is determined that the image data should be enhanced, then program instructions to determine which of the plurality of image enhancing process should be applied to the image data, wherein determination is based on the reward value for each of the plurality of enhancement processes; and
program instructions to apply the determined enhanced processes to the image data.

### Claim 10 (depends on 9)

The computer program product of claim 9, further comprising:
program instructions to analyze the image data to identify features contained within the image data; and
program instructions to count the identified features contained in the image data.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein the determination for determine if the image data should be enhanced or not is further based on the count of identified features.

### Claim 12 (depends on 9)

The computer program product of claim 9, wherein the plurality of enhancement processes is comprised of deblurring, denoising, glare removal, brightness adjustment, and resolution enhancement.

### Claim 13 (depends on 9)

The computer program product of claim 9, further comprising:
program instructions to utilize the neural network to analyze the enhanced image data and program instructions to calculate a new reward value for each of the plurality of enhancement processes to be applied to the enhanced image data;
program instructions to determine if the enhanced image data should be enhanced again or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the enhanced image data;
wherein when it is determined that the enhanced image data should be enhanced again, then program instructions to determine which of the plurality of image enhancing process should be applied to the enhanced image data, wherein determination is based on the reward value for each of the plurality of enhancement processes; and
program instructions to apply the determined enhanced processes to the enhanced image data.

### Claim 14 (depends on 13)

The computer program product of claim 13, further comprising:
wherein when it is determined that the enhanced image data should not be enhanced again, program instructions to output the enhanced image to a user computing device.

### Claim 15 (depends on 14)

The computer program product of claim 14, further comprising:
program instructions to generate an enhancement report and to output the enhancement report with the enhanced image data, wherein the enhancement report includes each enhancement process applied to the image data and the enhancement values associated with each enhancement process.

### Claim 16 (depends on 9)

The computer program product of claim 9, wherein the neural network is a Dueling Double Deep Q Network (D3QN).

### Claim 17

A computer system for automatically enhancing image data, the computer system comprising:
one or more computer processors, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising:
program instructions to receive image data;
program instructions to utilize a neural network to analyze the image data to predict how each of a plurality of image enhancement processes will affect the image data;
program instructions to utilize the neural network to calculate a reward value for each of the plurality of enhancement processes that can be applied to the image data;
program instructions to determine if the image data should be enhanced or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the image data;
wherein when it is determined that the image data should be enhanced, then program instructions to determine which of the plurality of image enhancing process should be applied to the image data, wherein determination is based on the reward value for each of the plurality of enhancement processes; and
program instructions to apply the determined enhanced processes to the image data.

### Claim 18 (depends on 17)

The computer system of claim 17, further comprising:
program instructions to utilize the neural network to analyze the enhanced image data and program instructions to calculate a new reward value for each of the plurality of enhancement processes to be applied to the enhanced image data;
program instructions to determine if the enhanced image data should be enhanced again or not, wherein the determination is based on the predictions how each of the plurality of image enhancement processes will affect the enhanced image data;
wherein when it is determined that the enhanced image data should be enhanced again, then program instructions to determine which of the plurality of image enhancing process should be applied to the enhanced image data, wherein determination is based on the reward value for each of the plurality of enhancement processes;
program instructions to apply the determined enhanced processes to the enhanced image data; and
wherein when it is determined that the enhanced image data should not be enhanced again, program instructions to output the enhanced image to a user computing device.

### Claim 19 (depends on 17)

The computer system of claim 17, further comprising:
program instructions to generate an enhancement report and to output the enhancement report with the enhanced image data, wherein the enhancement report includes each enhancement process applied to the image data and the enhancement values associated with each enhancement process.

### Claim 20 (depends on 17)

The computer system of claim 17, wherein the neural network is a Dueling Double Deep Q Network (D3QN).
