# US 2019/0378247 A1 · Image processing method, electronic device and non-transitory computer-readable recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378247 A1
- Document ID: US20190378247A1
- Status: Published application
- Kind: A1
- Application number: 16/212,289
- Filed: 2018-12-06
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Beijing Kuangshi Technology Co., Ltd.
- Applicants: Beijing Kuangshi Technology Co., Ltd.
- Inventors: Haibin Huang; Chi-Hao Wu
- Web page: https://patentgrep.com/patent/US20190378247A1

## CPC

- G06T 5/70: Denoising; Smoothing
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/73: Deblurring; Sharpening
- G06T 2207/20024: Filtering details
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

This disclosure provides an image processing method, an electronic device and a non-transitory computer-readable storage medium, and relates to the field of image processing technique. The image processing method and apparatus and the electronic device provided by this application perform denoising on the to-be-processed image to obtain a denoised image, and then perform detail enhancement on the target area image extracted from the denoised image to obtain a detail enhanced image, thereafter fuse the denoised image with the detail enhanced image to obtain an output image. Since detail enhancement has been performed on the target area image, the obtained output image is sharper and the display effect is better.

## Claims

### Claim 1

An image processing method, comprising:
performing denoising on a to-be-processed image to generate a denoised image;
extracting a target area image corresponding to a shooting target from the denoised image;
performing detail enhancement on the target area image to generate a detail enhanced image; and
fusing the denoised image with the detail enhanced image to obtain an output image.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the step of performing denoising on a to-be-processed image to generate a denoised image comprises:
inputting the to-be-processed image into a noise reduction network; and
using an output of the noise reduction network as the denoised image;
the noise reduction network employing a convolution-deconvolution neural network, and the denoised image and the to-be-processed image having the same size.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the convolution-deconvolution neural network includes a convolution neural network and a deconvolution neural network;
wherein the convolution neural network includes at least one convolution layer, each of the at least one convolution layer includes one or more first convolution kernels, and a denoised feature map is obtained after the first convolution kernels traverse a pixel matrix of the to-be-processed image;
wherein the deconvolution neural network employs a structure symmetrical with the convolution neural network, the deconvolution neural network includes at least one deconvolution layer, each of the at least one deconvolution layer includes second convolution kernels whose number is the same as the corresponding convolution layers, and the denoised image is obtained after the second convolution kernels traverse a feature matrix of the denoised feature map.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the step of extracting a target area image corresponding to a shooting target from the denoised image comprises:
determining, by a target extraction network, an area where the shooting target resides in the denoised image, the target extraction network employing a convolution neural network; and
isolating the area where the shooting target resides from the denoised image to obtain the target area image.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the step of performing detail enhancement on the target area image to generate a detail enhanced image comprises:
inputting the target area image into a detail enhancement network, using an output of the detail enhancement network as the detail enhanced image, the detail enhancement network employing a convolution-deconvolution neural network, and the detail enhanced image and the target area image having the same size.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the step of fusing the denoised image with the detail enhanced image to obtain an output image comprises:
replacing the target area image in the denoised image with the detail enhanced image; and
linearly fusing a boundary area where the detail enhanced image intersects the denoised image.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the step of replacing the target area image in the denoised image with the detail enhanced image comprises:
implementing a replacement process through the following formula:
R_final=aR_206+(1−a)R_202
where R_final is the output image, R_206 is the detail enhanced image, R_202 is the denoised image, a is the area where the target area image resides in the denoised image, and 1−a is other area than the target area image in the denoised image.

### Claim 8 (depends on 6)

The method according to claim 6, wherein the step of linearly fusing a boundary area where the detail enhanced image intersects the denoised image comprises:
linearly fusing the boundary area where the detail enhanced image intersects the denoised image by a smoothing filtering or wavelet reconstruction.

### Claim 9 (depends on 2)

The method according to claim 2, wherein before the step of performing denoising on the to-be-processed image comprises:
obtaining a training image sample set, the training image sample set including a plurality of groups of paired training images; and
training the noise reduction network, the target extraction network, or the detail enhancement network by using the training image sample set.

### Claim 10 (depends on 9)

The method according to claim 9, wherein each group of paired training images comprises: a first image and a second image respectively acquired in the same shooting environment;
wherein the step of acquiring the first image comprises: capturing the first image according to a set first exposure time and a set first sensitivity parameter;
wherein the step of acquiring the second image comprises: capturing the second image according to a set second exposure time and a set second sensitivity parameter; and
wherein the second exposure time is greater than the first exposure time, and the second sensitivity parameter is smaller than the first sensitivity parameter.

### Claim 11 (depends on 1)

The method according to claim 1, wherein before the step of performing denoising on the to-be-processed image, the method further comprises:
determining whether a light brightness in a current shooting environment is less than a set brightness threshold; and
if the light brightness in the current shooting environment is less than the set brightness threshold, using the captured image as the to-be-processed image.

### Claim 12

An electronic device, comprising:
an image acquisition assembly;
a memory; and
a processor;
wherein the image acquisition assembly is configured to acquire image data;
wherein the memory stores thereon a computer program capable of being executed on the processor, wherein steps of an image processing method are carried out when the computer program is executed by the processor, the image processing method comprises:
performing denoising on a to-be-processed image to generate a denoised image,
extracting a target area image corresponding to a shooting target from the denoised image,
performing detail enhancement on the target area image to generate a detail enhanced image, and
fusing the denoised image with the detail enhanced image to obtain an output image.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein the step of performing denoising on a to-be-processed image to generate a denoised image carried out when the computer program is executed by the processor comprises:
inputting the to-be-processed image into a noise reduction network, using an output of the noise reduction network as the denoised image, wherein the noise reduction network employs a convolution-deconvolution neural network, and wherein the denoised image and the to-be-processed image have the same size.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the convolution-deconvolution neural network includes a convolution neural network and a deconvolution neural network;
wherein the convolution neural network includes at least one convolution layer, each of the at least one convolution layer includes one or more first convolution kernels, and a denoised feature map is obtained after the first convolution kernels traverse a pixel matrix of the to-be-processed image;
wherein the deconvolution neural network employs a structure symmetrical with the convolution neural network, the deconvolution neural network includes at least one deconvolution layer, each of the at least one deconvolution layer includes second convolution kernels whose number is the same as the corresponding convolution layers, and the denoised image is obtained after the second convolution kernels traverse a feature matrix of the denoised feature map.

### Claim 15 (depends on 1)

The electronic device according to claim 1, wherein the step of extracting a target area image corresponding to a shooting target from the denoised image carried out when the computer program is executed by the processor comprises:
determining, by a target extraction network, an area where the shooting target resides in the denoised image, wherein the target extraction network employs a convolution neural network, and
isolating the area where the shooting target resides from the denoised image to obtain the target area image.

### Claim 16 (depends on 12)

The electronic device according to claim 12, wherein the step of performing detail enhancement on the target area image to generate a detail enhanced image carried out when the computer program is executed by the processor comprises:
inputting the target area image into a detail enhancement network, and
using an output of the detail enhancement network as the detail enhanced image;
wherein the detail enhancement network employs a convolution-deconvolution neural network, and the detail enhanced image and the target area image have the same size.

### Claim 17 (depends on 12)

The electronic device according to claim 12, wherein the step of fusing the denoised image with the detail enhanced image to obtain an output image carried out when the computer program is executed by the processor comprises:
replacing the target area image in the denoised image with the detail enhanced image, and
linearly fusing a boundary area where the detail enhanced image intersects the denoised image.

### Claim 18 (depends on 17)

The electronic device according to claim 17, wherein the step of replacing the target area image in the denoised image with the detail enhanced image carried out when the computer program is executed by the processor comprises:
implementing a replacement process through the following formula:
R_final=aR_206+(1−a)R_202
where R_final is the output image, R_206 is the detail enhanced image, R_202 is the denoised image, a is the area where the target area image resides in the denoised image, and 1−a is other area than the target area image in the denoised image.

### Claim 19 (depends on 17)

The electronic device according to claim 17, wherein the step of linearly fusing a boundary area where the detail enhanced image intersects the denoised image carried out when the computer program is executed by the processor comprises:
linearly fusing a boundary area where the detail enhanced image intersects the denoised image by means of smoothing filtering or wavelet reconstruction.

### Claim 20

A non-transitory computer-readable storage medium having stored thereon a computer program, wherein steps of an image processing method are carried out when the computer program is executed by a processor, the image processing method comprising:
performing denoising on a to-be-processed image to generate a denoised image;
extracting a target area image corresponding to a shooting target from the denoised image;
performing detail enhancement on the target area image to generate a detail enhanced image; and
fusing the denoised image with the detail enhanced image to obtain an output image.
