Image processing method, image processing apparatus, imaging apparatus, and storage medium

#The present invention relates to an image processing method that processes a plurality of input images using a convolution neural network for the plurality of input images.

#Recently, in order to improve the image quality, an image processing technology using a convolutional neural network (CNN) has begun to be used. The CNN is a learning type image processing technology that convolves a filter generated by learning with an input image, adds a bias generated by the learning, then repeats a nonlinear operation, and convert it into a desired output image. This learning is performed with a learning image including a set of an input learning image and an output learning image. Briefly, learning is to prepare a large amount (such as several tens of thousands) of input learning images corresponding to input images and output learning images corresponding to output images, and to learn a relationship between input and output images based on these learning images.

#For example, Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, Luc Van Gool, “DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks”, arXiv: 1704.02470v2, United States, 2017 (Ignatov et al.), discloses a CNN that acquires an input learning image through a smartphone camera and an output learning image through a digital single-lens camera, and converts a smartphone camera image as an input image into a digital single-lens camera image quality. Thereby, the small smartphone camera can obtain an image close to the image quality of a digital single-lens camera that is a large-sized imaging apparatus.

#FIGS. 1A and 1B are overviews of an imaging apparatus according to a first embodiment.

#FIG. 2 is a block diagram of an imaging apparatus according to the first embodiment.

#FIG. 3 is an explanatory view of a network structure that corrects an image according to the first and second embodiments.

#Referring now to the accompanying drawings, a description will be given of embodiments according to the present invention. Corresponding elements in respective figures will be designated by the same reference numerals, and a description thereof will be omitted.

#Prior to a specific description of embodiments, a gist of the present invention will be described. The present invention reduces a noise amount while restoring a high frequency component in a captured images acquired by a plurality of different image capturers using a convolutional neural network (CNN) as one deep learning type. Each of the embodiments described below generates a high quality image (third image) having a high resolution and a small noise amount particularly based on an image (first image) having a low resolution and a small noise amount and an image (second image) having a high resolution and a large noise amount.

#Referring now to FIGS. 1A, 1B, and 2, a description will be given of an imaging apparatus according to a first embodiment of the present invention. FIGS. 1A and 1B are overviews of the imaging apparatus 1. FIG. 1A is a perspective view, and FIG. 1B is a front view. FIG. 2 is a block diagram of the imaging apparatus 1. In this embodiment, the imaging apparatus 1 executes an image processing method and reduces the noise amount while restoring the high frequency component in a captured image.