Image processing apparatus using convolutional neural networks and operating method thereof

An operating method of an image processing apparatus for processing an image by using one or more convolutional neural networks, the operating method comprising:

• dividing a plurality of channels included in input information obtained from a first image into first groups;
• obtaining output data respectively corresponding to the first groups, based on input data respectively corresponding to the first groups by:
• obtaining first feature data based on a first convolution operation being performed between input data and a first kernel;
• dividing a plurality of channels included in the first feature data into second groups;
• obtaining second feature data based on a second convolution operation being performed between the first feature data respectively corresponding to the second groups and second kernels respectively corresponding to the second groups;
• obtaining shuffling data by shuffling the second feature data;
• and
• obtaining the output data by performing a convolution operation between data obtained by concatenating channels included in the shuffling data and a third kernel;
• obtaining output information corresponding to the input information, by summing channels included in the output data respectively corresponding to the first groups;
• obtaining third feature data based on a third convolution operation being performed between the output information and a fourth kernel;
• dividing a plurality of channels included in the third feature data into the first groups;
• obtaining fourth feature data based on a fourth convolution operation being performed between the third feature data respectively corresponding to the first groups and fifth kernels respectively corresponding to the first groups;
• dividing a plurality of channels included in the fourth feature data into the second groups and obtain second shuffling data by shuffling the fourth feature data respectively corresponding to the second groups;
• obtaining fifth feature data based on a fifth convolution operation being performed between the second shuffling data and sixth kernels respectively corresponding to the second groups;
• obtaining sixth feature data respectively corresponding to the first groups by summing channels included in the fifth feature data;
• generating an attention map including weight information corresponding to each of a plurality of pixels included in the first image, based on the sixth feature data;
• generating a spatially variable kernel corresponding to each of the plurality of pixels, based on the attention map and a spatial kernel including weight information according to a position relationship between each of the plurality of pixels and at least one neighboring pixel of each of the plurality of pixels;
• and
• generating a second image by applying the spatially variable kernel to the first image.

A non-transitory computer-readable recording medium having recorded thereon a program for performing an image processing method, the image processing method comprising:

• dividing a plurality of channels included in input information obtained from a first image into first groups;
• obtaining output data respectively corresponding to the first groups, based on input data respectively corresponding to the first groups by:
• obtaining first feature data based on a first convolution operation being performed between input data and a first kernel;
• dividing a plurality of channels included in the first feature data into second groups;
• obtaining second feature data based on a second convolution operation being performed between the first feature data respectively corresponding to the second groups and second kernels respectively corresponding to the second groups;
• obtaining shuffling data by shuffling the second feature data;
• and
• obtaining the output data by performing a convolution operation between data obtained by concatenating channels included in the shuffling data and a third kernel;
• obtaining output information corresponding to the input information, by summing channels included in the output data respectively corresponding to the first groups;
• obtaining third feature data based on a third convolution operation being performed between the output information and a fourth kernel;
• dividing a plurality of channels included in the third feature data into the first groups;
• obtaining fourth feature data based on a fourth convolution operation being performed between the third feature data respectively corresponding to the first groups and fifth kernels respectively corresponding to the first groups;
• dividing a plurality of channels included in the fourth feature data into the second groups and obtain second shuffling data by shuffling the fourth feature data respectively corresponding to the second groups;
• obtaining fifth feature data based on a fifth convolution operation being performed between the second shuffling data and sixth kernels respectively corresponding to the second groups;
• obtaining sixth feature data respectively corresponding to the first groups by summing channels included in the fifth feature data;
• generating an attention map including weight information corresponding to each of a plurality of pixels included in the first image, based on the sixth feature data;
• generating a spatially variable kernel corresponding to each of the plurality of pixels, based on the attention map and a spatial kernel including weight information according to a position relationship between each of the plurality of pixels and at least one neighboring pixel of each of the plurality of pixels;
• and
• generating a second image by applying the spatially variable kernel to the first image.