Compound-eye imaging device, image processing method, and recording medium

An image processing method comprising:

• acquiring a plurality of images acquired by imaging in a plurality of imaging regions onto which images of a same field of view are imaged, the plurality of images having different types of information and different resolutions;
• performing resolution enhancement processing on images of the plurality of images having the different resolutions that have relatively low resolution by using a high-resolution component included in an image of the plurality of images having the different resolutions that has relatively high resolution, thereby generating high-resolution images having different types of information;
• and
• combining the high-resolution images having the different types of information, thereby generating one or more combined high-resolution images,
• wherein the performing resolution enhancement processing includes:
• extracting, from the images having relatively low resolution, low-frequency components and high-frequency components of the images;
• extracting, from the image having relatively high resolution, a low-frequency component and a high-frequency component of the image;
• enlarging the low-frequency components of the images having relatively low resolution to a same resolution as the image having relatively high resolution, combining the enlarged low-frequency components and the low-frequency component of the image having relatively high resolution by weighted addition to generate combined low-frequency components;
• enlarging the high-frequency components of the images having relatively low resolution to a same resolution as the image having relatively high resolution, combining the enlarged high-frequency components and the high-frequency component of the image having relatively high resolution by weighted addition to generate combined high-frequency components;
• and
• combining the combined low-frequency components and the combined high-frequency components to generate the high-resolution images.

A compound-eye imaging device comprising:

• a camera device comprising:
• an imaging element including a plurality of imaging regions;
• a plurality of lenses disposed to correspond to the plurality of imaging regions and configured to image images of a same field of view onto the corresponding imaging regions;
• and
• a plurality of optical filters,
• wherein the plurality of imaging regions include at least one imaging region of a first type and a plurality of imaging regions of a second type having a same size and being smaller than the imaging region of the first type,
• wherein the plurality of lenses have focal lengths such that lenses corresponding to larger imaging regions have greater focal lengths, and
• wherein the plurality of optical filters are provided so that an image acquired in each of the imaging regions of the second type represents a different type of information from an image acquired in each of the at least one imaging region of the first type;
• and
• a processor including at least one resolution enhancer, the at least one resolution enhancer including:
• a reduction processor to receive, as a reference image, the image acquired in one of the at least one imaging region of the first type, and reduce the reference image to generate a reduced reference image having a same resolution as a low-resolution image acquired in one of the imaging regions of the second type;
• a coefficient calculator to calculate linear coefficients approximating a linear relationship between the reduced reference image and the low-resolution image;
• a coefficient map enlarger to enlarge a coefficient map including the linear coefficients calculated by the coefficient calculator to a same resolution as the reference image;
• and
• a linear converter to generate a high-resolution image having information represented by the low-resolution image on a basis of linear coefficients of the enlarged coefficient map and the reference image.