Camera module and super resolution image processing method thereof

A camera module, comprising:

• an image acquisition unit configured to acquire a plurality of image frames having a spatial phase difference therebetween;
• an image generation unit configured to generate image data having a resolution higher than a resolution of each of the plurality of image frames using the plurality of image frames;
• and
• a depth information extraction unit configured to extract depth information about an object using the image data,
• wherein the image acquisition unit comprises:
• an optical unit configured to change a path along which light for the object travels;
• an image sensor configured to sense light incident along different paths;
• and
• a controller configured to control the optical unit and the image sensor,
• wherein the plurality of image frames corresponds to results sensed in sequence by the image sensor, and
• wherein the image generation unit generates the image data using an intensity obtained below on each of the plurality of image frames:
• D−1Bk−1Ak−nk
• where 1≤k≤p, p represents a number of the image frames used to generate the image data, D−1 represents an inverse matrix of D, D represents a size of a pixel of the image sensor, BK−1 represents an inverse matrix of Bk, Bk represents optical characteristics with respect to the depth information, nk represents a noise component of a kth image frame among p image frames, and Ak represents an intensity of the kth image frame among the p image frames, and is as follows:
• Ak=DBkxϕ+nk,
• where xϕ represents an intensity of the image data and ϕ represents a degree of phase delay.

A super-resolution image processing method of a camera module, comprising:

• acquiring a plurality of image frames having a spatial phase difference therebetween;
• and
• extracting of distance information about an object from the plurality of image frames through an I-time-of-flight (I-ToF) method,
• wherein image data having a resolution higher than a resolution of each of the plurality of image frames is generated using the plurality of image frames, during a process of extracting the distance information,
• wherein the distance information includes a depth information, and
• wherein the depth information about the object is extracted using the generated image data.

A super-resolution image processing method of a camera module, comprising:

• (a) acquiring a plurality of image frames having a spatial phase difference therebetween;
• (b) generating image data having a resolution higher than a resolution of each of the plurality of image frames using the plurality of image frames;
• and
• (c) extracting depth information about an object using the image data,
• wherein step (b) comprises obtaining the image data using an intensity obtained below on each of the plurality of image frames:
• D−1Bk−1Ak−nk
• where 1≤k≤p, p represents a number of the image frames used to generate the image data, D−1 represents an inverse matrix of D, D represents a size of a pixel of an image sensor obtaining each of the image frames, Bk−1 represents an inverse matrix of Bk, Bk represents optical characteristics with respect to the depth information, nk represents a noise component of a kth image frame among p image frames, and Ak represents an intensity of the kth image frame among the p image frames, and is as follows:
• Ak=DBkxϕ+nk
• wherein xϕ represents an intensity of the image data and ϕ represents a degree of phase delay.