# US 8,542,290 B2 · Camera module, image processing apparatus, and image recording method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,290 B2 - Document ID: US8542290B2 - Status: Granted - Kind: B2 - Application number: 12/958,781 - Filed: 2010-12-02 - Published: 2011-06-30 - Granted: 2013-09-24 - Earliest claimed: 2009-12-24 - Assignees: Kabushiki Kaisha Toshiba - Applicants: Takayuki Ogasahara - Inventors: Takayuki Ogasahara - Web page: https://patentgrep.com/patent/US8542290B2 ## CPC - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices - H04N 23/81: for suppressing or minimising disturbance in the image signal generation - H04N 23/958: for extended depth of field imaging - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract According to one embodiment, a camera module includes a plurality of sub-camera modules. The sub-camera modules include imaging elements and imaging lenses. At least two of the sub-camera modules include the imaging lenses, subject distances of which when best-focused are set different from each other. ## Claims ### Claim 1 A camera module comprising: a plurality of sub-camera modules including imaging elements that pick up subject images and imaging lenses that make light captured from a subject incident on the imaging elements; a resolution restoring unit that carries out resolution restoration processing for the subject images; a subject-distance estimating unit that estimates subject distances of the subject images; and a matrix selecting unit that selects a deconvolution matrix based on the subject distances estimated by the subject-distance estimating unit, wherein at least two of the sub-camera modules include the imaging lenses, subject distances of which when best-focused are set different from each other, the subject distances being distances between the sub-camera modules and the subject, and the resolution restoring unit carries out, based on the deconvolution matrix selected by the matrix selecting unit, the resolution restoration processing for each of image data obtained by the sub-camera modules. ### Claim 2 (depends on 1) The camera module according to claim 1, wherein the subject-distance estimating unit estimates the subject distances based on the image data obtained by the at least two sub-camera modules, the subject distances of which when best-focused are set different from each other. ### Claim 3 (depends on 2) The camera module according to claim 2, wherein the subject-distance estimating unit estimates the subject distances by comparing space frequency characteristics quantified from the image data obtained by the at least two sub-camera modules. ### Claim 4 An image processing apparatus comprising: a resolution restoring unit that carries out resolution restoration processing for subject images picked up by a plurality of sub-camera modules; a subject-distance estimating unit that estimates subject distances between the sub-camera modules and a subject; and a matrix selecting unit that selects a deconvolution matrix based on the subject distances estimated by the subject-distance estimating unit, wherein the resolution restoring unit carries out, based on the deconvolution matrix selected by the matrix selecting unit, the resolution restoration processing for each of image data obtained by the sub-camera modules. ### Claim 5 (depends on 4) The image processing apparatus according to claim 4, wherein the sub-camera modules include at least two sub-camera modules, the subject distances of which when best-focused are set different from each other, and the subject-distance estimating unit estimates the subject distances based on the image data obtained by the at least two sub-camera modules. ### Claim 6 (depends on 5) The image processing apparatus according to claim 5, wherein the subject-distance estimating unit estimates the subject distances by comparing space frequency characteristics quantified from the image data obtained by the at least two sub-camera modules. ### Claim 7 (depends on 5) The image processing apparatus according to claim 5, wherein the matrix selecting unit selects, when the subject-distance estimating unit estimates that the subject distances are near distance, the deconvolution matrix for carrying out the resolution restoration processing in the near distance on the image data obtained by the sub-camera modules including the imaging lenses best-focused in infinity, and the matrix selecting unit selects, when the subject-distance estimating unit estimates that the subject distances are the infinite, the deconvolution matrix for carrying out the resolution restoration processing in the infinite on the image data obtained by the sub-camera modules including the imaging lenses best-focused in the near distance. ### Claim 8 (depends on 7) The image processing apparatus according to claim 7, wherein the resolution restoring unit omits the resolution restoration processing concerning the image data other than the image data on which the resolution restoration processing is carried out based on the deconvolution matrix selected by the matrix selecting unit and outputs the image data. ### Claim 9 An image recording method comprising: picking up subject images with a plurality of sub-camera modules including at least two sub-camera modules, subject distances of which when best-focused are set different from each other, the subject distances being distances between the sub-camera modules and a subject; carrying out resolution restoration processing for the subject images; estimating the subject distances of the subject images; selecting a deconvolution matrix based on the estimated subject distances; and carrying out, based on the selected deconvolution matrix, the resolution restoration processing for each of image data obtained by the sub-camera modules. ### Claim 10 (depends on 9) The image recording method according to claim 9, further comprising: picking up color components of the subject images respectively with the sub-camera modules; and picking up same color components of the subject images with the at least two sub-camera modules, the subject distances of which when best-focused are set different from each other. ### Claim 11 (depends on 9) The image recording method according to claim 9, further comprising estimating the subject distances based on the image data obtained by the at least two sub-camera modules, the subject distances of which when best-focused are set different from each other. ### Claim 12 (depends on 11) The image recording method according to claim 11, further comprising estimating the subject distances by comparing space frequency characteristics quantified from the image data obtained by the at least two sub-camera modules. ### Claim 13 (depends on 11) The image recording method according to claim 11, further comprising: selecting, when it is estimated that the subject distances are near distance, the deconvolution matrix for carrying out the resolution restoration processing in the near distance on the image data obtained by the sub-camera modules best-focused in infinity; and selecting, when it is estimated that the subject distances are the infinite, the deconvolution matrix for carrying out the resolution restoration processing in the infinite on the image data obtained by the sub-camera modules best-focused in the near distance. ### Claim 14 (depends on 13) The image recording method according to claim 13, further comprising omitting the resolution restoration processing concerning the image data other than the image data on which the resolution restoration processing is carried out based on the selected deconvolution matrix. ### Claim 15 (depends on 9) The image recording method according to claim 9, further comprising: setting one of the sub-camera modules as a reference sub-camera module; and shifting imaging positions of the subject images by the sub-camera modules other than the reference sub-camera module with respect to an imaging position of the subject image by the reference sub-camera module. ### Claim 16 (depends on 1) The camera module according to claim 1, wherein the sub-camera modules respectively pick up color components of the subject images, and at least two sub-camera modules that pick up same color components of the subject images among the sub-camera modules include the imaging lenses, subject distances of which when best-focused are set different from each other. ### Claim 17 (depends on 16) The camera module according to claim 16, wherein the same color components are green components. ### Claim 18 (depends on 1) The camera module according to claim 1, wherein the imaging lenses of the sub-camera modules configure a lenslet. ### Claim 19 (depends on 1) The camera module according to claim 1, wherein one of the sub-camera modules is set as a reference sub-camera module, and imaging positions of the subject images by the sub-camera modules other than the reference sub-camera module are shifted with respect to an imaging position of the subject image by the reference sub-camera module.