# US 2017/0257581 A1 · Imaging device, imaging method, and image processing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0257581 A1 - Document ID: US20170257581A1 - Status: Published application - Kind: A1 - Application number: 15/499,911 - Filed: 2017-04-28 - Published: 2017-09-07 - Earliest claimed: 2014-11-20 - Assignees: FUJIFILM Corporation - Applicants: FUJIFILM Corporation - Inventors: Shuji ONO - Web page: https://patentgrep.com/patent/US20170257581A1 ## CPC - H04N 25/615: involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] - G02B 13/0085: {employing wafer level optics} - G03B 19/07: having more than one objective - H10F 39/18: Complementary metal-oxide-semiconductor [CMOS] image sensors; Photodiode array image sensors - H10F 39/8063: {Microlenses} - H10F 39/811: {Interconnections} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION ## Abstract The imaging device includes a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property, an image sensor in which a first light receiving element 25A having a first microlens and a second light receiving element 25B having a second microlens having a different focusing degree from the first microlens are two-dimensionally arranged, and a crosstalk removal processing unit that removes a crosstalk component from each of a first crosstalk image acquired from the first light receiving element 25A of the image sensor and a second crosstalk image acquired from the second light receiving element to generate a first image and a second image respectively having the first property and the second property of the multiple-property lens. ## Claims ### Claim 1 An imaging device comprising: a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property; an image sensor that has a first light receiving element having a first microlens and a second light receiving element having a second microlens having a different focusing degree from the first microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the first microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the second microlens being two-dimensionally arranged; and a crosstalk removal processing unit that removes a crosstalk component from each of a first crosstalk image acquired from the first light receiving element of the image sensor and a second crosstalk image acquired from the second light receiving element to generate a first image and a second image respectively having the first property and the second property of the multiple-property lens. ### Claim 2 An imaging device comprising: a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property; an image sensor that has a first light receiving element having a microlens and a second light receiving element not having a microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other; and a crosstalk removal processing unit that removes a crosstalk component from each of a first crosstalk image acquired from the first light receiving element of the image sensor and a second crosstalk image acquired from the second light receiving element to generate a first image and a second image respectively having the first property and the second property of the multiple-property lens. ### Claim 3 (depends on 1) The imaging device according to claim 1, wherein the crosstalk removal processing unit calculates the first image and the second image on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image. ### Claim 4 (depends on 2) The imaging device according to claim 2, wherein the crosstalk removal processing unit calculates the first image and the second image on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image. ### Claim 5 (depends on 1) The imaging device according to claim 1, wherein the first microlens and the second microlens are respectively disposed in each first light receiving element and each second light receiving element, and wherein the first microlens forms, on a light receiving surface of the first light receiving element, a pupil image of the multiple-property lens at a first focusing degree, and the second microlens forms, on a light receiving surface of the second light receiving element, a pupil image of the multiple-property lens at a second focusing degree that is different from the first focusing degree. ### Claim 6 (depends on 1) The imaging device according to claim 1, wherein the first microlens and the second microlens are different from each other in terms of at least one of a radius of curvature or a distance from the light receiving surface of the first light receiving element and from the light receiving surface of the second light receiving element. ### Claim 7 (depends on 2) The imaging device according to claim 2, wherein the first microlens and the second microlens are different from each other in terms of at least one of a radius of curvature or a distance from the light receiving surface of the first light receiving element and from the light receiving surface of the second light receiving element. ### Claim 8 (depends on 1) The imaging device according to claim 1, wherein the image sensor is a rear surface irradiation image sensor. ### Claim 9 (depends on 1) The imaging device according to claim 1, wherein the multiple-property lens includes a first optical system corresponding to the first area having the first property and a second optical system corresponding to the second area having the second property. ### Claim 10 (depends on 9) The imaging device according to claim 9, wherein the first optical system and the second optical system have a common optical axis. ### Claim 11 (depends on 10) The imaging device according to claim 10, wherein the first optical system is a circular central optical system, and the second optical system is an annular optical system that is concentrically arranged with respect to the central optical system. ### Claim 12 (depends on 9) The imaging device according to claim 9, wherein the first optical system and the second optical system are different from each other in terms of at least one of a focal length, a focal distance, or a modulation transfer function. ### Claim 13 An imaging method in an imaging device including a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property, and an image sensor that has a first light receiving element having a first microlens and a second light receiving element having a second microlens having a different focusing degree from the first microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the first microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the second microlens being two-dimensionally arranged, the method comprising: a step of acquiring a first crosstalk image and a second crosstalk image that are imaged by the imaging device at the same time, the first crosstalk image being photoelectrically converted by the first light receiving element of the image sensor, and the second crosstalk image being photoelectrically converted by the second light receiving element; and a step of generating a first image and a second image respectively having the first property and the second property of the multiple-property lens by removing a crosstalk component from each of the first crosstalk image and the second crosstalk image acquired. ### Claim 14 An imaging method in an imaging device including a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property, and an image sensor that has a first light receiving element having a microlens and a second light receiving element not having a microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other being two-dimensionally arranged, the method comprising: a step of acquiring a first crosstalk image and a second crosstalk image that are imaged by the imaging device at the same time, the first crosstalk image being photoelectrically converted by the first light receiving element of the image sensor, and the second crosstalk image being photoelectrically converted by the second light receiving element; and a step of generating a first image and a second image respectively having the first property and the second property of the multiple-property lens by removing a crosstalk component from each of the first crosstalk image and the second crosstalk image acquired. ### Claim 15 (depends on 13) The imaging method according to claim 13, wherein in the step of generating the first image and the second image, the first image and the second image are calculated on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image. ### Claim 16 (depends on 14) The imaging method according to claim 14, wherein in the step of generating the first image and the second image, the first image and the second image are calculated on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image. ### Claim 17 A non-transitory computer readable recording medium storing an image processing program of an imaging device including a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property, and an image sensor that has a first light receiving element having a first microlens and a second light receiving element having a second microlens having a different focusing degree from the first microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the first microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the second microlens being two-dimensionally arranged, the program causing a computer to realize: a step of acquiring a first crosstalk image and a second crosstalk image that are imaged by the imaging device at the same time, the first crosstalk image being photoelectrically converted by the first light receiving element of the image sensor, and the second crosstalk image being photoelectrically converted by the second light receiving element; and a step of generating a first image and a second image respectively having the first property and the second property of the multiple-property lens by removing a crosstalk component from each of the first crosstalk image and the second crosstalk image acquired. ### Claim 18 A non-transitory computer readable recording medium storing an image processing program of an imaging device including a multiple-property lens that includes a first area having a first property and a second area having a second property different from the first property, and an image sensor that has a first light receiving element having a microlens and a second light receiving element not having a microlens, the first light receiving element on which a luminous flux passing the first area of the multiple-property lens and a luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other through the microlens and the second light receiving element having a different crosstalk ratio due to having a different focusing degree from the first light receiving element and on which the luminous flux passing the first area of the multiple-property lens and the luminous flux passing the second area of the multiple-property lens are incident in crosstalk with each other being two-dimensionally arranged, the program causing a computer to realize: a step of acquiring a first crosstalk image and a second crosstalk image that are imaged by the imaging device at the same time, the first crosstalk image being photoelectrically converted by the first light receiving element of the image sensor, and the second crosstalk image being photoelectrically converted by the second light receiving element; and a step of generating a first image and a second image respectively having the first property and the second property of the multiple-property lens by removing a crosstalk component from each of the first crosstalk image and the second crosstalk image acquired. ### Claim 19 (depends on 17) The non-transitory computer readable recording medium storing the image processing program according to claim 17, wherein in the step of generating the first image and the second image, the first image and the second image are calculated on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image. ### Claim 20 (depends on 18) The non-transitory computer readable recording medium storing the image processing program according to claim 18, wherein in the step of generating the first image and the second image, the first image and the second image are calculated on the basis of the first crosstalk image, the second crosstalk image, and an inverse matrix of a matrix configured of a detected gain distribution and a crosstalk gain distribution of each of the first crosstalk image and the second crosstalk image.