# US 2025/0095103 A1 · Image processing apparatus, image pickup apparatus, image processing method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095103 A1 - Document ID: US20250095103A1 - Status: Published application - Kind: A1 - Application number: 18/966,821 - Filed: 2024-12-03 - Published: 2025-03-20 - Earliest claimed: 2021-02-24 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: YUICHI KUSUMI - Web page: https://patentgrep.com/patent/US20250095103A1 ## CPC - G06T 5/10: using non-spatial domain filtering - G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/73: Deblurring; Sharpening - H04N 25/48: Increasing resolution by shifting the sensor relative to the scene - H04N 25/61: the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" - H04N 25/615: involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/20192: Edge enhancement; Edge preservation - G06T 2207/30208: Marker matrix - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing apparatus includes at least one processor or circuit configured to execute a plurality of tasks including an image acquiring task configured to acquire a first image and a second image that have been obtained by imaging at imaging positions different from each other, an image restoration processing task configured to acquire a first restored image and a second restored image by performing image restoration processing for the first image and the second image respectively, and a pixel increasing processing task configured to acquire a pixel increased image using the first restored image and the second restored image. ## Claims ### Claim 1 An image processing apparatus comprising at least one processor or circuit configured to execute a plurality of tasks including: an image acquiring task configured to acquire a first image and a second image that have been obtained by imaging at imaging positions different from each other; an image restoration processing task configured to acquire a first restored image and a second restored image by performing image restoration processing for the first image and the second image respectively; and a pixel increasing processing task configured to acquire a pixel increased image using the first restored image and the second restored image. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the pixel increased image has pixels more than those of each of the first image and the second image. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the tasks further include a restoration filter acquiring task configured to acquire an image restoration filter, and wherein the image restoration processing task performs the image restoration processing for the first image and the second image using the image restoration filter. ### Claim 4 (depends on 3) The image processing apparatus according to claim 3, wherein the image restoration filter is generated based on an optical transfer function at a frequency lower than a Nyquist frequency and an optical transfer function at a frequency higher than the Nyquist frequency. ### Claim 5 (depends on 3) The image processing apparatus according to claim 3, wherein the image restoration filter is generated based on a restoration characteristic at a frequency lower than a Nyquist frequency and a restoration characteristic at a frequency higher than the Nyquist frequency. ### Claim 6 (depends on 3) The image processing apparatus according to claim 3, wherein the image restoration filter is generated based on optical information on an imaging optical system. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the tasks further include a processing determining task configured to determine whether to perform first processing or second processing, wherein in a case where the processing determining task determines that the first processing is to be performed, the image restoration processing task performs the image restoration processing for the first image and the second image, and acquires the first restored image and the second restored image, and the pixel increasing processing task acquires the pixel increased image using the first restored image and the second restored image, and wherein in a case where the processing determining task determines that the second processing is to be performed, the pixel increasing processing task acquires the pixel increased image using the first image and the second image, and the image restoration processing task performs the image restoration processing for the pixel increased image and acquires a restored pixel increased image. ### Claim 8 (depends on 7) The image processing apparatus according to claim 7, wherein the processing determining task determines, based on restoration target data, whether to perform the first processing or the second processing, and wherein the processing determining task determines that the first processing is to be performed in a case where the restoration target data relates to a moving image, and that the second image is to be performed in a case where the restoration target data relates to a still image. ### Claim 9 (depends on 7) The image processing apparatus according to claim 7, wherein the processing determining task determines whether to perform the first processing or the second processing based on a point spread function or an optical transfer function of an imaging optical system. ### Claim 10 (depends on 9) The image processing apparatus according to claim 9, wherein the processing determining task determines that the first processing is to be performed in a case where the point spread function is larger than a predetermined value, and that the second processing is to be performed in a case where the point spread function is smaller than the predetermined value. ### Claim 11 (depends on 9) The image processing apparatus according to claim 9, wherein the processing determining task determines that the first processing is to be performed in a case where an absolute value of the optical transfer function is smaller than a threshold at a predetermined frequency, and that the second processing is to be performed in a case where the absolute value of the optical transfer function is larger than the threshold at the predetermined frequency. ### Claim 12 (depends on 7) The image processing apparatus according to claim 7, wherein the processing determining task determines whether to perform the first processing or the second processing, based on an F-number of an imaging optical system. ### Claim 13 (depends on 12) The image processing apparatus according to claim 12, wherein the processing determining task determines that the first processing is to be performed in a case where the F-number is larger than a predetermined F-number, and that the second processing is to be performed in a case where the F-number is smaller than the predetermined F-number. ### Claim 14 (depends on 12) The image processing apparatus according to claim 12, wherein the processing determining task determines that the first processing is to be performed in a case where the F-number is smaller than a predetermined F-number, and that the second processing is to be performed in a case where the F-number is larger than the predetermined F-number. ### Claim 15 (depends on 7) The image processing apparatus according to claim 7, wherein the number of taps in an image restoration filter for the image restoration processing in the first processing is different from that in the second processing. ### Claim 16 An image pickup apparatus comprising: an image sensor configured to photoelectrically convert an optical image formed by an imaging optical system; and an image processing apparatus, wherein the image processing apparatus includes at least one processor or circuit configured to execute a plurality of tasks including: an image acquiring task configured to acquire a first image and a second image that have been obtained by imaging at imaging positions different from each other; an image restoration processing task configured to acquire a first restored image and a second restored image by performing image restoration processing for the first image and the second image respectively; and a pixel increasing processing task configured to acquire a pixel increased image using the first restored image and the second restored image. having imaging positions different from each other. ### Claim 17 (depends on 16) The image pickup apparatus according to claim 16, wherein the imaging optical system includes a phase mask. ### Claim 18 An image processing method comprising the steps of: acquiring a first image and a second image that have been obtained by imaging at imaging positions different from each other; acquiring a first restored image and a second restored image by performing image restoration processing for the first image and the second image respectively; and acquiring a pixel increased image using the first restored image and the second restored image. ### Claim 19 (depends on 18) A storage medium storing a program for causing a computer to execute the image processing method according to claim 18.