# US 11,308,580 B2 · Image processing apparatus, image processing method, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,580 B2 - Document ID: US11308580B2 - Status: Granted - Kind: B2 - Application number: 16/893,231 - Filed: 2020-06-04 - Published: 2020-12-10 - Granted: 2022-04-19 - Earliest claimed: 2019-06-07 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Tohru Ishibashi - Web page: https://patentgrep.com/patent/US11308580B2 ## CPC - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 5/20: using local operators - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/90: Dynamic range modification of images or parts thereof - G06T 2207/20212: Image combination - G06T 2207/20224: Image subtraction - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus includes an acquisition unit configured to acquire a plurality of continuous images, an extraction unit configured to extract a difference between two images among the plurality of continuous images, a first image processing unit configured to acquire a third image by removing a difference between portions smaller than a predetermined size from the difference extracted by the extraction unit, and a first combining unit configured to combine the two images by using the third image acquired by the first image processing unit. The image processing apparatus may implement the aforementioned units using a processor(s) that execute(s) a program of instructions stored in memory. ## Claims ### Claim 1 An image processing apparatus comprising: at least one memory that stores a program of instructions; and at least one processor that executes the program of instructions to cause the image processing apparatus to implement: an acquisition unit configured to acquire a plurality of continuous images; an extraction unit configured to extract a difference between two images among the plurality of continuous images; a first image processing unit configured to acquire a third image by removing a difference between portions smaller than a predetermined size from the difference extracted by the extraction unit, and the predetermined size is determined based on imaging information obtained when the plurality of continuous images is acquired, wherein the imaging information includes information on a focal length of an imaging lens or a pixel pitch of an imaging element, the acquired images being display on a display unit; and a first combining unit configured to combine the two images by using the third image acquired by the first image processing unit. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the at least one processor executes the program of instructions to cause the image processing apparatus to implement a second combining unit configured to select and combine brighter portions of a plurality of images, and wherein the second combining unit combines an image combined by the first combining unit with one of the plurality of continuous images or an image already combined by the second combining unit. ### Claim 3 (depends on 2) The image processing apparatus according to claim 2, wherein the at least one processor executes the program of instructions to cause the image processing apparatus to implement a determination unit configured to determine whether the first combining unit combines the two images based on an extraction result of the extraction unit, and wherein, in a case where the determination unit determines that the first combining unit combines the two images, the first combining unit combines the two images by using the third image acquired by the first image processing unit. ### Claim 4 (depends on 3) The image processing apparatus according to claim 3, wherein the extraction unit extracts the difference by comparing integration results obtained by integrating brightness for each region in the two images, and wherein the determination unit determines that the first combining unit combines the two images in a case where there is a region where the difference between the integration results is equal to or greater than a predetermined value. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the first image processing unit performs a minimum-value filtering process and a maximum-value filtering process to acquire an image obtained by removing a difference between portions smaller than the predetermined size. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, wherein the two images are temporally continuous images. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the at least one processor executes the program of instructions to cause the image processing apparatus to implement a second image processing unit configured to perform gamma correction on the third image acquired by the first image processing unit, and wherein the first combining unit combines the two images by using the image subjected to the gamma correction. ### Claim 8 An image processing method comprising: acquiring a plurality of continuous images; extracting a difference between two images among the plurality of continuous images; performing image processing to acquire a third image by removing a difference between portions smaller than a predetermined size from the difference extracted in the extracting, and the predetermined size is determined based on imaging information obtained when the plurality of continuous images is acquired, wherein the imaging information includes information on a focal length of an imaging lens or a pixel pitch of an imaging element, the acquired images being display on a display unit; and first combining the two images by using the third image acquired in the image processing. ### Claim 9 (depends on 8) The image processing method according to claim 8, further comprising second combining, wherein the second combining includes: selecting and combining brighter portions of a plurality of images, and combining combines an image combined by the first combining with one of the plurality of continuous images or an image already combined by the second combining. ### Claim 10 (depends on 8) The image processing method according to claim 8, further comprising performing a minimum-value filtering process and a maximum-value filtering process to acquire an image obtained by removing a difference between portions smaller than the predetermined size. ### Claim 11 (depends on 8) The image processing method according to claim 8, wherein the two images are temporally continuous images. ### Claim 12 (depends on 8) The image processing method according to claim 8, further comprising performing gamma correction on the acquired third image, wherein the first combining combines the two images by using the image subjected to the gamma correction. ### Claim 13 A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method, the method comprising: acquiring a plurality of continuous images; extracting a difference between two images among the plurality of continuous images; performing image processing to acquire a third image by removing a difference between portions smaller than a predetermined size from the difference extracted in the extracting, and the predetermined size is determined based on imaging information obtained when the plurality of continuous images is acquired, wherein the imaging information includes information on a focal length of an imaging lens or a pixel pitch of an imaging element, the acquired images being display on a display unit; and first combining the two images by using the third image acquired in the image processing. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further comprising second combining, wherein the second combining includes: selecting and combining brighter portions of a plurality of images, and combining combines an image combined by the first combining with one of the plurality of continuous images or an image already combined by the second combining. ### Claim 15 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further comprising performing a minimum-value filtering process and a maximum-value filtering process to acquire an image obtained by removing a difference between portions smaller than the predetermined size. ### Claim 16 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, wherein the two images are temporally continuous images. ### Claim 17 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further comprising performing gamma correction on the acquired third image, wherein the first combining combines the two images by using the image subjected to the gamma correction.