# US 2021/0243377 A1 · Apparatus to perform alignment to images, image processing method to perform alignment to images, and computer readable non-transitory memory to perform alignment to images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243377 A1 - Document ID: US20210243377A1 - Status: Published application - Kind: A1 - Application number: 17/153,637 - Filed: 2021-01-20 - Published: 2021-08-05 - Earliest claimed: 2020-01-30 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Takao Sasaki - Web page: https://patentgrep.com/patent/US20210243377A1 ## CPC - H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics - H04N 23/62: Control of parameters via user interfaces - H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters} - H04N 23/635: {Region indicators; Field of view indicators} - H04N 23/673: {based on contrast or high frequency components of image signals, e.g. hill climbing method} - H04N 23/675: {comprising setting of focusing regions} - H04N 23/676: {Bracketing for image capture at varying focusing conditions} - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An apparatus includes at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions to detect a feature amount from each of a plurality of images different in in-focus position in an optical axis direction, and align the plurality of images by using the feature amount. The at least one processor further executes instructions to detect the feature amount by preferentially using a focusing area of each of the plurality of images. ## Claims ### Claim 1 An apparatus, comprising: at least one memory configured to store instructions; and at least one processor in communication with the at least one memory and configured to execute the instructions to: detect a feature amount from each of a plurality of images different in in-focus position in an optical axis direction; and align the plurality of images by using the feature amount, wherein the at least one processor further executes instructions to detect the feature amount by preferentially using a focusing area of each of the plurality of images. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the at least one processor further executes instructions to generate a composite image by extracting the focusing areas of the respective images. ### Claim 3 (depends on 2) The apparatus according to claim 2, wherein the composite image has a depth of field deeper than a depth of field of each of the plurality of images. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein the feature amount is a motion vector. ### Claim 5 (depends on 4) The apparatus according to claim 4, wherein the at least one processor further executes instructions to detect the motion vector from a contrast of each of the plurality of images. ### Claim 6 (depends on 4) The apparatus according to claim 4, wherein the at least one processor further executes instructions to: arrange a plurality of blocks in each of the plurality of images; and detect the motion vector by using the plurality of blocks. ### Claim 7 (depends on 6) The apparatus according to claim 6, wherein the at least one processor further executes instructions to: arrange the plurality of blocks in each of the plurality of images; detect the motion vector from the plurality of blocks; and in a case where the detected motion vectors do not satisfy a predetermined condition, re-arrange the plurality of blocks. ### Claim 8 (depends on 7) The apparatus according to claim 7, wherein the at least one processor further executes instructions to arrange a plurality of blocks similar to the plurality of blocks in each of the plurality of blocks. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein the at least one processor further executes instructions to re-arrange the plurality of blocks based on the focusing area. ### Claim 10 (depends on 9) The apparatus according to claim 9, wherein the at least one processor further executes instructions to re-arrange the plurality of blocks with at least one of a changed number of blocks, a changed size of each of the blocks, and changes positions of the blocks. ### Claim 11 (depends on 7) The apparatus according to claim 7, wherein the predetermined condition is a condition where a number of the motion vectors detected is greater than a predetermined number. ### Claim 12 (depends on 7) The apparatus according to claim 7, wherein the predetermined condition is a condition where a magnitude of each of the motion vectors is greater than a predetermined threshold value. ### Claim 13 (depends on 7) The apparatus according to claim 7, wherein the predetermined condition is a condition where deviation of the motion vectors is less than a predetermined threshold value. ### Claim 14 (depends on 1) The apparatus according to claim 1, wherein the at least one processor further executes instructions to detect the focusing area of each of the plurality of images by using contrast of the plurality of images. ### Claim 15 An apparatus comprising: a sensor to pick up a plurality of images at different in-focus positions in an optical axis direction; at least one memory configured to store instructions; and at least one processor in communication with the at least one memory and configured to execute the instructions to: detect a feature amount from each of the plurality of images; and align the plurality of images by using the feature amount, wherein the at least one processor further executes instructions to detect the feature amount by preferentially using a focusing area of each of the plurality of images. ### Claim 16 (depends on 15) The apparatus according to claim 15, wherein the sensor records the focusing areas of the plurality of images in imaging. ### Claim 17 A method comprising: detecting a feature amount from each of a plurality of images different in in-focus position in an optical axis direction; and aligning the plurality of images by using the feature amount, wherein, in the detecting, the feature amount is detected by preferentially using a focusing area of each of the plurality of images. ### Claim 18 (depends on 17) The method according to claim 17, further comprising: generating a composite image by extracting the focusing areas of the respective images. ### Claim 19 A non-transitory computer-readable storage medium which stores a program for causing a computer of an apparatus to execute a method, the method comprising: detecting a feature amount from each of a plurality of images different in in-focus position in an optical axis direction; and aligning the plurality of images by using the feature amount, wherein, in the detecting, the feature amount is detected by preferentially using a focusing area of each of the plurality of images. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium according to claim 19, further comprising: generating a composite image by extracting the focusing areas of the respective images.