# US 2010/0166319 A1 · Image processing apparatus, image processing method, and image processing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0166319 A1 - Document ID: US20100166319A1 - Status: Published application - Kind: A1 - Application number: 12/647,253 - Filed: 2009-12-24 - Published: 2010-07-01 - Earliest claimed: 2008-12-26 - Assignees: FUJIFILM Corporation - Inventors: Yitong Zhang - Web page: https://patentgrep.com/patent/US20100166319A1 ## CPC - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/73: Deblurring; Sharpening - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06V 10/757: {Matching configurations of points or features} - G06T 2207/20192: Edge enhancement; Edge preservation - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The outlines of specific subjects becoming unnaturally deformed is prevented, while reducing the amount of calculations. A reference image setting section sets a reference image which is to become a reference from among a plurality of images. A specific subject detecting section detects a specific subject from within the reference image. A feature point extracting section extracts a plurality of feature points from within the reference image such that the average density of the feature points become higher in the vicinity of the outline of the specific subject. A corresponding point obtaining means corresponding point obtaining section obtains corresponding points within the other images that correspond to the extracted feature points. A coordinate converting section converts the coordinates of the positions of each pixel within the reference image and/or the other images such that the positions of the feature points and the positions of the corresponding points match. ## Claims ### Claim 1 An image processing apparatus, comprising: reference image setting means, for setting a reference image to become a reference from among a plurality of images; detecting means, for detecting a specific subject from within the reference image set by the reference image setting means; feature point extracting means, for extracting a plurality of feature points from within the reference image such that the average density of the feature points become higher in the vicinity of the outline of the specific subject detected by the detecting means than the average density of feature points at other regions; corresponding point obtaining means, for obtaining corresponding points within the plurality of images other than the reference image that correspond to the feature points extracted by the feature point extracting means; and coordinate converting means, for converting the coordinates of the positions of each pixel within at least one of the reference image and the other images such that the positions of the feature points and the positions of the corresponding points that correspond to the feature points match. ### Claim 2 (depends on 1) An image processing apparatus as defined in claim 1, wherein: the plurality of images includes at least three images; and the coordinate converting means converts the coordinates of the positions of each pixel within those images other than the reference image such that the positions of the corresponding points obtained by the corresponding point obtaining means match the positions of the feature points. ### Claim 3 An image processing apparatus, comprising: image group setting means, for setting at least two image groups such that each of at least three images is included into at least one of the image groups; reference image setting means, for setting a final reference image, which is to become a final reference for the at least three images, and for setting reference images, which are to become references in each of the image groups set by the image group setting means; detecting means, for detecting a specific subject from within the reference images of each of the image groups, set by the reference image setting means; feature point extracting means, for extracting a plurality of feature points from within the reference images such that the average density of the feature points become higher in the vicinity of the outline of the specific subject detected by the detecting means than the average density of feature points at other regions; corresponding point obtaining means, for obtaining corresponding points that correspond to the feature points extracted by the feature point extracting means, within images included in each of the image groups other than the reference images; converting function obtaining means, for obtaining coordinate converting functions for positioning the reference images with respect to the final reference image; and coordinate converting means, for converting the coordinates of the positions of each pixel within the other images such that the positions of the corresponding points that correspond to the feature points match the positions of the feature points, and then converting the coordinates of the positions of each pixel of the coordinate converted images employing the coordinate converting functions obtained by the converting function obtaining means. ### Claim 4 (depends on 3) An image processing apparatus as defined in claim 3, wherein: the at least three images are images which are obtained sequentially; and the setting of the image groups is performed to set the at least two image groups such that at least two images which are obtained in sequence are included in each of the image groups. ### Claim 5 (depends on 1) An image processing apparatus as defined in claim 1, wherein: the specific subject is a face or a moving object. ### Claim 6 (depends on 3) An image processing apparatus as defined in claim 3, wherein: the specific subject is a face or a moving object. ### Claim 7 (depends on 1) An image processing apparatus as defined in claim 1, wherein: the specific subject is a face; the detecting means functions to discriminate whether images within a detection frame is an image that represents the specific subject, and performs detection by scanning the detection frame within the reference image; and the vicinity of the outline is at least one of a region inside and a region outside the detection frame having a predetermined width, at a position where the image within the detection frame is discriminated to be an image of the specific subject. ### Claim 8 (depends on 3) An image processing apparatus as defined in claim 3, wherein: the specific subject is a face; the detecting means functions to discriminate whether images within a detection frame is an image that represents the specific subject, and performs detection by scanning the detection frame within the reference image; and the vicinity of the outline is at least one of a region inside and a region outside the detection frame having a predetermined width, at a position where the image within the detection frame is discriminated to be an image of the specific subject. ### Claim 9 (depends on 1) An image processing apparatus as defined in claim 1, wherein: the vicinity of the outline is at least one of a region inside and a region outside the outline of the specific subject having a predetermined width. ### Claim 10 (depends on 3) An image processing apparatus as defined in claim 3, wherein: the vicinity of the outline is at least one of a region inside and a region outside the outline of the specific subject having a predetermined width. ### Claim 11 An image processing method to be executed by an image processing apparatus, comprising the steps of: setting a reference image to become a reference from among a plurality of images; detecting a specific subject from within the set reference image; extracting a plurality of feature points from within the reference image such that the average density of the feature points become higher in the vicinity of the outline of the detected specific subject than the average density of feature points at other regions; obtaining corresponding points within the plurality of images other than the reference image that correspond to the extracted feature points; and coordinate converting means, for converting the coordinates of the positions of each pixel within at least one of the reference image and the other images such that the positions of the feature points and the positions of the corresponding points that correspond to the feature points match. ### Claim 12 An image processing method to be executed by an image processing apparatus, comprising the steps of: setting at least two image groups such that each of at least three images is included into at least one of the image groups; setting a final reference image, which is to become a final reference for the at least three images, and for setting reference images, which are to become references in each of the set image groups; detecting a specific subject from within the reference images set within each of the image groups; extracting a plurality of feature points from within the reference images such that the average density of the feature points become higher in the vicinity of the outline of the detected specific subject than the average density of feature points at other regions; obtaining corresponding points that correspond to the extracted feature points, within images included in each of the image groups other than the reference images; obtaining coordinate converting functions for positioning the reference images with respect to the final reference image; and converting the coordinates of the positions of each pixel within the other images such that the positions of the corresponding points that correspond to the feature points match the positions of the feature points, and then converting the coordinates of the positions of each pixel of the coordinate converted images employing the obtained coordinate converting functions. ### Claim 13 A computer readable recording medium having recorded therein an image processing program that causes a computer to execute the procedures of: setting a reference image to become a reference from among a plurality of images; detecting a specific subject from within the set reference image; extracting a plurality of feature points from within the reference image such that the average density of the feature points become higher in the vicinity of the outline of the detected specific subject than the average density of feature points at other regions; obtaining corresponding points within the plurality of images other than the reference image that correspond to the extracted feature points; and coordinate converting means, for converting the coordinates of the positions of each pixel within at least one of the reference image and the other images such that the positions of the feature points and the positions of the corresponding points that correspond to the feature points match. ### Claim 14 A computer readable recording medium having recorded therein an image processing program that causes a computer to execute the procedures of: setting at least two image groups such that each of at least three images is included into at least one of the image groups; setting a final reference image, which is to become a final reference for the at least three images, and for setting reference images, which are to become references in each of the set image groups; detecting a specific subject from within the reference images set within each of the image groups; extracting a plurality of feature points from within the reference images such that the average density of the feature points become higher in the vicinity of the outline of the detected specific subject than the average density of feature points at other regions; obtaining corresponding points that correspond to the extracted feature points, within images included in each of the image groups other than the reference images; obtaining coordinate converting functions for positioning the reference images with respect to the final reference image; and converting the coordinates of the positions of each pixel within the other images such that the positions of the corresponding points that correspond to the feature points match the positions of the feature points, and then converting the coordinates of the positions of each pixel of the coordinate converted images employing the obtained coordinate converting functions.