# US 2010/0165113 A1 · Subject tracking computer program product, subject tracking device and camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0165113 A1 - Document ID: US20100165113A1 - Status: Published application - Kind: A1 - Application number: 12/449,403 - Filed: 2008-03-13 - Published: 2010-07-01 - Earliest claimed: 2007-03-16 - Assignees: Nikon Corporation - Inventors: Hiroyuki Abe - Web page: https://patentgrep.com/patent/US20100165113A1 ## CPC - G06T 7/248: {involving reference images or patches} - G06F 18/28: Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries - G06V 10/7515: {Shifting the patterns to accommodate for positional errors} - G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries - G06T 2207/10016: Video; Image sequence - G06T 2207/30201: Face - G06T 2207/30241: Trajectory - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer performs a template-matching step of template-matching an input image with a plurality of template images with different magnifications by a program stored in an object racking computer program product, selecting the template image with the highest similarity to an image in a predetermined region of the input image among the plurality of template images and extracting the predetermined region of the input image as a matched region, a determination step of determining whether or not the matching results of the template-matching step satisfy update conditions for updating the plurality of template images, and an updating step of updating at least one of the plurality of template images when the update conditions are determined to be satisfied by the determination step. ## Claims ### Claim 1 A subject tracking computer program product containing a subject tracking program executed by a computer in order to track movement of a subject through a plurality of input images input in time series, wherein: the subject tracking program enables the computer to execute: a template matching step through which each input image is matched with a plurality of template images at various magnification factors through template matching, a template image among the plurality of template images achieving a highest level of similarity to an image within a specific area in the input image is selected as a chosen template image and the specific area in the input image is extracted as a match-up area; a decision step through which a decision is made as to whether or not matching results obtained through the template matching step satisfy an update condition for updating the plurality of template images; and an update step through which at least one of the plurality of template images is updated upon deciding through the decision step that the update condition is satisfied. ### Claim 2 (depends on 1) A subject tracking computer program product according to claim 1, wherein: the plurality of template images include a current chosen template image selected as the chosen template image as the plurality of template images are updated through the update step, an enlarged template image obtained by enlarging the current chosen template image by a predetermined magnification factor and a reduced template image obtained by reducing the current chosen template image by a predetermined magnification factor. ### Claim 3 (depends on 2) A subject tracking computer program product according to claim 2, wherein: in the decision step, the decision as to whether or not the update condition is satisfied is made based upon a criterion that either the enlarged template image or the reduced template image has been consecutively selected as the chosen template image a predetermined number of times or more. ### Claim 4 (depends on 2) A subject tracking computer program product according to claim 2, wherein: in the decision step, the decision as to whether or not the update condition is satisfied is made based upon a criterion that levels of similarity determined in correspondence to either the enlarged template image or the reduced template image having been consecutively selected as the chosen template image a predetermined number of times or more, are all equal to or greater than a predetermined similarity level value or that an average of the said levels of similarity is equal to or greater than the predetermined similarity level value. ### Claim 5 A subject tracking computer program product containing a subject tracking program executed by a computer in order to track movement of a subject through a plurality of input images input in time series, wherein: the subject tracking program enables the computer to execute: a template matching step through which each input image is matched through template matching with a plurality of template images including a plurality of template images (magnification factor—varied template images) at various magnification factors and a plurality of template images (time-lagged template images) obtained at different time points, a template image among the plurality of template images achieving a highest level of similarity to an image within a specific area in the input image is selected as a chosen template image and the specific area in the input image is extracted as a match-up area; a decision step through which a decision is made as to whether or not matching results obtained through the template matching step satisfy an update condition for updating the plurality of template images; and an update step through which at least one of the plurality of template images is updated upon deciding through the decision step that the update condition is satisfied. ### Claim 6 (depends on 5) A subject tracking computer program product according to claim 5, wherein: the plurality of magnification factor-varied template images include a current chosen template image selected as the chosen template image as the plurality of template images are updated through the update step, an enlarged template image obtained by enlarging the current chosen template image by a predetermined magnification factor and a reduced template image obtained by reducing the current chosen template image by a predetermined magnification factor; and the plurality of time-lagged template images include a previous chosen template image having been set as the current chosen template image until the plurality of template images are updated through the update step, a new chosen template image constituted with an image contained in the match-up area when the plurality of template images are updated through the update step and an update-exempt template image having been obtained at an earliest time point among the plurality of template images, which is not updated through the update step. ### Claim 7 (depends on 5) A subject tracking computer program product according to claim 5, wherein: the plurality of time-lagged template images include a previous chosen template image having been set as the current chosen template image until the plurality of template images are updated through the update step, a new chosen template image constituted with an image contained in the match-up area when the plurality of template images are updated through the update step and an update-exempt template image having been obtained at an earliest time point among the plurality of template images, which is not updated through the update step; and the plurality of magnification factor-varied template images include the new chosen template image, an enlarged template image obtained by enlarging the new chosen template image by a predetermined magnification factor and a reduced template image obtained by reducing the new chosen template image by a predetermined magnification factor. ### Claim 8 (depends on 6) A subject tracking computer program product according to claim 6, wherein: in the decision step, the decision as to whether or not the update condition is satisfied is made based upon a criterion that any one of the previous chosen template image, the new chosen template image, the enlarged template image, the reduced template image and the update-exempt template image has been consecutively selected as the chosen template image a predetermined number of times or more. ### Claim 9 (depends on 6) A subject tracking computer program product according to claim 6, wherein: in the decision step, the decision as to whether or not the update condition is satisfied is made based upon a criterion that levels of similarity determined in correspondence to any one of the previous chosen template image, the new chosen template image, the enlarged template image, the reduced template image and the update-exempt template image having been consecutively selected as the chosen template image a predetermined number of times or more are all equal to or greater than a predetermined similarity level value or that an average of the said levels of similarity is equal to or greater than a predetermined similarity level value. ### Claim 10 (depends on 1) A subject tracking computer program product according to claim 1, wherein: the plurality of template images at various magnification factors are either a plurality of template images of uniform template image size, each obtained by adjusting an image magnification factor alone, or a plurality of template images each obtained by adjusting the image magnification factor and assuming a template image size adjusted in correspondence to the adjusted magnification factor. ### Claim 11 (depends on 1) A subject tracking device comprising: an execution unit that executes the program contained in the subject tracking computer program product according to claim 1. ### Claim 12 (depends on 1) A camera comprising: an imaging unit that captures a subject image and outputs image signals; and an execution unit that executes the program contained in the computer program product according to claim 1. ### Claim 13 (depends on 5) A subject tracking computer program product according to claim 5, wherein: the plurality of template images at various magnification factors are either a plurality of template images of uniform template image size, each obtained by adjusting an image magnification factor alone, or a plurality of template images each obtained by adjusting the image magnification factor and assuming a template image size adjusted in correspondence to the adjusted magnification factor. ### Claim 14 (depends on 5) A subject tracking device comprising: an execution unit that executes the program contained in the subject tracking computer program product according to claim 5. ### Claim 15 (depends on 5) A camera comprising: an imaging unit that captures a subject image and outputs image signals; and an execution unit that executes the program contained in the computer program product according to claim 5.