# US 7,613,321 B2 · Moving object tracking method using occlusion detection of the tracked object, and image processing apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,613,321 B2 - Document ID: US7613321B2 - Status: Granted - Kind: B2 - Application number: 11/055,979 - Filed: 2005-02-14 - Published: 2005-09-08 - Granted: 2009-11-03 - Earliest claimed: 2004-03-05 - Assignees: Sony Corporation - Inventors: Jean-Aymeric Altherr; Masami Ogata; Shinichiro Gomi - Web page: https://patentgrep.com/patent/US7613321B2 ## CPC - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 2207/10016: Video; Image sequence - G06T 2207/30241: Trajectory - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus for performing a tracking of a moving object by setting a tracking area corresponding to a moving object appearing in a video image, by setting a plurality of feature points in the tracking area, and by tracking the plurality of feature points, comprises a tracking processing unit for tracking each of the plurality of feature points from the preceding frame to the present frame, and a motion calculation processing unit for specifying a position of the tracking area in the present frame by estimating the motion of the tracking area based the tracking result of each of the feature points, wherein the tracking processing unit calculates reliability indicating the height of the possibility that a feature point exists in the moving object to each of the feature points, and the motion calculation processing unit calculates the motion of the tracking area by using the reliability. ## Claims ### Claim 1 An image processing apparatus for performing a tracking of a moving object by setting a tracking area corresponding to a moving object appearing in a video image, by setting a plurality of feature points in the tracking area, and by tracking the plurality of feature points, comprising: a tracking processing unit configured to track each of the plurality of feature points from the preceding frame to the present frame; a motion calculation processing unit configured to specify a position of the tracking area in the present frame by estimating the motion of the tracking area based on the tracking result of each of the feature points; and an occlusion detecting unit configured to detect the occurrence of an occlusion of the moving object overlapping with an other object, said occlusion detecting unit calculates an occlusion value indicating the possibility of the occurrence of the occlusion of the moving object by utilizing a change of a number of feature points being successfully tracked, wherein said tracking processing unit calculates a reliability indicator indicating that a feature point has been tracked in the moving object for each of the feature points; and said motion calculation processing unit calculates the motion of the tracking area using the reliability indicator of each feature point and the occlusion value; wherein said occlusion detecting unit sets a plurality of groups of feature points among the feature points based on the respective reliability indicators; and said occlusion detecting unit calculates the occlusion value based on a change of the number of the feature points being successfully tracked among different groups of the plurality of groups of the feature points. ### Claim 2 (depends on 1) The image processing apparatus as cited in claim 1, wherein said reliability indicators indicate the period during which each feature point has been tracked in the moving object. ### Claim 3 (depends on 2) The image processing apparatus as cited in claim 2, wherein said reliability indicators indicate a history which reflects the number of frames in which tracking continuously succeeds for each feature point. ### Claim 4 (depends on 1) The image processing apparatus as cited in claim 1, further comprising: a selection processing unit configured to select a feature point, wherein said tracking processing unit detects the changes of a position before and after tracking to detect a still point among the feature points the tracking of which succeeds on the basis of a detection result; and said selection processing unit selects a new feature point to replace a subset of the feature points among the detected still points. ### Claim 5 (depends on 1) The image processing apparatus as cited in claim 1, further comprising: a selection processing unit configured to select a feature point, wherein said tracking processing unit judges whether or not the selected feature point is suitable for tracking; and said selection processing unit selects a new feature point in place of the feature point which is judged not to be suitable. ### Claim 6 (depends on 1) The image processing apparatus as cited in claim 1, wherein said motion calculation processing unit includes a plurality of motion calculating units different from each other and configured to calculate the motion of the tracking area to synthesize the plurality of motion calculating units according to the occlusion value. ### Claim 7 (depends on 1) The image processing apparatus as cited in claim 1, wherein weighting is performed according to the reliability indicator of each of the feature points at the time of the calculation of the motion by the motion calculation processing unit. ### Claim 8 (depends on 1) The image processing apparatus as cited in claim 1, wherein said motion calculation processing unit calculates the motion of the tracking area using information pertaining to the past motion of the tracking area. ### Claim 9 A moving object tracking method for tracking a moving object by setting a tracking area corresponding to the moving object appearing in an image, setting a plurality of feature points in the tracking area, and tracking the plurality of feature points, comprising steps of: tracking each of the plurality of feature points from the preceding frame to the present frame; specifying the position of the tracking area in the present frame by estimating the motion of the tracking area on the basis of the tracking result of each of the feature points, detecting the occurrence of an occlusion of the moving object overlapping with an other object, and calculating an occlusion value indicating the possibility of the occurrence of the occlusion of the moving object by utilizing a change of a number of feature points being successfully tracked, wherein said moving object tracking method calculates a reliability indicator indicating a probability that a feature point has been tracked in the moving object for each of the feature points; calculates the motion of the tracking area using the reliability indicator of each feature point and the occlusion value; setting a plurality of groups of feature points among the feature points based on the respective reliability indicators; and calculating the occlusion value based on a change of the number of the feature points being successfully tracked among different groups of the plurality of groups of feature points. ### Claim 10 A computer readable medium encoded with a program for making a computer execute a moving object tracking method for tracking a moving object by setting a tracking area corresponding to the moving object appearing in an image, by setting a plurality of feature points in the tracking area, and by tracking the plurality of feature points, said moving object tracking method comprising the steps of: tracking each of the plurality of feature points from the preceding frame to the present frame; specifying the position of the tracking area in the present frame by estimating the motion of the tracking area based the tracking result of each of the feature points; detecting the occurrence of an occlusion of the moving object overlapping with an other object; and calculating an occlusion value indicating the possibility of the occurrence of the occlusion of the moving object by utilizing a change of a number of feature points being successfully tracked, wherein said moving object tracking method calculates a reliability indicator indicating a probability that a feature point has been tracked in the moving object for each of the feature points, calculates the motion of the tracking area using the reliability indicator for each feature point and the occlusion value; setting a plurality of groups of feature points among the feature points based on the respective reliability indicators; and calculating the occlusion value based on a change of the number of the feature points being successfully tracked among different groups of the plurality of groups of feature points. ### Claim 11 A monitoring apparatus comprising: an image processing unit configure to receive a picture signal output from imaging means to track a moving object appearing in an image indicated by the picture signal; and a security processing unit configured to monitor the moving object utilizing a tracking result output by the image processing unit, wherein said image processing unit sets a tracking area in an image corresponding to the moving object, and sets a plurality of feature points in the tracking area, and further tracks the plurality of feature points, and said image processing unit includes a tracking processing unit configured to track each of the plurality of feature points from the preceding frame to the present frame, a motion calculation processing unit configured to estimate the motion of the tracking area on the basis of a tracking result of each of the feature points to specify the position of the tracking area in the present frame, and an occlusion detecting unit configured to detect the occurrence of an occlusion of the moving object overlapping with an other object, said occlusion detecting unit calculates an occlusion value indicating the possibility of the occurrence of the occlusion of the moving object by utilizing a change of a number of feature points being successfully tracked, wherein said tracking processing unit calculates a reliability indicator indicating a probability that a feature point has been tracked in the moving object for each of the feature points, and said motion calculation processing unit calculates the motion of the tracking area using the reliability indicator for each of the feature points and the occlusion value; wherein said occlusion detecting unit sets a plurality of groups of feature points among the feature points based on the respective reliability indicators; and said occlusion detecting unit calculates the occlusion value based on a change of the number of the feature points being successfully tracked among different groups of the plurality of groups of feature points. ### Claim 12 A game apparatus comprising: an image processing unit configured to track a moving object appearing in an image encoded in an input picture signal; a user's operation unit configured to accept an operation of a user; and a game processing unit configured to execute a game program on the basis of a tracking result output by the image processing unit and an operation signal from the user's operation unit, wherein said image processing unit sets a tracking area in an image corresponding to the moving object, and sets a plurality of feature points in the tracking area to track the plurality of feature points, and said image processing unit includes a tracking processing unit configured to track each of the plurality of feature points from the preceding frame to the present frame, a motion calculation processing unit configured to estimate the motion of the tracking area based on the tracking result of each of the feature points to specify the position of the tracking area in the present frame, and an occlusion detecting unit configured to detect the occurrence of an occlusion of the moving object overlapping with an other object, said occlusion detecting unit calculates an occlusion value indicating the possibility of the occurrence of the occlusion of the moving object by utilizing a change of a number of feature points being successfully tracked, wherein said tracking processing unit calculates a reliability indicator indicating a probability that a feature point has been tracked in the moving object for each of the feature points, and said motion calculation processing unit calculates the motion of the tracking area using the reliability indicator for each of the feature points and the occlusion value; wherein said occlusion detecting unit sets a plurality of groups of feature points among the feature points based on the respective reliability indicators; and said occlusion detecting unit calculates the occlusion value based on a change of the number of the feature points being successfully tracked among different groups of the plurality of groups of feature points. ### Claim 13 (depends on 1) The image processing apparatus of claim 1, wherein said occlusion detecting unit detects the occurrence of the occlusion based on the simultaneity of changes of the number of feature points among the plurality of groups. ### Claim 14 (depends on 9) The moving object tracking method of claim 9, wherein said detecting includes detecting the occurrence of the occlusion based on the simultaneity of changes of the number of feature points among the plurality of groups. ### Claim 15 (depends on 10) The computer readable medium of claim 10, wherein said detecting includes detecting the occurrence of the occlusion based on the simultaneity of changes of the number of feature points among the plurality of groups. ### Claim 16 (depends on 11) The monitoring apparatus of claim 11, wherein said occlusion detecting unit detects the occurrence of the occlusion based on the simultaneity of changes of the number of feature points among the plurality of groups. ### Claim 17 (depends on 12) The game apparatus of claim 12, wherein said occlusion detecting unit detects the occurrence of the occlusion based on the simultaneity of changes of the number of feature points among the plurality of groups.