Articulation region display apparatus, articulation region detecting apparatus, articulation region belongingness calculating apparatus, articulated object region belongingness calculating apparatus, and articulation region display method

An articulation region display apparatus which (i) detects an articulation region using trajectories which correspond to regions included in a moving object in video and (ii) displays the detected articulation region, said articulation region display apparatus comprising:

• an articulatedness calculating unit configured to:
• calculate a pair-wise distances listed below for a pair of given trajectories among trajectories each of which indicates a motion of a block between two or more pictures included in the video:
• (i) a point-to-point distance that is a direct distance between the pair of trajectories;
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and calculate an articulatedness, for a temporal change in the point-to-point distance between the trajectories and a temporal change in the geodetic distance between the trajectories, such that the articulatedness results in a larger positive value as the temporal change in an Euclidean (point-to-point) distance is larger or the temporal change in the geodetic distance is smaller, the block including one or more pixels each of which constitutes a corresponding one of the pictures;
• an articulation detecting unit configured to detect a region corresponding to a first trajectory, as an articulation region, based on the articulatedness between the trajectories obtained by said articulatedness calculating unit, if the first trajectory satisfies the conditions listed below:
• 1. a region corresponding to the first trajectory and a region corresponding to a second trajectory are on a same rigid body;
• 2. the region corresponding to the first trajectory and a region corresponding to a third trajectory are on a same rigid body;
• 3. the regions, each of which corresponding to the second trajectory and the third trajectory are connected to each other via a same joint;
• and
• a display control unit configured to transform the articulation region detected by said articulation detecting unit into a form that can be visually recognized by a user, and output the transformed articulation region.

An articulation region display apparatus which (i) detects an articulation region using trajectories corresponding, in a one-to-one relationship, to regions included in a moving object in video and (ii) displays the detected articulation region, said articulation region display apparatus comprising:

• an articulatedness calculating unit configured to:
• calculate a pair-wise distances listed below for a pair of given trajectories among trajectories, each of which indicates a motion of a block between two or more pictures included in a video sequence, each of the blocks being included in a picture and having one or more pixels;
• (i) a point-to-point distance that is a direct distance between a pair of trajectories;
• and
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and calculate an articulatedness, using a graph in which the temporal change in the point-to-point distance and the temporal change in the geodetic distance between the pair of trajectories are indicated by a vertical axis and a horizontal axis, such that the articulatedness results in a larger value as a distance from a straight line on which the temporal change in the point-to-point distance is equal to the temporal change in the geodetic distance, to a value of the temporal change in the point-to-point distance and a value of the temporal change in the geodetic distance of the target pair that is a pair of the target trajectories is larger;
• an articulation detecting unit configured to detect, as an articulation region, a region corresponding to a first trajectory, based on the articulatedness between the trajectories obtained by said articulatedness calculating unit, if the first trajectory satisfies the conditions listed below:
• 1. the region corresponding to the first trajectory and a region corresponding to a second trajectory are on a same rigid body;
• 2. the region corresponding to the first trajectory and a region corresponding to a third trajectory are on a same rigid body;
• 3. the regions each of which corresponding to the second trajectory and the third trajectory are connected to each other via a same joint;
• and
• a display control unit configured to transform the articulation region detected by said articulation detecting unit into a form that can be visually recognized by a user, and output each of the pictures.

An articulation region detecting apparatus which detects an articulation region using trajectories corresponding, in a one-to-one relationship, to regions included in a moving object in video, said articulation region detecting apparatus comprising:

• an articulatedness calculating unit configured to:
• calculate a pair-wise distances listed below for a pair of given trajectories among trajectories each of which indicates a motion of a block between two or more pictures included in a video sequence, each of the blocks being included in a picture and having one or more pixels;
• (i) a point-to-point distance that is a direct distance between a pair of trajectories;
• and
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and calculate an articulatedness based on a temporal change in the point-to-point distance between the trajectories and a temporal change in the geodetic distance between the trajectories, and the articulatedness being a degree of how likely the regions corresponding, in a one-to-one relationship, to the trajectories are connected via a same joint;
• and
• an articulation detecting unit configured to detect, as an articulation region, a region corresponding to a first trajectory, based on the articulatedness between the trajectories obtained by said articulatedness calculating unit, if the first trajectory satisfies the conditions listed below:
• 1. the region corresponding to the first trajectory and a region corresponding to a second trajectory are on a same rigid body;
• 2. the region corresponding to the first trajectory and a region corresponding to a third trajectory are on a same rigid body;
• 3. the regions each of which corresponding to the second trajectory and the third trajectory are connected to each other via a same joint.

An articulation region belongingness calculating apparatus which calculates a belongingness to an articulation region for each of trajectories corresponding, in a one-to-one relationship, to regions included in a moving object in video, said articulation region belongingness calculating apparatus comprising:

• a distance calculating unit configured to calculate a pair-wise distances listed below for a pair of given trajectories among trajectories each of which indicates a motion of a block between two or more pictures included in a video sequence, each of the blocks being included in a picture and having one or more pixels;
• (i) a point-to-point distance that is a direct distance between the pair of trajectories;
• and
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and
• an articulation region belongingness calculating unit configured to calculate a jointness as a belongingness of a first trajectory to an articulation region, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance of two given trajectories among the first trajectory, a second trajectory, and a third trajectory, the jointness value indicating a degree at which three relationships listed below are simultaneously satisfied:
• 1. a region corresponding to the first trajectory and a region corresponding to the second trajectory are on a same rigid body;
• 2. the region corresponding to the first trajectory and a region corresponding to the third trajectory are on a same rigid body;
• 3. the regions each of which corresponding to the second trajectory and the region corresponding to the third trajectory are on different rigid bodies connected via a same joint.

An articulated object region belongingness calculating apparatus which calculates, for each of trajectories corresponding, in a one-to-one relationship, to regions included in a moving object in video, a belongingness to an articulated object region including regions connected via the articulation region, said articulated object region belongingness calculating apparatus comprising:

• a distance calculating unit configured to calculate a pair-wise distances listed below for a pair of given trajectories among trajectories each of which indicates a motion of a block between two or more pictures included in a video sequence, each of the blocks being included in a picture and having one or more pixels:
• (i) a point-to-point distance that is a direct distance between the pair of trajectories;
• and
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and
• an articulated object region belongingness calculating unit configured to calculate an articulated object region likelihood as a belongingness of a target trajectory to an articulated object region, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance of two given trajectories among the target trajectory, a first trajectory, a second trajectory, and a third trajectory, the articulated object region likelihood being calculated based on a degree at which three relationships listed below are simultaneously satisfied;
• 1. a region corresponding to the first trajectory and a region corresponding to the second trajectory are on a same rigid body;
• 2. the region corresponding to the first trajectory and a region corresponding to the third trajectory are on a same rigid body;
• 3. the regions each of which corresponding to the second trajectory and the region corresponding to the third trajectory are on different rigid bodies connected via a same joint.

An articulation region display method for (i) detecting an articulation region, using trajectories corresponding, in a one-to-one relationship, to regions included in a moving object in video and (ii) displaying the detected articulation region, said articulation region display method comprising:

• Calculating, a pair-wise distances listed below for a pair of given trajectories among trajectories, each of which indicates a motion of a block between two or more pictures included in a video sequence, each of the blocks being included in a picture and having one or more pixels:
• (i) a point-to-point distance that is a direct distance between the pair of trajectories;
• (ii) a geodetic distance that is a distance of a path that passes through, as a relay point, a trajectory other than the pair of trajectories, from one of the pair of trajectories to reach the other;
• and an articulatedness which is a pair-wise measure to describe how likely a pair of trajectories are on a same articulated object, based on a temporal change in the point-to-point distance between the trajectories and a temporal change in the geodetic distance between the trajectories;
• and
• detecting a region corresponding to a first trajectory as an articulation region, based on the articulatedness between the trajectories obtained in said calculating, if the first trajectory satisfies the conditions listed below:
• 1. a region corresponding to the first trajectory and a region corresponding to a second trajectory are present on a same rigid body;
• 2. a region corresponding to the first trajectory and a region corresponding to a third trajectory are present on a same rigid body;
• 3. the regions each of which corresponding to the second trajectory and the third trajectory are connected to each other via a same joint;
• and
• transforming the articulation region detected in said detecting, into a form that can be visually recognized by a user, and outputting each of the pictures.