Articulated object position and posture estimation device, method and program

An articulated object position and posture estimation device comprising:

• an articulated object frame model storage section configured to store as a physique parameter a size of rigid body of each portion of an articulated object frame model provided by modeling an articulated object based on a multi-articulated object tree structure with rigid bodies;
• a posture model storage section configured to store basis posture vectors and a standard posture vector of a posture model modeled with low-dimensional parameters under movement restraint obtained through principal component analysis on time-series postures of said articulated object frame model, wherein said time-series postures correspond to a predetermined limited movement of a plurality of articulated object;
• a position and posture initial value storage section configured to store a position and posture estimation initial value;
• an image obtaining section configured to obtain an input image of a target articulated object;
• a first articulated object silhouette obtaining section configured to obtain an articulated object silhouette image (pixel vector) from said input image;
• an input-judgment-involved position and posture description conversion section configured to obtain said position and posture estimation initial value from said position and posture initial value storage section and to judge a description type of said position and posture estimation initial value, wherein said input-judgment-involved position and posture description conversion section is configured not to convert said description type in case of a posture model description and to convert said description type into said posture model description by using said posture model in case of an articulated object model description;
• an articulated object model image generation section configured to, based on position and posture (posture model description), generate an articulated object model image adjusted to camera parameters of said image obtaining section from said posture model and said articulated object frame model;
• a second articulated object silhouette obtaining section configured to obtain an articulated object silhouette image (pixel vector) from said articulated object model image;
• a position and posture estimation section configured to calculate an evaluation function by using said two pixel vectors inputted, to repeat an estimation of said position and posture (posture model description) through an optimization method such that a value of said estimation function is minimized and to output final estimation results as an evaluation value and said position and posture (posture model description), wherein said estimation function is expressed by an square of magnitude of a difference between said pixel vectors;
• and
• a position and posture description conversion section configured to convert said position and posture (posture model description) as an estimation result into said articulated object frame model description.

An articulated object position and posture estimation device comprising:

• an articulated object frame model storage section configured to store as a physique parameter a size of rigid body of each portion of an articulated object frame model provided by modeling an articulated object based on a multi-articulated object tree structure with rigid bodies;
• a posture model storage section configured to store basis posture vectors and a standard posture vector of a posture model modeled with low-dimensional parameters under movement restraint obtained through principal component analysis on time-series postures of said articulated object frame model, wherein said time-series postures correspond to predetermined limited movements of a plurality of articulated object;
• a position and posture initial value storage section configured to store a position and posture estimation initial value;
• a first image obtaining section configured to obtain an input image of a target articulated object;
• a second image obtaining section configured to obtain another input image of said target articulated object;
• a first articulate object silhouette obtaining section configured to obtain a first pixel vector from said input image from said first image obtaining section;
• a second articulated object silhouette obtaining section configured to obtain a second pixel vector from said another input image from said second image obtaining section;
• a first pixel vector connecting section configured to connects said first pixel vector and said second pixel vector in a determined order to provide a first connected pixel vector;
• an input-judgment-involved position and posture description conversion section configured to obtain said position and posture estimation initial value from said position and posture initial value storage section and to judge a description type of said position and posture estimation initial value, wherein said input-judgment-involved position and posture description conversion section is configured not to convert said description type in case of a posture model description and to convert said description type into said posture model description by using said posture model in case of an articulated object model description;
• an articulated object model image generation section configured to, based on position and posture (posture model description), generate articulated model images adjusted to camera parameters of said two image obtaining sections from said posture model and said articulated frame model;
• a third articulated object silhouette obtaining section configured to obtain a third pixel vector from one of said articulated model images;
• a forth articulated object silhouette obtaining section configured to obtain a forth pixel vector from another of said articulated model images;
• a second pixel vector connecting section configured to connects said third pixel vector and said forth pixel vector in said predetermined order to provide a second connected pixel vector;
• a position and posture estimation section configured to calculate an evaluation function expressed by an square of magnitude of a difference between said first connected pixel vector and said second connected pixel vector, to repeat an estimation of said position and posture (posture model description) through an optimization method such that a value of said estimation function is minimized and to output final estimation results as an evaluation value and said position and posture (posture model description);
• and
• a position and posture description conversion section configured to convert said position and posture (posture model description) as an estimation result into said articulated object frame model description.

An articulated object position and posture estimation device comprising:

• an articulated object frame model storage section configured to store as a physique parameter a size of rigid body of each portion of an articulated object frame model provided by modeling an articulated object based on a multi-articulated object tree structure with rigid bodies;
• a posture model storage section configured to store basis posture vectors and a standard posture vector of a posture model modeled with low-dimensional parameters under movement restraint obtained through principal component analysis on time-series postures of said articulated object frame model, wherein said time-series postures correspond to predetermined limited movements of a plurality of articulated object;
• a position and posture initial value storage section configured to store a position and posture estimation initial value;
• an image obtaining section configured to obtain an input image of a target articulated object;
• a first articulated object silhouette obtaining section configured to obtain a first articulated object silhouette image (pixel vector) from said input image;
• a first contour smoothing processing section configured to generate a processed image (first pixel vector) by not performing a blurring processing toward inside from a contour of said first articulated object silhouette image but by performing a blurring processing toward outside from said contour such that a position of said contour is not changed;
• an input-judgment-involved position and posture description conversion section configured to obtain said position and posture estimation initial value from said position and posture initial value storage section and to judge a description type of said position and posture estimation initial value, wherein said input-judgment-involved position and posture description conversion section is configured not to convert said description type in case of a posture model description and to convert said description type into said posture model description by using said posture model in case of an articulated object model description;
• an articulated object frame model image generation section configured to, based on position and posture (posture model description), generate an articulated model image adjusted to camera parameters of said image obtaining section from said posture model and said articulated frame model;
• a second articulated object silhouette obtaining section configured to obtain a second articulated object silhouette image (pixel vector) from said articulated object image;
• a second contour smoothing processing section configured to generate a processed image (second pixel vector) by not performing a blurring processing toward inside from a contour of said second articulated object silhouette image but by performing a blurring processing toward outside from said contour such that a position of said contour is not changed;
• a position and posture estimation section configured to calculate an evaluation function expressed by an square of magnitude of a difference between said first pixel vector and said second pixel vector, to repeat an estimation of said position and posture (posture model description) through an optimization method such that a value of said estimation function is minimized and to output final estimation results as an evaluation value and said position and posture (posture model description);
• and
• a position and posture description conversion section configured to convert said position and posture (posture model description) as an estimation result into said articulated object frame model description.

An articulated object position and posture estimation device comprising:

• a storage section configured to store a standard posture vector and a basis posture vector group;
• and
• a position and posture estimation device configured to estimate a posture of an estimation target articulated object by using a posture model expressed by a linear combination of said standard posture vector and said basis posture vector group,
• wherein for each of a plurality of cases different in physique parameters indicating sizes of portions of an articulated object frame model, a plurality of posture vectors are obtained which indicate time-series postures of said articulated object frame model corresponding to a predetermined movement of an articulated object,
• for each of said plurality of cases, a data set is obtained by subtracting an average posture vector as an average of said plurality of posture vectors from each of said plurality of posture vectors,
• said basis posture vector group is obtained as an eigenvector group by performing a principal component analysis on a set of said data set corresponding to a whole of said plurality of cases,
• said standard posture vector is obtained as an average in a set of said plurality of posture vectors corresponding to said whole of said plurality of cases, and
• said position and posture estimation device is configured to obtain an articulated object silhouette image based on said posture model and said articulated object frame model, to obtain an estimation target articulated object silhouette image as a silhouette image of said estimation target articulated object, and to change an coefficient group of said basis posture vector group in said linear combination such that said articulated object silhouette image and said estimation target articulated object silhouette image match with each other.

An articulated object position and posture estimation method comprising:

• obtaining a standard posture vector and a basis posture vector group;
• and
• estimating a posture of an estimation target articulated object by using a posture model expressed by a linear combination of said standard posture vector and said basis posture vector group,
• wherein said obtaining said standard posture vector and said basis posture vector group includes:
• for each of a plurality of cases different in physique parameters indicating sizes of portions of an articulated object frame model, obtaining a plurality of posture vectors indicating time-series postures of said articulated object frame model corresponding to a predetermined movement of an articulated object;
• for each of said plurality of cases, obtaining a data set by subtracting an average posture vector as an average of said plurality of posture vectors from each of said plurality of posture vectors;
• obtaining said basis posture vector group as an eigenvector group by performing a principal component analysis on a set of said data set corresponding to a whole of said plurality of cases;
• and
• obtaining said standard posture vector as an average in a set of said plurality of posture vectors corresponding to said whole of said plurality of cases, and
• said estimating said posture includes:
• obtaining an articulated object silhouette image based on said posture model and said articulated object frame model;
• obtaining an estimation target articulated object silhouette image as a silhouette image of said estimation target articulated object;
• and
• changing an coefficient group of said basis posture vector group in said linear combination such that said articulated object silhouette image and said estimation target articulated object silhouette image match with each other.

A computer program product for a computer to execute an articulated object position and posture estimation method which comprises:

• obtaining a standard posture vector and a basis posture vector group;
• and
• estimating a posture of an estimation target articulated object by using a posture model expressed by a linear combination of said standard posture vector and said basis posture vector group,
• wherein said obtaining said standard posture vector and said basis posture vector group includes:
• for each of a plurality of cases different in physique parameters indicating sizes of portions of an articulated object frame model, obtaining a plurality of posture vectors indicating time-series postures of said articulated object frame model corresponding to a predetermined movement of an articulated object;
• for each of said plurality of cases, obtaining a data set by subtracting an average posture vector as an average of said plurality of posture vectors from each of said plurality of posture vectors;
• obtaining said basis posture vector group as an eigenvector group by performing a principal component analysis on a set of said data set corresponding to a whole of said plurality of cases;
• and
• obtaining said standard posture vector as an average in a set of said plurality of posture vectors corresponding to said whole of said plurality of cases, and
• said estimating said posture includes:
• obtaining an articulated object silhouette image based on said posture model and said articulated object frame model;
• obtaining an estimation target articulated object silhouette image as a silhouette image of said estimation target articulated object;
• and
• changing an coefficient group of said basis posture vector group in said linear combination such that said articulated object silhouette image and said estimation target articulated object silhouette image match with each other.