Image production processing apparatus and structural data generating apparatus for generating structural data used in the image production processing apparatus

A recording medium which stores a program instructing a computer system to execute a process in an image production processing apparatus for producing a body image from a visual point based on body images obtained by taking pictures of a body from a plurality of visual points and positions of feature points, corresponding to each other, set on the body images from the plurality of visual points, said program comprising:

• feature point determining means for determining the feature points by calculating a difference, for each visual point, between a projected three-dimensional feature point candidate and a closest feature point candidate, and incrementing a score when the difference is less than a predetermined threshold, and selecting the feature point when the total score is greater than another predetermined threshold;
• visual point selecting means for selecting a predetermined number of visual points from among the visual points stored in said input visual point storage means when a visual point is specified, the selected visual points being closest to the specified visual point;
• image producing means for producing a body image from the specified visual point based on the body images from the selected visual points, based on relationships between the feature points on the body images from the visual points selected by said visual point selecting means and relationships between the selected visual points and the specified visual point;
• coordinate transformation deciding means for deciding coordinate transformation equations based on the relationships between the feature points on the body images from the selected visual points, the coordinate transformation equations transforming positions on the body images from the selected visual points into positions on the body image from the specified visual point;
• image conversion executing means for converting the body images from the selected visual points into body images from the specified visual point using the coordinate transformation equations decided by said coordinate transformation deciding means;
• and
• image synthesizing means for synthesizing, based on the relationships between the selected visual points and the specified visual point, the body images into which the body images from the selected visual points are converted by said image conversion executing means, a synthesized body image being output as the body image from the specified visual point,
• wherein said image synthesizing means has means for averaging data of pixels, corresponding to each other, of the body images from the selected visual points with weights based on the relationships between the selected visual points and the specified visual points so that data of a corresponding pixel of the body image from the specified visual point is calculated.

A structural data generating apparatus comprising:

• input visual point image storage means for storing body images obtained by taking pictures of a body from a plurality of visual points so that each of the body images corresponds to one of said plurality of visual points;
• feature point candidate calculating means for extracting candidates of feature points from the body images from the respective visual points stored in said input visual point image storage means;
• input visual point setting means for setting positions of the visual points from which pictures of the body are taken to obtain the body images stored in said input visual point image storage means;
• input visual point storage means for storing the position of the visual points set by said input visual point setting means;
• three-dimensional coordinate candidate extracting means for extracting three-dimensional coordinates of points on the body corresponding to the candidates of the feature points, based on the positions of the candidates of the feature points on the body images from the respective visual points calculated by said feature point candidate calculating means and based on the positions of the visual points stored in said input visual point storage means;
• three-dimensional coordinate calculating means for calculating three-dimensional coordinates of positions on the body which should be the feature points, said three-dimensional coordinate calculating means projecting the extracted candidates of the three-dimensional coordinates on the body images from the respective visual points so as to generate projection points on the body images from the respective visual points, and determining the extracted candidates as the true three-dimensional coordinates when a score is incremented each time it is detected for one of the visual points that a difference between each projection point and a feature point candidate closest to the projection point is less than a predetermined threshold, and the resulting score for all of the visual points is larger than a predetermined threshold, so that the three-dimensional coordinates are calculated;
• three-dimensional coordinate storage means for storing the three-dimensional coordinates calculated by said three-dimensional coordinate calculating means;
• and
• input feature point deciding means for projecting the points represented by the calculated three-dimensional coordinates toward each of the visual points so that the feature points are calculated.

A recording medium which stores a program instructing a computer system to execute a process for generating structural data used in an image production processing apparatus for producing images of a body from a visual point, said program comprising:

• feature point candidate calculating means for extracting candidates of feature points from body images obtained by taking pictures of a body from a plurality of visual points, the candidates of the feature points satisfying a predetermined condition;
• input visual point setting means for setting positions of the visual points from which pictures of the body are taken to obtain the body images, the positions of the visual points being stored in an input visual point storage unit;
• three-dimensional coordinate candidate extracting means for extracting three-dimensional coordinates of points on the body corresponding to the candidates of the feature points, based on the positions of the candidates of the feature points on the body images from the respective visual points calculated by said feature point candidate calculating means and based on the positions of the visual points stored in said input visual point storage unit;
• three-dimensional coordinate calculating means for calculating three-dimensional coordinates of positions on the body which should be the feature points, said three-dimensional coordinate calculating means projecting the extracted candidates of the three-dimensional coordinates on the body images from the respective visual points so as to generate projection points on the body images from the respective visual points, and determining the extracted candidates as the true three-dimensional coordinates when a score is incremented each time it is detected for one of the visual points that a difference between each projection point and a feature point candidate closest to the projection point is less than a predetermined threshold, and the resulting score for all of the visual points is larger than a predetermined threshold, so that the three-dimensional coordinates are calculated;
• and
• input feature point deciding means for projecting the points represented by the calculated three-dimensional coordinates toward each of the visual points so that the feature points are calculated.

A method for image production processing comprising:

• storing positions of feature points, corresponding to each other, on body images of a body obtained from a plurality of visual points so that each of the body images corresponds to one of the plurality of visual points, the feature points determined by calculating a difference, for each visual point, between a projected three-dimensional feature point candidate and a closest feature point candidate, and incrementing a score when the difference is less than a predetermined threshold, and selecting the feature point when the total score is greater than another predetermined threshold;
• selecting a predetermined number of visual points from the plurality of visual points when a visual point is specified, the selected visual points being closest to the specified visual point;
• and
• synthesizing the body image for the specified visual point using the feature points for the selected visual points to determine coordinate transformation equations to transform positions on the body images for the selected visual points into positions on a body image of the specified visual point, and including averaging data of pixels, corresponding to each other, of the body images for the selected visual points with weights based on the relationships between the selected visual points and the specified visual point so that data of a corresponding pixel of the body image for the specified visual point are calculated.