Information processing apparatus, three-dimensional position calculation method, and program

A method performed by an information processing apparatus, the method comprising:

• extracting a first region having predetermined color information from a first image sensed by an imaging apparatus in a first viewpoint and a second region having the predetermined color information from a second image sensed by an imaging apparatus in a second viewpoint different from the first viewpoint;
• first determining to determine whether a first area of the extracted first region in the first image is larger than a first threshold value, and determine whether a second area of the extracted second region in the second image is larger than a second threshold value;
• obtaining first intersection points where a first line set in the first image intersects first pixels of an outline of the first region, and the first determining determines that the first area of which in the first image is larger than the first threshold value, and obtaining second intersection points where a second line set in the second image intersect second pixels of an outline of the second region, and the first determining determines that the second area of which in the second image is larger than the second threshold value;
• attribute determining to determine first attributes which indicate whether the first pixels at the obtained first intersection points are on a right side or on a left side of the first region in the first image, and determine second attributes which indicate whether the second pixels at the obtained second intersection points are on a right side or on a left side of the second region in the second image;
• second determining to determine whether first attributes of the first intersection points on the first line set in the first image match with second attributes of the second intersection points on the second line set in the second image;
• obtaining corresponding points between the first image and the second image based on a result of the second determining;
• and
• calculating at least a depth value of an object based on the obtained corresponding points.

A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method comprising:

• extracting a first region having predetermined color information from a first image sensed by an imaging apparatus in a first viewpoint and a second region having the predetermined color information from a second image sensed by an imaging apparatus in a second viewpoint different from the first viewpoint;
• first determining whether an area of the extracted first region in the first image and the second image is larger than a first threshold value, and determining whether an area of the extracted second region in the second image is larger than a second threshold value;
• obtaining first intersection points where a first line set in the first image intersects first pixels of outline of the first region, and the first determining determines that the first area of which in the first image is larger than the first threshold value, and obtaining second intersection points where a second line set in the second image intersect second pixels of outline of the second region, and the first determining determines that the second area of which in the second image is larger than the second threshold value;
• attribute determining to determine first attributes which indicate whether the first pixels at the obtained first intersection points are on a right side or on a left side of the first region in the first image, and determine second attributes which indicate whether the second pixels at the obtained second intersection points are on a right side or on a left side of the second region in the second image;
• second determining to determine whether first attributes of the first intersection points on the first line set in the first image match with second attributes of the second intersection points on the second line set in the second image;
• obtaining corresponding points between the first image and the second image based on a result of the second determining;
• and
• calculating at least a depth value of an object based on the obtained corresponding points.

A system comprising:

• a head mount display having an imaging unit which senses a real image and a display unit which displays a composite image;
• and
• an image processing apparatus which generates the composite image to be displayed in the display unit, by composing the real image with a virtual image,
• wherein the image processing apparatus further comprises:
• a region extraction unit configured to extract a first region having a predetermined color information from a first image sensed by an imaging apparatus in a first viewpoint and a second region having the predetermined color information from a second image sensed by an imaging apparatus in a second viewpoint different from the first viewpoint;
• a first determination unit configured to determine whether a first area of the extracted first region in the first image and the second image is larger than a first threshold value and determine whether a second area of the extracted second region in the second image is larger than a second threshold value;
• an obtaining unit configured to obtain first intersection points where a first line set in the first image intersects first pixels of an outline of the first region, and the first determination unit determines that the first area of which in the first image is larger than the first threshold value, and obtain second intersection points where a second line set in the second image intersects second pixels of an outline of the second region, and the first determination unit determines that the second area of which in the second image is larger than the second threshold value;
• an attribute determination unit configured to determine first attributes which indicate whether the first pixels at the obtained first intersection points are on a right side or on a left side of the first region in the first image, and determine second attributes which indicate whether the second pixels at the obtained second intersection points are on a right side or on a left side of the second region in the second image;
• a second determination unit configured to determine whether the first attributes of the first intersection points on the first line set in the first image match with the second attributes of the second intersection points on the second line set in the second image;
• a correspondence processing unit configured to obtain corresponding points between the first image and the second image based on a result of the second determination unit;
• a calculation unit configured to calculate at least a depth value of an object based on the obtained corresponding points;
• and
• a composition unit configured to compose each of the first image and the second image including the object with the virtual image, using a calculated depth value of the object.