Measurement of depth image considering time delay

A depth image measurement apparatus for continuously acquiring depth information of a scene, comprising:

• image input means for inputting a sequence of images of the scene at a first sequence of viewpoints;
• depth image generation means for generating a first sequence of depth images from the scene images sequentially input at said first sequence of viewpoints by said image input means;
• position/posture estimation means for estimating, based on information relating to displacement of the first viewpoint, a sequence of viewpoint position/posture information of a second sequence of viewpoints viewed from the first sequence of viewpoints;
• and
• warping means for sequentially warping the first sequence of depth images generated by said depth image generation means to a second sequence of depth images at the second sequence of viewpoints on the basis of the viewpoint position/posture information of the second sequence of viewpoints estimated by said position/posture estimation means.

An augmented reality presentation system comprising:

• a depth image measurement apparatus for continuously acquiring depth information of a scene, comprising:
• image input means for inputting a sequence of images of the scene at a first sequence of viewpoints;
• depth image generation means for generating a first sequence of depth images from the scene images sequentially input at said first sequence of viewpoints by said image input means;
• position/posture estimation means for estimating, based on information relating to displacement of the first sequence of viewpoints, a sequence of viewpoint position/posture information of a second sequence of viewpoints viewed from the first sequence of viewpoints;
• and
• warping means for sequentially warping the first sequence of depth images generated by said depth image generation means to a second sequence of depth images at the second sequence of viewpoints on the basis of the viewpoint position/posture information of the second sequence of viewpoints estimated by said position/posture estimation means;
• and
• a head mount display comprising a plurality of video cameras for inputting images in front of an observer, and a display for displaying a three-dimensional grayscale image or color image,
• wherein a three-dimensional grayscale or color image generated according to the second depth image is presented to the observer.

A depth image measurement method for continuously acquiring depth information of a scene, comprising:

• the image input step of inputting a sequence of images of the scene from a first sequence of viewpoints;
• the depth image generation step of generating a first sequence of depth images from the scene images inputted in the image input step;
• the position/posture estimation step of estimating, based on information relating to displacement of the first sequence of viewpoints, a sequence of viewpoint position/posture information for a second sequence of viewpoints viewed from the first sequence of viewpoints;
• and
• the warping step of continuously warping the first sequence of depth images generated in the depth image generation step to a second sequence of depth images at the second sequence of viewpoints on the basis of the viewpoint position/posture information of the second sequence of viewpoints estimated in the position/posture estimation step.

An augmented reality presentation method comprising:

• the image input step of inputting a sequence of images of a scene from a first sequence of viewpoints, using a stereo camera for outputting a stereo image in front of an observer;
• the depth image generation step of generating a first sequence of depth images from the scene images continuously input in the image input step;
• the position/posture estimation step of estimating, based on information relating to displacement of the first sequence of viewpoints, a sequence of viewpoint position/posture information at a second sequence of viewpoints, when viewed from the first sequence of viewpoints;
• the warping step of warping the first sequence of depth images continuously generated in the depth image generation step to a second sequence of depth images at the second sequence of viewpoints on the basis of the sequence of viewpoint position/posture information of the second sequence of viewpoints estimated in the position/posture estimation step;
• the step of discriminating depth ordering of a virtual three-dimensional grayscale image and a real world on the-basis of the second depth image;
• and
• the step of displaying the virtual three-dimensional grayscale image onto a head mount display to merge the grayscale image with the real world.

A storage medium that stores an image processing program, which is implemented on a computer and continuously presents three-dimensional images to an observer, storing:

• an image input program code of inputting a sequence of images from a first sequence of viewpoints;
• a depth image generation program code of generating a first sequence of depth images from the continuously input images;
• a position/posture information estimation program code of estimating, based on information relating to displacement of the first sequence of viewpoints, a sequence of viewpoint position/posture information of a second sequence of viewpoints, when viewed from the first sequence of viewpoints;
• a warping program code of continuously warping the continuously generated first sequence of depth images into second sequence of depth images at the second sequence of viewpoints on the basis of the viewpoint position/posture information;
• and
• a program code of presenting to the observer three-dimensional grayscale images or color images generated according to the second depth images.