Image generation apparatus, image display apparatus, image generation method and non-transitory computer readable medium

An image generation apparatus which uses a plurality of two-dimensional images to generate a virtual-viewpoint image to be captured with a virtual camera arranged at a virtual location in a three-dimensional space, the image generation apparatus comprising:

• a disparity-image-generating section which generates a disparity image representing a disparity value of each pixel of one of the plurality of two-dimensional image, by using the plurality of two-dimensional images;
• a disparity-image-correcting section which performs correcting the disparity value of each of pixels of the disparity image generated by the disparity-image-generating section, based on a contour line extracted from the one of the plurality of two-dimensional images and a contour line extracted from the disparity image;
• a pixel-shift-processing section which shifts each pixel in the one of the plurality of two-dimensional images according to the disparity image which has been corrected, to generate an intermediate image;
• and
• a unfixed-pixel-area-interpolating section which interpolates unfixed pixels to which shifted pixels of the one of the two-dimensional image have not been assigned in the intermediate image, to generate the virtual-viewpoint image.

An image generation apparatus which uses a two-dimensional image, a depth image representing depth information of the two-dimensional image, and a parameter of a virtual camera arranged at a virtual location in a three-dimensional space, to generate a virtual-viewpoint image to be captured with the virtual camera at the virtual location, the image generation apparatus comprising:

• a disparity-image-generating section which generates a disparity image representing a disparity value of each pixel of the two-dimensional image, by using the depth image and the parameter of the virtual camera;
• a pixel-shift-processing section which shifts each pixel of the two-dimensional image according to the disparity image, to generate an intermediate image;
• a noise-in-unfixed-pixel-area-eliminating section which eliminates noise in an area of unfixed pixels to which shifted pixels of the two-dimensional image have not been assigned in the intermediate image;
• and
• a unfixed-pixel-area-interpolating section which interpolates the unfixed pixels in the intermediate image wherein the noise has been eliminated, to generate the virtual-viewpoint image.

An image generation apparatus which uses a plurality of two-dimensional images to generate a virtual-viewpoint image to be captured with a virtual camera arranged at a virtual location in a three-dimensional space, the image generation apparatus comprising:

• a disparity-image-generating section which generates a disparity image representing a disparity value of each pixel of one of the plurality of two-dimensional image, by using the plurality of two-dimensional images;
• a pixel-shift-processing section which shifts each pixel in the one of the plurality of two-dimensional images according to the disparity image, to generate an intermediate image;
• a noise-in-unfixed-pixel-area-eliminating section which eliminates noise in an area of unfixed pixels to which shifted pixels of the one of the two-dimensional images have not been assigned in the intermediate image;
• and
• a unfixed-pixel-area-interpolating section which interpolates the unfixed pixels in the intermediate image wherein the noise has been eliminated, to generate the virtual-viewpoint image.

An image generation method which uses a two-dimensional image, a depth image representing depth information of the two-dimensional image, and a parameter of a virtual camera arranged at a virtual location in a three-dimensional space, to generate a virtual-viewpoint image to be captured with the virtual camera at the virtual location, the image generation method comprising:

• a disparity-image-generating step of generating a disparity image representing a disparity value of each pixel of the two-dimensional image, by using the depth image and the parameter of the virtual camera;
• a disparity-image-correcting step of performing one of
• correcting the disparity value of each of pixels of the disparity image generated in the disparity-image-generating step, based on a contour line extracted from the two-dimensional image and a contour line extracted from the disparity image, and
• correcting the depth information of each of pixels of the depth image to be used for generating the disparity image in the disparity-image-generating step, based on a contour line extracted from the two-dimensional image and a contour line extracted from the depth image;
• an pixel-shift-processing step of shifting each pixel of the two-dimensional image according to the disparity image which has been corrected, to generate an intermediate image;
• and
• a unfixed-pixel-interpolating step of interpolating unfixed pixels to which shifted pixels of the two-dimensional image have not been assigned in the intermediate image, to generate the virtual-viewpoint image.

A non-transitory computer readable medium storing an image generation program for use in an apparatus which uses a two-dimensional image, a depth image representing depth information of the two-dimensional image, and a parameter of a virtual camera arranged at a virtual location in a three-dimensional space, to generate a virtual-viewpoint image to be obtained when an object is captured with the virtual camera at the virtual location, the image generation program causing the apparatus to execute processing comprising:

• a disparity-image-generating step of generating a disparity image representing a disparity value of each pixel of the two-dimensional image, by using the depth image and the parameter of the virtual camera;
• a disparity-image-correcting step of performing one of
• correcting the disparity value of each of pixels of the disparity image generated in the disparity-image-generating step, based on a contour line extracted from the two-dimensional image and a contour line extracted from the disparity image, and
• correcting the depth information of each of pixels of the depth image to be used for generating the disparity image in the disparity-image-generating step, based on a contour line extracted from the two-dimensional image and a contour line extracted from the depth image;
• an pixel-shift-processing step of shifting each pixel of the two-dimensional image according to the disparity image which has been corrected, to generate an intermediate image;
• and
• a unfixed-pixel-interpolating step of interpolating unfixed pixels to which shifted pixels of the two-dimensional image have not been assigned in the intermediate image, to generate the virtual-viewpoint image.