Display apparatus, image processing apparatus and image processing method, imaging apparatus, and program

An image processing apparatus for processing images captured from a plurality of positions , the image processing apparatus comprising:

• a viewpoint detecting unit configured to detect a plurality of viewpoints of a plurality of users;
• a viewpoint processing unit configured to determine whether the plurality of viewpoints are within a predetermined distance of each other;
• an image selecting mechanism configured to select a viewpoint image of a predetermined object, as viewed from the viewpoints of the plurality of users, based on the plurality of detected viewpoints of the plurality of users, from the images captured respectively from the plurality of positions;
• an integrating mechanism configured to integrate the plurality of detected viewpoints of the plurality of users into a position of a barycenter of the plurality of detected viewpoints of the plurality of users, and configured to integrate the plurality of detected viewpoints of the plurality of users into a single viewpoint, based on the determination by the viewpoint processing unit;
• and
• a viewpoint-image converting mechanism configured to Generate a projected image based on the selected viewpoint images, the plurality of detected viewpoints of the plurality Of users, the determination by the viewpoint processing unit, the integration by the integrating mechanism, and a predetermined three dimensional shape of an interior display surface, the projected image being projected onto the interior display surface by a projector and viewed on an outer surface of the interior display surface.

A non-transitory computer-readable storage medium having embedded therein instructions, which when executed by a computer, cause the computer to perform a method of image processing for processing images captured from a plurality of positions, the method comprising:

• detecting a plurality of viewpoints of a plurality of users;
• determining whether the plurality of viewpoints are within a predetermined distance of each other;
• selecting viewpoint images of a predetermined object as viewed from the plurality of viewpoints of the plurality of users, based on the plurality of detected viewpoints of the plurality of users, from the images captured respectively from the plurality of positions;
• integrating the plurality of detected viewpoints of the plurality of users into a position of a barycenter of the plurality of detected viewpoints of the plurality of users, the integrating step further integrating the plurality of detected viewpoints of the plurality of users into a single viewpoint, based on the determination in the determining step;
• and
• generating a projected image based on the selected viewpoint images, the plurality of detected viewpoints of the plurality of users, the determination in the determining step, the integration in the integrating step, and a predetermined three dimensional shape of an interior display surface, the projected image being projected onto the interior display surface by a projector and viewed on an outer surface of the interior display surface.

An imaging apparatus for capturing an image, comprising:

• an omnidirectional camera configured to obtain omnidirectional-image data including an image of surroundings of an object in all directions;
• an object-imaging unit, including a plurality of cameras surrounding the object in horizontal and vertical directions, configured to obtain surrounding-image data including an image of the object from the plurality of positions corresponding to respective different polar and azimuth angles of a spherical surface in a spherical coordinate system;
• and
• a multiplexer configured to receive the omnidirectional-image data and the surrounding-image data, and to multiplex the surrounding-image data and the omnidirectional-image data in association with each other, wherein
• the image of the surrounding obtained by the omnidirectional imaging unit is displayed as an immersion image for presenting a plurality of users with a sense of immersion, and
• the image of the object obtained by the object-imaging unit is displayed on an interior display surface as at least one bird's-eye image of the object as viewed from a plurality of detected viewpoints of a plurality of users based on a determination of whether the plurality of viewpoints are within a predetermined distance of each other, an integration of the plurality of detected viewpoints of the plurality of users into a position of a barycenter of the plurality of detected viewpoints of the plurality of users, and an integration of the plurality of detected viewpoints of the plurality of users into a single viewpoint, based on the determination of whether the plurality of viewpoints are within the predetermined distance of each other.

An image processing apparatus for processing images captured from a plurality of positions, the image processing apparatus comprising:

• a viewpoint detecting unit configured to detect a plurality of viewpoints of a plurality of users;
• a viewpoint processing unit configured to determine whether the plurality of viewpoints are within a predetermined distance of each other;
• an integrating mechanism configured to integrate the plurality of detected viewpoints of the plurality of users into a single viewpoint, based on the determination by the viewpoint processing unit;
• an image selecting mechanism configured to select viewpoint images of a predetermined object, as viewed from the plurality of viewpoints of the plurality of users, based on the plurality of detected viewpoints of the plurality of users, from the images captured respectively from the plurality of positions, and to calculate a weighted sum of the viewpoint images respectively associated with the plurality of detected viewpoints of the plurality of users in accordance with distances of the respective detected viewpoints of the plurality of users and the single viewpoints to generate a viewpoint image for the single viewpoint;
• and
• a viewpoint-image converting mechanism configured to generate a projected image based on the selected viewpoint images, the plurality of detected viewpoints of the plurality of users, the determination by the viewpoint processing unit, the viewpoint image for the single viewpoint, and a predetermined three dimensional shape of an interior display surface, the projected image being projected onto the interior display surface by a projector and viewed on an outer surface of the interior display surface.