Image processing method and apparatus

An image processing method comprising the steps of:

• inputting images of an object sensed at a plurality of viewpoint positions together with information indicating the plurality of viewpoint positions;
• detecting a viewpoint position of an observer;
• selecting an image from the images input in the image input step on the basis of the detected viewpoint position, a position of said object and a virtual plane corresponding to the viewpoint position;
• generating an image on a virtual plane defined at the detected viewpoint position by copying pixel data of the image selected by said selecting step to pixel data of the image on the virtual plane in accordance with the relation of the viewpoint position and a photographing position of the selected image;
• and
• displaying the image generated in the image generation step.

An image processing apparatus comprising:

• image sensing means for sensing an image of an object;
• memory means for storing a plurality of images at a plurality of image sensing positions sensed by said image sensing means in correspondence with the image sensing position information;
• and
• image generation means for generating an observation image upon observing the object at an arbitrary viewpoint position on the basis of the plurality of images, the corresponding image sensing position information stored in said memory means and a virtual plane corresponding to the arbitrary viewpoint position,
• wherein said image generation means selects, from the plurality of images stored in said memory means, an image which corresponds to a point at which a line, passing through an arbitrary point on the object and the arbitrary viewpoint position crosses the observation image, and generates the observation image defined at the arbitrary viewpoint position by copying on the virtual plane pixels of the selected image to pixels of the observation image.

An image processing apparatus comprising:

• multi-viewpoint image input means for inputting a plurality of images of an object having different viewpoint positions as multi-viewpoint image data;
• image conversion means for analyzing the input multi-viewpoint image data into a plurality of rays of light corresponding to pixels of the image data;
• recording means for recording pixel data corresponding to each of the rays directed on image planes defined to be around an object;
• viewpoint input means for inputting a viewpoint position and a direction of line of sight of a user;
• image generation means for selecting the image plane on the basis of a relation between an output of said viewpoint input means and a position of the object, copying the pixel data of the selected image plane, calculating a position to a pixel data on a required scan line and pixel position on the scan line in an arbitrary plane defined at the viewpoint position and generating a desired image on the arbitrary plane on the basis of the pixel data and the pixel positions.

An image processing method comprising the steps of:

• inputting a plurality of images having different viewpoint positions as multi-viewpoint image data;
• analyzing the input multi-viewpoint image data into a plurality of rays of light corresponding to pixels of the image data;
• recording pixel data corresponding to each of the rays directed on each of image planes defined to be around an object, the image plane having an angle the plane makes with the ray of light;
• inputting a viewpoint position and a direction of line of sight of a user;
• selecting the image plane on the basis of a relation between an output of said viewpoint input step and a position of an object, calculating a position of a required scan line and a pixel position on the scan line in an arbitrary plane corresponding to the direction of the line of sight, and generating a desired image on the arbitrary plane on the basis of the pixel data and the pixel position;
• copying the pixel data on the selected image plane to pixels corresponding to the desired image;
• and
• displaying the generated image on a display screen.