Method, system and apparatus for capture of image data for free viewpoint video

A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to execute method for generating a virtual viewpoint image, the method comprising:

• determining, based on changing of position of a moving object in an image target region which an image capture device captures, a region included in an image which is obtained by the image device devices and which corresponds to the image target region, wherein the image capture device captures the image target region without changing a position and a pose of the image capture device and has a sensor, and information of all pixels of the sensor cannot be obtained at a frame rate of generating the virtual viewpoint image;
• obtaining an image corresponding to the determined region from the image capture device at a frame rate determined based on the frame rate of generating the virtual viewpoint image without obtaining an image which does not correspond to the determined region and which is a part of the image that is obtained by the image capture device and that corresponds to the image target region, wherein the frame rate of obtaining the image corresponding to the determined region from the image capture device is higher than the frame rate of obtaining the image corresponding to the image target region from the image capture device and wherein the frame rate of obtaining the image corresponding to the image target region from the image capture device, is lower than the frame rate of generating the virtual viewpoint image;
• and
• generating the virtual viewpoint based on the image obtained at the frame rate determined based on the frame rate of generating the virtual viewpoint image.

An apparatus comprising:

• one or more processors;
• and
• one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus to perform operations including:
• determining, based on changing of position of a moving object in an image target region which an image capture device captures, a region included in an image which is obtained by the image capture device and which corresponds to the image target region, wherein the image capture device captures the image target region without changing a position and a pose of the image capture device and has a sensor, and information of all pixels of the sensor cannot be obtained at a frame rate of generating the virtual viewpoint image,
• obtaining an image corresponding to the determined region from the image capture device at a frame rate determined based on the frame rate of generating the virtual viewpoint image without obtaining an image which does not correspond to the determined region and which is a part of the image that is obtained by the image capture device and that corresponds to the image target region, wherein the frame rate of obtaining the image corresponding to the determined region from the image capture device is higher than the frame rate of obtaining the image corresponding to the image target region from the image capture device and wherein the frame rate of obtaining the image corresponding to the image target region from the image capture device, is lower than the frame rate of generating the virtual viewpoint image;
• and
• generating the virtual viewpoint based on the image obtained at the frame rate determined based on the frame rate of generating the virtual viewpoint image.