Image production system and image production method to generate an output image based on a free viewpoint image

An image production method, comprising:

• generating, by at least one processor, estimation information for a subject based on at least one of a captured first image or sensor information;
• obtaining, by the at least one processor based on a simultaneous capture of a plurality of second images from a plurality of viewpoints, a plurality of pieces of captured image data;
• generating, by the at least one processor, a first three-dimensional model of the subject based on the plurality of pieces of captured image data;
• generating, by the at least one processor, a free viewpoint image based on the first three-dimensional model, wherein the free viewpoint image is an image of an arbitrary viewpoint for the subject;
• generating, by the at least one processor, a three-dimensional image based on the estimation information and the first three-dimensional model of the subject;
• and
• generating, by the at least one processor, an output image by selectively using the three-dimensional image and a live-action image including the free viewpoint image.

A non-transitory computer-readable medium having stored thereon, computer executable instructions, which when executed by at least one processor, cause the at least one processor to execute operations, the operations comprising:

• generating estimation information for a subject based on at least one of a captured first image or sensor information;
• obtaining, based on a simultaneous capture of a plurality of second images from a plurality of viewpoints, a plurality of pieces of captured image data;
• generating a first three-dimensional model of the subject based on the plurality of pieces of captured image data;
• generating a free viewpoint image based on the first three-dimensional model, wherein the free viewpoint image is an image of an arbitrary viewpoint for the subject;
• generating a three-dimensional image based on the estimation information and the first three-dimensional model of the subject;
• and
• generating an output image by selectively using the three-dimensional image and a live-action image including the free viewpoint image.

An image production system, comprising:

• at least one processor configured to:
• generate estimation information regarding a subject based on at least one of a captured first image or sensor information;
• obtain, based on a simultaneous capture of a plurality of second images from a plurality of viewpoints, a plurality of pieces of captured image data;
• generate a three-dimensional model of the subject based on the plurality of pieces of captured image data;
• generate a free viewpoint image based on the three-dimensional model, wherein the free viewpoint image is an image of an arbitrary viewpoint of the subject;
• generate a three-dimensional image based on the estimation information and the three-dimensional model of the subject;
• and
• generate an output image, for each time period, by selectively using the three-dimensional image and a live-action image including the free viewpoint image.