Generation apparatus, system and method for generating virtual viewpoint image
AssigneeCanon Kabushiki Kaisha
InventorNaoki Umemura
Life of this patent
Estimating the term…
Abstract
To make it possible to obtain a natural virtual viewpoint image in which a structure or the like existing within an image capturing scene is represented three-dimensionally so as to be the same as a real one while suppressing a network load at the time of transmission of multi-viewpoint image data. The generation device according to the present invention generates a virtual viewpoint image based on three-dimensional shape data corresponding to an object, three-dimensional shape data corresponding to a structure, background data corresponding to a background different at least from the object and the structure, and information indicating a virtual viewpoint.
Claims20
A system comprising:
- one or more memories storing instructions;
- and
- one or more processors configured to execute the instructions to perform operations including:
- generating three-dimensional shape data corresponding to an object captured from a plurality of directions,
- acquiring three-dimensional shape data corresponding to a structure captured from the plurality of directions,
- acquiring background data corresponding to a background different at least from the object and the structure both captured from the plurality of directions,
- acquiring information indicating a specified viewpoint, and
- generating an image based on the generated three-dimensional shape data corresponding to the object, the acquired three-dimensional shape data corresponding to the structure, the acquired background data, and the acquired specified viewpoint information,
- wherein the generated three-dimensional shape data corresponding to the object is generated based on (A) an image including (i) an area of the object, (ii) an area of the structure, and (iii) an area of the background and (B) an image not including the area of the object but including the area of the structure and the area of the background.
A method for a system, the method comprising:
- generating three-dimensional shape data corresponding to an object captured from a plurality of directions;
- acquiring three-dimensional shape data corresponding to a structure captured from the plurality of directions;
- acquiring background data corresponding to a background different at least from the object and the structure both captured from the plurality of directions;
- acquiring information indicating a specified viewpoint;
- and
- generating an image based on the generated three-dimensional shape data corresponding to the object, the acquired three-dimensional shape data corresponding to the structure, the acquired background data, and the acquired specified viewpoint information,
- wherein the generated three-dimensional shape data corresponding to the object is generated based on (A) an image including (i) an area of the object, (ii) an area of the structure, and (iii) an area of the background and (B) an image not including the area of the object but including the area of the structure and the area of the background.
A non-transitory computer readable storage medium storing a program to cause a computer to perform a method for a system, the method comprising:
- generating three-dimensional shape data corresponding to an object captured from a plurality of directions;
- acquiring three-dimensional shape data corresponding to a structure captured from the plurality of directions;
- acquiring background data corresponding to a background different at least from the object and the structure both captured from the plurality of directions;
- acquiring information indicating a specified viewpoint;
- and
- generating an image based on the generated three-dimensional shape data corresponding to the object, the acquired three-dimensional shape data corresponding to the structure, the acquired background data, and the acquired specified viewpoint information,
- wherein the generated three-dimensional shape data corresponding to the object is generated based on (A) an image including (i) an area of the object, (ii) an area of the structure, and (iii) an area of the background and (B) an image not including the area of the object but including the area of the structure and the area of the background.
Dependencies are inferred from the claim text.