# US 10,771,761 B2 · Information processing apparatus, information processing method and storing unit > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,761 B2 - Document ID: US10771761B2 - Status: Granted - Kind: B2 - Application number: 16/214,685 - Filed: 2018-12-10 - Published: 2019-06-13 - Granted: 2020-09-08 - Earliest claimed: 2017-12-12 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Shogo Mizuno - Web page: https://patentgrep.com/patent/US10771761B2 ## CPC - H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking - G06T 1/0007: {Image acquisition} - G06T 15/20: Perspective computation - G06T 15/205: {Image-based rendering} - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - H04N 23/63: by using electronic viewfinders - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An information processing apparatus includes one or more memories storing instructions, and one or more processors executing the instructions to determine, among a plurality of captured images obtained by a plurality of image capturing apparatuses, a display target image related to a virtual viewpoint image, based on a position of a virtual viewpoint and a view direction from the virtual viewpoint, the virtual viewpoint image being generated based on the display target image and the position of the virtual viewpoint and the view direction from the virtual viewpoint, and to cause a displaying unit to display the determined display target image in a displaying mode according to a degree of contribution of the determined display target image to generation of the virtual viewpoint image. ## Claims ### Claim 1 An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions: to determine, among a plurality of captured images obtained by a plurality of cameras, a display target image related to a virtual viewpoint image, based on a position of a virtual viewpoint and a view direction from the virtual viewpoint, the virtual viewpoint image being generated based on the display target image and the position of the virtual viewpoint and the view direction from the virtual viewpoint; and to cause a displaying unit to display the determined display target image in a displaying mode according to a degree of contribution of the determined display target image to generation of the virtual viewpoint image. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein an image contributing to determination of a three-dimensional shape of an object included in the virtual viewpoint image is determined to be the display target image, the virtual viewpoint image being generated based on the determination of the three-dimensional shape of the object. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein an image contributing to determination of a color of an object included in the virtual viewpoint image is determined to be the display target image, the virtual viewpoint image being generated based on the determination of the color of the object. ### Claim 4 (depends on 1) The information processing apparatus according to claim 1, wherein the display target image is displayed in a displaying mode according to a positional relationship between the virtual viewpoint and a camera obtaining the determined display target image. ### Claim 5 (depends on 4) The information processing apparatus according to claim 4, wherein the display target image is displayed in the displaying mode based on which direction a camera obtaining the determined display target image is located with respect to the virtual viewpoint. ### Claim 6 (depends on 4) The information processing apparatus according to claim 4, wherein the display target image is displayed in the displaying mode according to how far a camera obtaining the determined display target image is located from the virtual viewpoint. ### Claim 7 (depends on 6) The information processing apparatus according to claim 6, wherein, in a case when, among cameras respectively obtaining a plurality of determined display target images, a first camera is located farther from the virtual viewpoint than a second camera, a captured image obtained by the first camera being smaller than a captured image obtained by the second camera. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the virtual viewpoint image, the display target image, and the attribute information of a camera obtaining the display target image are displayed. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, wherein at least two captured images are determined, as the display target images, from among the plurality of captured images obtained by the plurality of cameras, based on a distance between cameras among the plurality of cameras. ### Claim 10 (depends on 9) The information processing apparatus according to claim 9, wherein captured images respectively obtained by two cameras having a greatest distance among the plurality of cameras. ### Claim 11 (depends on 9) The information processing apparatus according to claim 9, wherein a virtual viewpoint image being based on an intermediate position between cameras respectively obtaining the at least the two captured images is further displayed. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions: to accept a user operation for designating one or more images among a plurality of determined display target images; and to control operation, not to use, in the generation of the virtual viewpoint image, the image designated by the accepted user operation. ### Claim 13 An information processing method that is performed by an information processing apparatus, the information processing method comprising: determining, among a plurality of captured images obtained by a plurality of cameras, a display target image related to a virtual viewpoint image, based on a position of a virtual viewpoint and a view direction from the virtual viewpoint, the virtual viewpoint image being generated based on the display target image and the position of the virtual viewpoint and the view direction from the virtual viewpoint; and causing a displaying unit to display the determined display target image in a displaying mode according to a degree of contribution of the determined display target image to generation of the virtual viewpoint image. ### Claim 14 A non-transitory computer-readable storing unit that stores a program for causing a computer to function as each of following units of an information processing apparatus: a determining unit configured to determine, among a plurality of captured images obtained by a plurality of cameras, a display target image related to a virtual viewpoint image, based on a position of a virtual viewpoint and a view direction from the virtual viewpoint, the virtual viewpoint image being generated based on the display target image and the position of the virtual viewpoint and the view direction from the virtual viewpoint; and a display controlling unit configured to cause a displaying unit to display the determined display target image in a displaying mode according to a degree of contribution of the determined display target image to generation of the virtual viewpoint image.