# US 12,112,425 B2 · Information processing apparatus, method of operating information processing apparatus, and program for generating virtual viewpoint image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,425 B2 - Document ID: US12112425B2 - Status: Granted - Kind: B2 - Application number: 17/846,026 - Filed: 2022-06-22 - Published: 2022-10-06 - Granted: 2024-10-08 - Earliest claimed: 2019-12-26 - Assignees: FUJIFILM Corporation - Applicants: FUJIFILM Corporation - Inventors: Kazunori Tamura; Takashi Aoki; Fuminori Irie; Masahiko Miyata; Yasunori Murakami - Web page: https://patentgrep.com/patent/US12112425B2 ## CPC - G06T 15/20: Perspective computation - H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras} - G06T 7/0002: {Inspection of images, e.g. flaw detection} - H04N 21/2187: Live feed - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors - G06T 2207/30168: Image quality inspection - G06T 2207/30221: Sports video; Sports image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An information processing apparatus includes a processor and a memory connected to or incorporated in the processor. The processor acquires reference imaging device information corresponding to a position, an imaging direction, and an angle of view of a reference imaging device, and generates the virtual viewpoint image by using the reference imaging device information as a reference, on a condition that an instruction to start generating a virtual viewpoint image based on a plurality of images obtained by imaging an imaging region with a plurality of imaging devices is given. ## Claims ### Claim 1 An information processing apparatus comprising: a processor; and a memory that is connected to or incorporated in the processor, wherein the processor acquires reference imaging device information corresponding to a position, an imaging direction, and an angle of view of a reference imaging device, and on a condition that an instruction to start generating a virtual viewpoint image based on a plurality of images obtained by imaging an imaging region with a plurality of imaging devices is given, generates the virtual viewpoint image by using the reference imaging device information as a reference, wherein a position, an imaging direction, and an angle of view of the virtual viewpoint image match the position, the imaging direction, and the angle of view of the reference imaging device. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein in a case where the reference imaging device information is continuously transmitted by a transmission device, the processor is configured to generate the virtual viewpoint image by using, as a reference, the reference imaging device information transmitted from the transmission device, on a condition that the instruction is given. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to update the reference imaging device information each time the processor acquires the reference imaging device information. ### Claim 4 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to acquire image quality information indicating an image quality of a first reference image obtained by imaging the imaging region with the reference imaging device, and determine an image quality of the virtual viewpoint image on the basis of the image quality information on a condition that the instruction is given. ### Claim 5 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to output a superimposed image in which the virtual viewpoint image is superimposed on a second reference image obtained by imaging the imaging region with the reference imaging device, on a condition that the instruction is given. ### Claim 6 (depends on 5) The information processing apparatus according to claim 5, wherein the processor is configured to gradually change a ratio between the second reference image and the virtual viewpoint image in the superimposed image. ### Claim 7 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to output the virtual viewpoint image to a display, and receive a change signal for continuously changing at least one of a viewpoint position, a line-of-sight direction, or an angle of view in the output virtual viewpoint image. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the reference imaging device is an imaging device capable of changing at least one of the position, the imaging direction, or the angle of view. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to acquire the reference imaging device information on the basis of a third reference image obtained by imaging the imaging region with the reference imaging device. ### Claim 10 (depends on 1) The information processing apparatus according to claim 1, wherein the reference imaging device information is the position, the imaging direction, and the angle of view of the reference imaging device. ### Claim 11 (depends on 1) The information processing apparatus according to claim 1, wherein the reference imaging device is one of the plurality of imaging devices. ### Claim 12 (depends on 11) The information processing apparatus according to claim 11, wherein the reference imaging device is capable of being switched between the plurality of imaging devices. ### Claim 13 (depends on 1) The information processing apparatus according to claim 1, wherein the processor is configured to output a fourth reference image obtained by imaging the imaging region with the reference imaging device, acquire the reference imaging device information, and generate a virtual viewpoint image corresponding to the fourth reference image by using the reference imaging device information as a reference, on a condition that the instruction is given. ### Claim 14 A method of operating an information processing apparatus including a processor and a memory connected to or incorporated in the processor, the method comprising: acquiring reference imaging device information corresponding to a position, an imaging direction, and an angle of view of a reference imaging device; and on a condition that an instruction to start generating a virtual viewpoint image based on a plurality of images obtained by imaging an imaging region with a plurality of imaging devices is given, generating the virtual viewpoint image by using the reference imaging device information as a reference, wherein, a position, an imaging direction, and an angle of view of the virtual viewpoint image match the position, the imaging direction, and the angle of view of the reference imaging device.