# US 2020/0402281 A1 · Image processing apparatus, image processing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402281 A1
- Document ID: US20200402281A1
- Status: Published application
- Kind: A1
- Application number: 16/901,351
- Filed: 2020-06-15
- Published: 2020-12-24
- Earliest claimed: 2019-06-18
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Kazuna Maruyama
- Web page: https://patentgrep.com/patent/US20200402281A1

## CPC

- G06T 15/20: Perspective computation
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/65: on geographic maps
- H04N 5/268: Signal distribution or switching
- G06T 2207/30228: Playing field
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing apparatus includes: a first identifying unit configured to identify image-capturing conditions concerning a position and an orientation of an image-capturing apparatus which obtains a captured image of an image-capturing target region; a second identifying unit configured to identify viewpoint conditions concerning a position and an orientation of a virtual viewpoint for a virtual viewpoint image generated based on a plurality of images of the image-capturing target region obtained by a plurality of the image-capturing apparatuses at different positions; and a display control unit configured to allow a display apparatus to display information indicating a degree of match between the identified image-capturing conditions and the identified viewpoint conditions before an image presented to a viewer is switched between the captured image and the virtual viewpoint image.

## Claims

### Claim 1

An image processing apparatus comprising:
a first identifying unit configured to identify image-capturing conditions concerning a position and an orientation of an image-capturing apparatus which obtains a captured image of an image-capturing target region;
a second identifying unit configured to identify viewpoint conditions concerning a position and an orientation of a virtual viewpoint for a virtual viewpoint image generated based on a plurality of images of the image-capturing target region obtained by a plurality of image-capturing apparatuses at different positions; and
a display control unit configured to allow a display apparatus to display information indicating a degree of match between the identified image-capturing conditions and the identified viewpoint conditions before an image presented to a viewer is switched between the captured image and the virtual viewpoint image.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match includes information indicating whether or not the identified image-capturing conditions and the identified viewpoint conditions substantially match each other.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match includes information indicating a relationship between the position of the image-capturing apparatus and the position of the virtual viewpoint.

### Claim 4 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match indicates at least one of a direction of the position of the image-capturing apparatus relative to the position of the virtual viewpoint and a distance between the virtual viewpoint and the image-capturing apparatus.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match includes an image which represents the image-capturing target region viewed from a predetermined viewpoint, and on which information indicating the position of the image-capturing and information indicating the position of the virtual viewpoint are superimposed.

### Claim 6 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match is displayed on the display apparatus together with the virtual viewpoint image.

### Claim 7 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match is displayed on the display apparatus together with the captured image.

### Claim 8 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match is displayed on the display apparatus together with the captured image and the virtual viewpoint image.

### Claim 9 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match is displayed on the display apparatus together with information indicating whether control of bringing the image-capturing conditions and the viewpoint conditions close to each other is being executed.

### Claim 10 (depends on 1)

The image processing apparatus according to claim 1, wherein the information indicating the degree of match is displayed on the display apparatus together with information indicating a remaining time until the image presented to the viewer is switched between the captured image and the virtual viewpoint image.

### Claim 11 (depends on 1)

The image processing apparatus according to claim 1, wherein
the identified image-capturing conditions include the position, the orientation, and an angle of view of the image-capturing apparatus, and
the identified viewpoint conditions include the position, the orientation, and an angle of view of the virtual viewpoint.

### Claim 12 (depends on 1)

The image processing apparatus according to claim 1, wherein the image presented to the viewer is outputted to at least one of a broadcasting facility and a webcasting server.

### Claim 13 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
an obtaining unit configured to obtain the captured image and the virtual viewpoint image; and
an output unit configured to perform image output while switching the captured image and the virtual viewpoint image thus obtained.

### Claim 14 (depends on 1)

The image processing apparatus according to claim 1, further comprising a control unit configured to perform control of bringing the image-capturing conditions and the viewpoint conditions close to each other before the image presented to the viewer is switched between the captured image and the virtual viewpoint image.

### Claim 15 (depends on 14)

The image processing apparatus according to claim 14, wherein the control changes the viewpoint conditions depending on the image-capturing conditions.

### Claim 16 (depends on 1)

The image processing apparatus according to claim 1, further comprising a control unit configured to control, based on the image-capturing conditions, the viewpoint conditions after the image presented to the viewer is switched between the captured image and the virtual viewpoint image.

### Claim 17

An image processing method comprising:
identifying image-capturing conditions concerning a position and an orientation of an image-capturing apparatus which obtains a captured image of an image-capturing target region;
identifying viewpoint conditions concerning a position and an orientation of a virtual viewpoint for a virtual viewpoint image generated based on a plurality of images of the image-capturing target region obtained by a plurality of image-capturing apparatuses at different positions; and
allowing a display apparatus to display information indicating a degree of match between the identified image-capturing conditions and the identified viewpoint conditions before an image presented to a viewer is switched between the captured image and the virtual viewpoint image.

### Claim 18 (depends on 17)

The image processing method according to claim 17, wherein the information indicating the degree of match is displayed on the display apparatus together with the captured image and the virtual viewpoint image.

### Claim 19 (depends on 17)

The image processing method according to claim 17, wherein the information indicating the degree of match is displayed on the display apparatus together with information indicating whether control of bringing the image-capturing conditions and the viewpoint conditions close to each other is being executed.

### Claim 20

A non-transitory computer readable storage medium storing a program for causing a computer to perform a method of processing an image, the method comprising:
identifying image-capturing conditions concerning a position and an orientation of an image-capturing apparatus which obtains a captured image of an image-capturing target region;
identifying viewpoint conditions concerning a position and an orientation of a virtual viewpoint for a virtual viewpoint image generated based on a plurality of images of the image-capturing target region obtained by a plurality of image-capturing apparatuses at different positions; and
allowing a display apparatus to display information indicating a degree of match between the identified image-capturing conditions and the identified viewpoint conditions before an image presented to a viewer is switched between the captured image and the virtual viewpoint image.
