# US 2022/0036644 A1 · Image processing apparatus, image processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036644 A1
- Document ID: US20220036644A1
- Status: Published application
- Kind: A1
- Application number: 17/309,645
- Filed: 2019-12-05
- Published: 2022-02-03
- Earliest claimed: 2018-12-19
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: GOH KOBAYASHI; HIROSHI ORYOJI
- Web page: https://patentgrep.com/patent/US20220036644A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 7/593: from stereo images
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G06T 2207/30224: Ball; Puck
- G06T 2207/30241: Trajectory
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present technology relates to an image processing apparatus, an image processing method, and a program that enable a strobe image using a 3D model to be generated. A strobe model in which 3D models of an object at a plurality of times generated from a plurality of viewpoint images captured from a plurality of viewpoints are disposed in a three-dimensional space is generated. When the strobe model is generated, a target object that is a target in which the 3D model is disposed in the strobe model is set according to a degree of object relevance indicating relevance with a key object serving as a reference for disposition of the 3D model in the strobe model.

## Claims

### Claim 1

An image processing apparatus comprising:
a generation unit configured to generate 3D models of an object generated from a plurality of viewpoint images captured from a plurality of viewpoints, from the viewpoint images at a plurality of times, to generate a strobe model in which the 3D models of the object at the plurality of times are disposed in a three-dimensional space, wherein
the generation unit disposes 3D models of an object different from a key object in the strobe model according to a degree of object relevance indicating a relevance between the key object and the different object, the key object serving as a reference for a disposition of the 3D models in the three-dimensional space.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein the generation unit sets a target object that is a target of the disposition of the 3D models in the strobe model according to the degree of object relevance.

### Claim 3 (depends on 2)

The image processing apparatus according to claim 2, wherein the generation unit sets a non-key object other than the key object as the target object according to a positional relationship between the key object and the non-key object.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3, wherein the generation unit sets the non-key object of which a distance to the key object is within a predetermined distance as the target object.

### Claim 5 (depends on 4)

The image processing apparatus according to claim 4, wherein the generation unit sets, as the target objects, the non-key object of which a distance to the key object is within the predetermined distance in one frame, and the non-key object of which a distance to the key object is within the predetermined distance in another frame.

### Claim 6 (depends on 4)

The image processing apparatus according to claim 4, wherein the generation unit sets, as the target objects, the non-key object of which a distance to the key object is within the predetermined distance and the other non-key object of which a distance to the non-key object of which the distance to the key object is within a predetermined distance is within the predetermined distance.

### Claim 7 (depends on 4)

The image processing apparatus according to claim 4, wherein the predetermined distance is set for each key object or each non-key object.

### Claim 8 (depends on 1)

The image processing apparatus according to claim 1, wherein the key object is set according to an operation of a user.

### Claim 9 (depends on 8)

The image processing apparatus according to claim 8, wherein an object designated by the user is set as the key object for a UI (User Interface) in which the object is reflected.

### Claim 10

An image processing method comprising:
generating 3D models of an object generated from a plurality of viewpoint images captured from a plurality of viewpoints, from the viewpoint images at a plurality of times, to generate a strobe model in which the 3D models of the object at the plurality of times are disposed in a three-dimensional space, wherein
3D models of an object different from a key object are disposed in the strobe model according to a degree of object relevance indicating a relevance between the key object and the different object, the key object serving as a reference for a disposition of the 3D models in the three-dimensional space.

### Claim 11

A program for causing a computer to function as a generation unit configured to generate 3D models of an object generated from a plurality of viewpoint images captured from a plurality of viewpoints, from the viewpoint images at a plurality of times, to generate a strobe model in which the 3D models of the object at the plurality of times are disposed in a three-dimensional space, wherein the generation unit disposes 3D models of an object different from a key object in the strobe model according to a degree of object relevance indicating a relevance between the key object and the different object, the key object serving as a reference for a disposition of the 3D models in the three-dimensional space.
