# US 2025/0061651 A1 · Information processing device, information processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061651 A1
- Document ID: US20250061651A1
- Status: Published application
- Kind: A1
- Application number: 18/715,800
- Filed: 2022-11-25
- Published: 2025-02-20
- Earliest claimed: 2021-12-09
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: YOSHIYUKI TAKAO
- Web page: https://patentgrep.com/patent/US20250061651A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 7/564: from contours
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present technology relates to an information processing device, an information processing method, and a program that makes it possible to appropriately generate a 3D model of a modeling target on the basis of captured images obtained by capturing the modeling target from multiple viewpoints. 3D shape data representing a 3D shape of a modeling target, which is a target for generating a 3D model, is generated on the basis of a plurality of captured images obtained by capturing the modeling target from different viewpoint positions, and 3D shape data of a missing section of the modeling target is interpolated, the missing section being missing in the 3D shape data.

## Claims

### Claim 1

An information processing device comprising:
a 3D shape data generation unit configured to generate 3D shape data representing a 3D shape of a modeling target, which is a target for generating a 3D model, on a basis of a plurality of captured images obtained by capturing the modeling target from different viewpoint positions; and
an interpolation unit configured to interpolate 3D shape data of a missing section of the modeling target, the missing section being missing in 3D shape data generated by the 3D shape data generation unit.

### Claim 2 (depends on 1)

The information processing device according to claim 1, wherein the 3D shape data generation unit generates the 3D shape data on a basis of an image region of the modeling target in each of the plurality of captured images.

### Claim 3 (depends on 2)

The information processing device according to claim 2, wherein the 3D shape data generation unit generates the 3D shape data using a visual-volume intersection method.

### Claim 4 (depends on 2)

The information processing device according to claim 2, further comprising
an image recognition unit configured to recognize the modeling target in each of the plurality of captured images and detect the image region of the modeling target that has been recognized.

### Claim 5 (depends on 4)

The information processing device according to claim 4, wherein the image recognition unit recognizes the modeling target by an inference model having a structure of a neural network generated by machine learning.

### Claim 6 (depends on 1)

The information processing device according to claim 1, wherein the interpolation unit specifies a three-dimensional region in which the missing section should originally exist on a basis of a three-dimensional position of a predetermined portion of the modeling target estimated by a position of a predetermined portion of the modeling target in each of the plurality of captured images, and adds 3D shape data of the missing section to the three-dimensional region that has been specified.

### Claim 7 (depends on 6)

The information processing device according to claim 6, further comprising
a position estimation unit configured to estimate the three-dimensional position of the predetermined portion on a basis of a position of the predetermined portion of the modeling target in each of the plurality of captured images.

### Claim 8 (depends on 7)

The information processing device according to claim 7, wherein the position estimation unit calculates a plurality of three-dimensional positions as the three-dimensional position of the predetermined portion on a basis of a position of the predetermined portion of the modeling target in each of the plurality of captured images, and estimates the three-dimensional position of the predetermined portion on a basis of the plurality of three-dimensional positions that have been calculated.

### Claim 9 (depends on 8)

The information processing device according to claim 8, wherein the position estimation unit estimates a barycentric position of the plurality of the three-dimensional positions that have been calculated, as the three-dimensional position of the predetermined portion.

### Claim 10 (depends on 6)

The information processing device according to claim 6, wherein the predetermined portion of the modeling target is a portion included in the missing section.

### Claim 11 (depends on 1)

The information processing device according to claim 1, further comprising
a texture generation unit configured to generate a texture to be pasted to the 3D shape data on a basis of an image of the modeling target in each of the plurality of captured images.

### Claim 12 (depends on 1)

The information processing device according to claim 1, wherein any one or more of the plurality of captured images are captured images captured by a rolling shutter type.

### Claim 13 (depends on 1)

The information processing device according to claim 1, wherein the modeling target includes a plurality of types of objects.

### Claim 14 (depends on 1)

The information processing device according to claim 1, wherein the modeling target includes a person.

### Claim 15 (depends on 1)

The information processing device according to claim 1, wherein the modeling target includes a tool used by a person.

### Claim 16

An information processing method performed by an information processing device including
a 3D shape data generation unit and an interpolation unit,
the method comprising:
generating, by the 3D shape data generation unit, 3D shape data representing a 3D shape of a modeling target, which is a target for generating a 3D model, on a basis of a plurality of captured images obtained by capturing the modeling target from different viewpoint positions; and
interpolating, by the interpolation unit, 3D shape data of a missing section of the modeling target, the missing section being missing in the 3D shape data.

### Claim 17

A program causing a computer to function as:
a 3D shape data generation unit configured to generate 3D shape data representing a 3D shape of a modeling target, which is a target for generating a 3D model, on a basis of a plurality of captured images obtained by capturing the modeling target from different viewpoint positions; and
an interpolation unit configured to interpolate 3D shape data of a missing section of the modeling target, the missing section being missing in 3D shape data generated by the 3D shape data generation unit.
