# US 2022/0284672 A1 · Three-dimensional model generation method and three-dimensional model generation device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284672 A1
- Document ID: US20220284672A1
- Status: Published application
- Kind: A1
- Application number: 17/748,331
- Filed: 2022-05-19
- Published: 2022-09-08
- Earliest claimed: 2019-11-20
- Assignees: Panasonic Intellectual Property Management Co., Ltd.
- Applicants: Panasonic Intellectual Property Management Co., Ltd.
- Inventors: Kensho TERANISHI; Satoshi YOSHIKAWA; Toru MATSUNOBU; Masaki FUKUDA
- Web page: https://patentgrep.com/patent/US20220284672A1

## CPC

- G06T 7/593: from stereo images
- G06T 17/05: Geographic models
- G06T 7/55: from multiple images
- G06T 7/579: from motion
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 2200/04: involving 3D image data
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/10012: Stereo images
- G06T 2207/10016: Video; Image sequence
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A three-dimensional model generation method includes: obtaining map information generated by camera calibration executed by controlling one or more cameras to shoot a subject from a plurality of viewpoints, the map information including three-dimensional points each indicating a position on the subject in a three-dimensional space; obtaining a first image from a first view point and a second image from a second viewpoint; determining a search range in the three-dimensional space, based on the map information, the search range including a first three-dimensional point on the subject, the first three-dimensional point corresponding to a first point in the first image; searching for a similar point that is similar to the first point, in a range in the second image which corresponds to the search range; and generating a three-dimensional model using a search result in the searching.

## Claims

### Claim 1

A three-dimensional model generation method executed by an information processing device, the three-dimensional model generation method comprising:
obtaining map information generated by camera calibration executed by controlling one or more cameras to shoot a subject from a plurality of viewpoints, the map information including three-dimensional points each indicating a position on the subject in a three-dimensional space;
obtaining a first image of the subject shot from a first view point and a second image of the subject shot from a second viewpoint;
determining a search range in the three-dimensional space, based on the map information, the search range including a first three-dimensional point on the subject, the first three-dimensional point corresponding to a first point in the first image;
searching for a similar point that is similar to the first point, in a range in the second image which corresponds to the search range; and
generating a three-dimensional model using a search result in the searching.

### Claim 2 (depends on 1)

The three-dimensional model generation method according to claim 1, wherein
the search range is a range in a shooting direction from the first viewpoint, and
in the searching, an epipolar line in the second image is limited to a length that is in accordance with the search range, and the similar point that is similar to the first point is searched for on the epipolar line in the second image, the epipolar line corresponding to the first point.

### Claim 3 (depends on 1)

The three-dimensional model generation method according to claim 1, wherein
the determining includes interpolating, using the three-dimensional points included in the map information, a three-dimensional point at which the subject is estimated to be present, the three-dimensional point being interpolated between two points included in the three-dimensional points, and
in the determining, the search range is determined using the three-dimensional points after interpolation in the interpolating.

### Claim 4 (depends on 3)

The three-dimensional model generation method according to claim 3, wherein
in the interpolating, the three-dimensional point at which the subject is estimated to be present is interpolated in a region defined by three points included in the three-dimensional points.

### Claim 5 (depends on 1)

The three-dimensional model generation method according to claim 1, wherein
in the determining:
a distance image is generated by estimating, for each of regions in a projection image obtained by projecting the three-dimensional points onto the first image, a three-dimensional position on the subject which corresponds to the region and has the first viewpoint as a reference, the distance image including the three-dimensional positions respectively estimated for the regions; and
the search range is determined based on each of the three-dimensional positions of the regions in the distance image.

### Claim 6 (depends on 5)

The three-dimensional model generation method according to claim 5, wherein
in the determining, for the search range which is determined for each of the regions in the distance image, the search range is set to be wider as an estimation accuracy of the three-dimensional position of the region is lower.

### Claim 7 (depends on 6)

The three-dimensional model generation method according to claim 6, wherein
the estimation accuracy of the three-dimensional position is higher as a distance of the three-dimensional position from the first viewpoint is less.

### Claim 8 (depends on 6)

The three-dimensional model generation method according to claim 6, wherein
the estimation accuracy of the three-dimensional position is higher as the three-dimensional position is closer to at least one three-dimensional point among the three-dimensional points.

### Claim 9 (depends on 6)

The three-dimensional model generation method according to claim 6, wherein
the estimation accuracy of the three-dimensional position is higher as a density of the three-dimensional points in the region in which the three-dimensional position is present is higher.

### Claim 10 (depends on 1)

The three-dimensional model generation method according to claim 1, wherein
in the camera calibration, positions and orientations of the one or more cameras are calculated, and the first image and the second image are included among images shot for the camera calibration.

### Claim 11

A three-dimensional model generation device comprising:
a processor; and
memory, wherein
using the memory, the processor:
obtains map information generated by camera calibration executed by controlling one or more cameras to shoot a subject from a plurality of viewpoints, the map information including three-dimensional points each indicating a position on the subject in a three-dimensional space;
obtains a first image of the subject shot from a first view point and a second image of the subject shot from a second viewpoint;
determines a search range in the three-dimensional space, based on the map information, the search range including a first three-dimensional point on the subject, the first three-dimensional point corresponding to a first point in the first image;
searches for a similar point that is similar to the first point, in a range in the second image which corresponds to the search range; and
generates a three-dimensional model using a result of the search.
