# US 2022/0027654 A1 · Point cloud data processing apparatus, point cloud data processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0027654 A1
- Document ID: US20220027654A1
- Status: Published application
- Kind: A1
- Application number: 17/496,078
- Filed: 2021-10-07
- Published: 2022-01-27
- Earliest claimed: 2019-06-14
- Assignees: FUJIFILM Corporation
- Applicants: FUJIFILM Corporation
- Inventors: Kazuchika IWAMI
- Web page: https://patentgrep.com/patent/US20220027654A1

## CPC

- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06T 7/11: Region-based segmentation
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06V 20/64: Three-dimensional [3D] objects
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A point cloud data processing apparatus includes: an image data acquisition unit that acquires image data of an object; a point cloud data acquisition unit that acquires point cloud data; a recognition unit that recognizes the object on the basis of the image data, and acquires a region of the object and attribute information for identifying the object; and an attribute assigning unit that selects, from the point cloud data, point cloud data that belongs to the region of the object, and assigns the identified attribute information to the selected point cloud data.

## Claims

### Claim 1

A point cloud data processing apparatus comprising:
a processor that is configured to
acquire image data of an object,
acquire point cloud data having a corresponding positional relationship with the image data and representing pieces of three-dimensional information of a large number of points on an outer surface of the object,
recognize the object on the basis of the image data, and acquire a region of the object and attribute information for identifying the object, and
select, from the point cloud data, point cloud data that belongs to the region of the object, and assign the acquired attribute information to the selected point cloud data.

### Claim 2 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to
acquire, in a case where the image data includes pieces of image data of a plurality of objects, a region of each object among the objects and attribute information for identifying the object on a per object basis, and
select, from the point cloud data, point cloud data that belongs to the region of each object among the objects on a per object basis, and assign the attribute information of the object to the point cloud data selected on a per object basis.

### Claim 3 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to
recognize, in a case where the image data includes image data of the object having a plurality of partial regions, a region of each partial region among the partial regions, and acquire the region of the partial region and attribute information for identifying the partial region, and
select, from the point cloud data, point cloud data that belongs to the region of each partial region among the partial regions on a per partial region basis, and assign the attribute information of the partial region to the point cloud data selected on a per partial region basis.

### Claim 4 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to
detect a missing part of the point cloud data that belongs to the region of the object on the basis of at least the region of the recognized object, and interpolate point cloud data of the detected missing part, and
assigns the attribute information to the complemented point cloud data.

### Claim 5 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to acquire a captured image acquired by image capturing of the object as the image data.

### Claim 6 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to
generate the image data on the basis of the point cloud data, and
acquire the generated image data.

### Claim 7 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is formed of a recognizer subjected to machine learning or deep learning.

### Claim 8 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the processor is configured to generate a three-dimensional model of the object on the basis of the point cloud data that is assigned the attribute information.

### Claim 9 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the image data and the point cloud data are respectively acquired by devices having the same optical axis.

### Claim 10 (depends on 1)

The point cloud data processing apparatus according to claim 1, wherein the image data and the point cloud data are respectively acquired by devices for which a positional relationship between the devices is known.

### Claim 11

A point cloud data processing method comprising:
acquiring image data of an object;
acquiring point cloud data having a corresponding positional relationship with the image data and representing pieces of three-dimensional information of a large number of points on an outer surface of the object;
recognizing the object on the basis of the image data, and acquiring a region of the object and attribute information for identifying the object; and
selecting, from the point cloud data, point cloud data that belongs to the region of the object, and assigning the acquired attribute information to the selected point cloud data.

### Claim 12

A non-transitory computer readable recording medium storing a program for causing a computer to perform a point cloud data process comprising:
acquiring image data of an object;
acquiring point cloud data having a corresponding positional relationship with the image data and representing pieces of three-dimensional information of a large number of points on an outer surface of the object;
recognizing the object on the basis of the image data, and acquiring a region of the object and attribute information for identifying the object; and
selecting, from the point cloud data, point cloud data that belongs to the region of the object, and assigning the acquired attribute information to the selected point cloud data.
