# US 2022/0032969 A1 · Information processing device, information processing method, computer program product, and vehicle control system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0032969 A1
- Document ID: US20220032969A1
- Status: Published application
- Kind: A1
- Application number: 17/190,681
- Filed: 2021-03-03
- Published: 2022-02-03
- Earliest claimed: 2020-07-30
- Assignees: Kabushiki Kaisha Toshiba
- Applicants: Kabushiki Kaisha Toshiba
- Inventors: Takayuki SUGIURA; Akihito SEKI
- Web page: https://patentgrep.com/patent/US20220032969A1

## CPC

- B60W 60/00272: {relying on extrapolation of current movement}
- G06F 18/2431: Multiple classes
- G06T 5/80: Geometric correction
- G06T 7/73: using feature-based methods
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- B60W 2420/403: Image sensing, e.g. optical camera
- G06T 2207/10012: Stereo images
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

According to an embodiment, an information processing device includes one or more processors. The processors are configured to: acquire a plurality of pieces of detection information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object; detect a feature point from each of the plurality of pieces of detection information; and estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position.

## Claims

### Claim 1

An information processing device comprising:
one or more processors configured to:
acquire a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object;
detect a feature point from each of the plurality of pieces of detection information; and
estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position.

### Claim 2 (depends on 1)

The device according to claim 1, wherein
the distortion is a displacement amount at each of the two-dimensional positions.

### Claim 3 (depends on 1)

The device according to claim 1, wherein
the one or more processors are configured to:
classify the acquired plurality of pieces of detection information into a plurality of groups including a plurality of pieces of detection information having similar distortion to each other; and
estimate the distortion map, the three-dimensional position, and the detection position for each of the plurality of groups.

### Claim 4 (depends on 3)

The device according to claim 3, wherein
the one or more processors are configured to classify, into a same group, a plurality of pieces of detection information in which relative positions between the detection devices and the transmission body are similar.

### Claim 5 (depends on 3)

The device according to claim 3, wherein
the one or more processors are configured to classify, into a same group, a plurality of pieces of detection information including distortions due to the transmission body in similar states.

### Claim 6 (depends on 1)

The device according to claim 1, wherein
the one or more processors are configured to:
generate a mask indicating a region in which the distortion map is estimated; and
correct the feature point based on the distortion map expressing the distortion at each position included in the region indicated by the mask.

### Claim 7 (depends on 6)

The device according to claim 6, wherein
the one or more processors are configured to generate the mask indicating a region including a detection result detected by an electromagnetic wave transmitted through the transmission body among regions included in the pieces of detection information.

### Claim 8 (depends on 6)

The device according to claim 6, wherein
the one or more processors are configured to generate the mask indicating a region including a specific object in the pieces of detection information.

### Claim 9 (depends on 6)

The device according to claim 6, wherein
the one or more processors are configured to generate the mask indicating a region including more feature points than other regions.

### Claim 10 (depends on 1)

The device according to claim 1, wherein
the distortion map is expressed at resolution smaller than that of the pieces of detection information, and
the feature point is corrected by using the distortion corrected according to a difference between the resolution of the distortion map and the resolution of the pieces of detection information.

### Claim 11 (depends on 1)

The device according to claim 1, wherein
the detection devices include a camera, and
the transmission body includes a transparent body that transmits visible light.

### Claim 12

An information processing method comprising:
acquiring a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object;
detecting a feature point from each of the plurality of pieces of detection information; and
estimating, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position.

### Claim 13

A computer program product comprising a non-transitory computer readable medium including programmed instructions, the instructions causing a computer to execute:
acquiring a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object;
detecting a feature point from each of the plurality of pieces of detection information; and
estimating, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position.

### Claim 14

A vehicle control system that controls a vehicle, the system comprising:
an information processing device configured to estimate a three-dimensional position of an object; and
a vehicle control device configured to control a drive mechanism to drive the vehicle based on the three-dimensional position,
the information processing device comprising:
one or more processors configured to:
acquire a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of the object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object,
detect a feature point from each of the plurality of pieces of detection information, and
estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position.
