# US 11,897,517 B2 · Information processing device, information processing method, computer program product, and vehicle control system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,517 B2 - Document ID: US11897517B2 - Status: Granted - Kind: B2 - Application number: 17/190,681 - Filed: 2021-03-03 - Published: 2022-02-03 - Granted: 2024-02-13 - Earliest claimed: 2020-07-30 - Assignees: Kabushiki Kaisha Toshiba - Applicants: Kabushiki Kaisha Toshiba - Inventors: Takayuki Sugiura; Akihito Seki - Web page: https://patentgrep.com/patent/US11897517B2 ## CPC - B60W 60/00272: {relying on extrapolation of current movement} - G06F 18/2431: Multiple classes - G06T 5/80: Geometric correction - G06T 7/73: using feature-based methods - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 2420/403: Image sensing, e.g. optical camera - G06T 2207/10012: Stereo images - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract According to an embodiment, an information processing device includes one or more processors. The processors are configured to: acquire a plurality of pieces of detection information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information including distortion due to the transmission body that exists between the detection devices and the object; detect a feature point from each of the plurality of pieces of detection information; and estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on a distortion map expressing distortion at each of the two-dimensional positions, the distortion map, the three-dimensional position, and the detection position. ## Claims ### Claim 1 An information processing device comprising: one or more processors configured to: acquire a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information each including distortion due to the transmission body that exists between the detection devices and the object; detect a feature point from each of the plurality of pieces of detection information; and estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on an initial distortion map expressing distortion at each of the two-dimensional positions, a revised distortion map, the three-dimensional position, and the detection position, wherein the initial distortion map is expressed at a resolution smaller than that of the pieces of detection information, and the feature point is corrected by using the distortion corrected according to a difference between the resolution of the initial distortion map and the resolution of the pieces of detection information. ### Claim 2 (depends on 1) The device according to claim 1, wherein the distortion is a displacement amount at each of the two-dimensional positions. ### Claim 3 (depends on 1) The device according to claim 1, wherein the one or more processors are configured to: classify the acquired plurality of pieces of detection information into a plurality of groups including a plurality of pieces of detection information having similar distortion to each other; and estimate the revised distortion map, the three-dimensional position, and the detection position for each of the plurality of groups. ### Claim 4 (depends on 3) The device according to claim 3, wherein the one or more processors are configured to classify, into a same group, a plurality of pieces of detection information in which relative positions between the detection devices and the transmission body are similar. ### Claim 5 (depends on 3) The device according to claim 3, wherein the one or more processors are configured to classify, into a same group, a plurality of pieces of detection information including distortions due to the transmission body in similar states. ### Claim 6 (depends on 1) The device according to claim 1, wherein the one or more processors are configured to: generate a mask indicating a region in which the revised distortion map is estimated; and correct the feature point based on the initial distortion map expressing the distortion at each position included in the region indicated by the mask. ### Claim 7 (depends on 6) The device according to claim 6, wherein the one or more processors are configured to generate the mask indicating a region including a detection result detected by an electromagnetic wave transmitted through the transmission body among regions included in the pieces of detection information. ### Claim 8 (depends on 6) The device according to claim 6, wherein the one or more processors are configured to generate the mask indicating a region including a specific object in the pieces of detection information. ### Claim 9 (depends on 6) The device according to claim 6, wherein the one or more processors are configured to generate the mask indicating a region including more feature points than other regions. ### Claim 10 (depends on 3) The device according to claim 3, wherein the plurality of pieces of detection information are a plurality of images, and the one or more processors are configured to classify the plurality of images into the plurality of groups. ### Claim 11 (depends on 1) The device according to claim 1, wherein the detection devices include a camera, and the transmission body includes a transparent body that transmits visible light. ### Claim 12 An information processing method comprising: acquiring a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information each including distortion due to the transmission body that exists between the detection devices and the object; detecting a feature point from each of the plurality of pieces of detection information; and estimating, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on an initial distortion map expressing distortion at each of the two-dimensional positions, a revised distortion map, the three-dimensional position; and the detection position, wherein the initial distortion map is expressed at a resolution smaller than that of the pieces of detection information, and the feature point is corrected by using the distortion corrected according to a difference between the resolution of the initial distortion map and the resolution of the pieces of detection information. ### Claim 13 A computer program product comprising a non-transitory computer readable medium including programmed instructions, the instructions causing a computer to execute: acquiring a plurality of pieces of detection formation that is information including detection results at two-dimensional positions different from each other acquired by detection of an object by one or more detection devices through a transmission body, the plurality of pieces of detection information each including distortion due to the transmission body that exists between the detection devices and the object; detecting a feature point from each of the plurality of pieces of detection information; and estimating, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on an initial distortion map expressing distortion at each of the two-dimensional positions, a revised distortion map, the three-dimensional position, and the detection position, wherein the initial distortion map is expressed at a resolution smaller than that of the pieces of detection information, and the feature point is corrected by using the distortion corrected according to a difference between the resolution of the initial distortion map and the resolution of the pieces of detection information. ### Claim 14 A vehicle control system that controls a vehicle, the system comprising: an information processing device configured to estimate a three-dimensional position of an object; and a vehicle control device configured to control a drive mechanism to drive the vehicle based on the three-dimensional position, the information processing device comprising: one or more processors configured to: acquire a plurality of pieces of detection information that is information including detection results at two-dimensional positions different from each other acquired by detection of the object by one or more detection devices through a transmission body, the plurality of pieces of detection information each including distortion due to the transmission body that exists between the detection devices and the object, detect a feature point from each of the plurality of pieces of detection information, and estimate, by minimizing an error between a three-dimensional position corresponding to the feature point and a detection position of the feature point corrected based on an initial distortion map expressing distortion at each of the two-dimensional positions, a revised distortion map, the three-dimensional position, and the detection position, wherein the initial distortion map is expressed at a resolution smaller than that of the pieces of detection information, and the feature point is corrected by using the distortion corrected according to a difference between the resolution of the initial distortion map and the resolution of the pieces of detection information.