# US 11,897,487 B2 · Sensor evaluation system, sensor evaluation device, and vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,897,487 B2
- Document ID: US11897487B2
- Status: Granted
- Kind: B2
- Application number: 17/470,263
- Filed: 2021-09-09
- Published: 2022-06-16
- Granted: 2024-02-13
- Earliest claimed: 2020-12-11
- Assignees: Toyota Jidosha Kabushiki Kaisha
- Applicants: Toyota Jidosha Kabushiki Kaisha
- Inventors: Hojung Jung; Hiromitsu Urano
- Web page: https://patentgrep.com/patent/US11897487B2

## CPC

- B60W 50/0205: {Diagnosing or detecting failures; Failure detection models}
- B60W 40/04: Traffic conditions
- G01S 7/40: Means for monitoring or calibrating
- G01S 7/497: Means for monitoring or calibrating
- G01S 13/865: {Combination of radar systems with lidar systems}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/931: of land vehicles
- G01S 17/931: of land vehicles
- G06T 7/73: using feature-based methods
- G06T 7/74: {involving reference images or patches}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]}
- G08G 1/0116: {from roadside infrastructure, e.g. beacons}
- G08G 1/0129: {for creating historical data or processing based on historical data}
- G08G 1/04: using optical or ultrasonic detectors
- G01S 2013/9316: {combined with communication equipment with other vehicles or with base stations}
- G01S 2013/9322: {using additional data, e.g. driver condition, road state or weather data}
- G01S 2013/9329: {cooperating with reflectors or transponders}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G07C 5/0808: {Diagnosing performance data (testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005)}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

A sensor evaluation system includes an infrastructure device installed outside a vehicle, and an information processing device. The infrastructure device includes an infrastructure sensor that detects the environment around the infrastructure device, and an infrastructure information processing device that performs information processing. The infrastructure information processing device calculates first feature point position information indicating information on a position of a feature point in the environment around the infrastructure device, based on detection information of the infrastructure sensor. The information processing device calculates second feature point position information indicating information on a position of a feature point in the environment around the vehicle, based on the detection information of a sensor. The information processing device evaluates a function of the sensor based on a difference between the first feature point position information and the second feature point position information, which are associated with the same feature point.

## Claims

### Claim 1

A sensor evaluation system which evaluates a sensor which is provided in a vehicle and which is used for detecting an environment around the vehicle, the sensor evaluation system comprising:
an infrastructure device installed outside the vehicle; and
an information processing device, wherein:
the infrastructure device includes an infrastructure sensor that detects the environment around the infrastructure device, and an infrastructure information processing device that performs information processing;
the infrastructure information processing device is configured to execute:
a first feature point detection process of detecting one or more feature points, each indicating a characteristic portion of the environment around the infrastructure device, based on detection information of the infrastructure sensor; and
a first feature point position calculation process of calculating first feature point position information which is information on positions of the one or more feature points detected in the first feature point detection process; and
the information processing device is configured to execute:
a second feature point detection process of detecting the one or more feature points in the environment around the vehicle based on the detection information of the sensor;
a second feature point position calculation process of calculating second feature point position information which is information on positions of the one or more feature points, detected in the second feature point detection process;
a difference calculation process of calculating a difference between the first feature point position information and the second feature point position information associated with the same one or more feature points; and
a function evaluation process of evaluating a function of the sensor based on a magnitude of the difference.

### Claim 2 (depends on 1)

The sensor evaluation system according to claim 1, wherein:
the first feature point position information includes information on a first detection time indicating a time when the infrastructure sensor detects the information on the position of the feature point associated with the first feature point position information;
the second feature point position information includes information on a second detection time indicating a time when the sensor detects the information on the position of the feature point associated with the second feature point position information; and
the information processing device is configured to calculate, in the difference calculation process, a difference between the first feature point position information and the second feature point position information in which the first detection time is equivalent to the second detection time.

### Claim 3 (depends on 1)

The sensor evaluation system according to claim 1, wherein the information processing device is configured to determine, in the function evaluation process, that the function of the sensor is abnormal in a case where the difference is equal to or larger than a predetermined threshold.

### Claim 4 (depends on 1)

The sensor evaluation system according to claim 1, wherein:
the vehicle is a vehicle configured to receive remote support by an operation of a remote support device located at a distance; and
the information processing device is configured to request, in the function evaluation process, the remote support from the remote support device upon being determined that the function of the sensor is abnormal.

### Claim 5

A sensor evaluation device which evaluates a sensor provided in a vehicle and which is used for detecting an environment around the vehicle, the sensor evaluation device comprising:
a memory that stores a program; and
a processor coupled to the memory,
wherein the processor is configured to, when executing the program, execute:
a process of acquiring detection information of the sensor;
a process of acquiring, from an infrastructure device installed outside the vehicle, first feature point position information which is information on positions of one or more feature points, each indicating a characteristic portion of the environment around the vehicle;
a feature point detection process of detecting the one or more feature points in the environment around the vehicle based on the detection information of the sensor;
a second feature point position calculation process of calculating second feature point position information which is information on positions of the one or more feature points, detected in the feature point detection process;
a difference calculation process of calculating a difference between the first feature point position information and the second feature point position information associated with the same one or more feature points; and
a function evaluation process of evaluating a function of the sensor based on a magnitude of the difference.

### Claim 6 (depends on 5)

The sensor evaluation device according to claim 5, wherein:
the first feature point position information includes information on a first detection time indicating a time when the infrastructure device detects the information on the position of the feature point associated with the first feature point position information;
the second feature point position information includes information on a second detection time indicating a time when the sensor detects the information on the position of the feature point associated with the second feature point position information; and
the processor is configured to calculate, in the difference calculation process, a difference between the first feature point position information and the second feature point position information in which the first detection time is equivalent to the second detection time.

### Claim 7 (depends on 5)

The sensor evaluation device according to claim 5, wherein the processor is configured to determine, in the function evaluation process, that the function of the sensor is abnormal in a case where the difference is equal to or larger than a predetermined threshold.

### Claim 8 (depends on 5)

The sensor evaluation device according to claim 5, wherein:
the vehicle is a vehicle configured to receive remote support by an operation of a remote support device located at a distance; and
in the function evaluation process, the remote support of the remote support device is requested upon determination that the function of the sensor is abnormal.

### Claim 9

A vehicle comprising:
a sensor that detects an environment around the vehicle; and
an information processing device,
wherein the information processing device is configured to execute:
a process of acquiring detection information from the sensor;
a process of acquiring first feature point position information which is information on positions of one or more feature points, each indicating a characteristic portion of the environment around the vehicle, from an infrastructure device installed outside the vehicle;
a feature point detection process of detecting the one or more feature points in the environment around the vehicle based on the detection information of the sensor;
a second feature point position calculation process of calculating second feature point position information which is information on positions of the one or more feature points;
a difference calculation process of calculating a difference between the first feature point position information and the second feature point position information associated with the same one or more feature points; and
a function evaluation process of evaluating a function of the sensor based on a magnitude of the difference.

### Claim 10 (depends on 9)

The vehicle according to claim 9, wherein:
the first feature point position information includes information on a first detection time indicating a time when the infrastructure device detects the information on the position of the feature point associated with the first feature point position information;
the second feature point position information includes information on a second detection time indicating a time when the sensor detects the information on the position of the feature point associated with the second feature point position information; and
the information processing device is configured to calculate, in the difference calculation process, a difference between the first feature point position information and the second feature point position information in which the first detection time is equivalent to the second detection time.

### Claim 11 (depends on 9)

The vehicle according to claim 9, wherein the information processing device is configured to determine, in the function evaluation process, that the function of the sensor is abnormal in a case where the difference is equal to or larger than a predetermined threshold.

### Claim 12 (depends on 9)

The vehicle according to claim 9, wherein:
the vehicle is configured to receive remote support by an operation of a remote support device located at a distance; and
the information processing device is configured to request, in the function evaluation process, the remote support from the remote support device upon being determined that the function of the sensor is abnormal.