# US 10,482,003 B1 · Method and system for modifying a control unit of an autonomous car

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,482,003 B1
- Document ID: US10482003B1
- Status: Granted
- Kind: B1
- Application number: 16/185,088
- Filed: 2018-11-09
- Granted: 2019-11-19
- Earliest claimed: 2018-11-09
- Assignees: AImotive Kft.
- Applicants: AImotive Kft.
- Inventors: András Bondor; László Kishonti
- Web page: https://patentgrep.com/patent/US10482003B1

## CPC

- G01M 17/06: Steering behaviour; Rolling behaviour
- G06F 11/3684: {for test design, e.g. generating new test cases}
- B60W 60/00: Drive control systems specially adapted for autonomous road vehicles
- B60W 60/0051: {from occupants to vehicle}
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G06F 30/15: Vehicle, aircraft or watercraft design
- B60W 60/0059: {Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity}
- B60W 2050/0075: {Automatic parameter input, automatic initialising or calibrating means}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G01M: TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Sensor data is recorded during testing of a control unit of an autonomous car in an autonomous driving mode. In response to being triggered by a disengagement event, recorded sensor data from an analysis time window prior to the disengagement event is logged. The logged sensor data is processed in a simulator using the control unit to control a virtual autonomous car to identify the inadequately operating component of the control unit. The component may be modified and retested until the disengagement event is avoided in the simulator. The control unit with the modified component may then be used to record new sensor data during testing of the control unit in the autonomous driving mode, and the simulation repeated on the new data.

## Claims

### Claim 1

A method for modifying a control unit of an autonomous car, the autonomous car being equipped with basic sensors adapted for recording sensor data based on which a fusion model of objects and an environment surrounding the autonomous car can be generated, wherein the sensor data recorded by the basic sensors enables the control unit to control the autonomous car in autonomous driving mode, the method comprising:
while the control unit is in a first state, recording the sensor data by the basic sensors during an autonomous test action performed in autonomous driving mode of the autonomous car,
in response to being triggered by a disengagement event, logging disengagement related data comprising the sensor data recorded by the basic sensors in an analysis time window before the disengagement event,
generating a virtual scenario in a simulator apparatus for the analysis time window based on the disengagement related data, wherein the virtual scenario comprises a virtual counterpart of the autonomous car having virtual counterparts of the basic sensors, and performing a first virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the first state of the control unit in the simulator apparatus, so that the first virtual autonomous run leads to the disengagement event in the virtual scenario,
identifying based on the virtual scenario an inadequately operating component of the control unit, wherein the disengagement event has been caused by inadequate operation of the inadequately operating component of the control unit, and
generating a second state of the control unit based on the first state of the control unit by modifying the inadequately operating component of the control unit, performing a second virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the second state of the control unit in the simulator apparatus, checking whether the disengagement event is avoided in the second virtual autonomous run, and storing the modified component of the control unit if the disengagement event is avoided in the second virtual autonomous run.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising applying validation data in generating the virtual scenario for validating a validation scenario part of the virtual scenario based on comparing a validation model of the validation scenario part to the fusion model of the validation scenario part for at least a part of the analysis time window, and the inadequately operating component of the control unit is identified by means of the comparison of the validation model of the validation scenario part and the fusion model of the validation scenario part for at least the part of the analysis time window.

### Claim 3 (depends on 2)

The method according to claim 2, characterized in that the autonomous car is equipped with a validator sensor adapted for recording the validation data.

### Claim 4 (depends on 3)

The method according to claim 3, characterized in that the validator sensor is different from the basic sensors.

### Claim 5 (depends on 4)

The method according to claim 4, characterized in that the validator sensor is a lidar device.

### Claim 6 (depends on 5)

The method according to claim 5, characterized in that a distance estimation component of the control unit is identified as the inadequately operating component of the control unit, and the distance estimation component of the control unit is modified for generating the second state of the control unit based on the first state of the control unit.

### Claim 7 (depends on 1)

The method according to claim 1, characterized in that, if the fusion model generated by means of the basic sensors in the analysis time window is found to be acceptable in the analysis time window, after performing the second virtual autonomous run, a third virtual autonomous run is performed according to the autonomous test action with the virtual counterpart of the autonomous car giving the disengagement related data to virtual counterparts of the basic sensors of the virtual counterpart of the autonomous car using the second state of the control unit in the simulator apparatus.

### Claim 8

A system for modifying a control unit of an autonomous car, the system comprising a computer connected to the control unit and including instructions that, when executed by the computer, cause the computer to:
while the control unit is in a first state, record sensor data from basic sensors of the autonomous car during an autonomous test action performed in autonomous driving mode of the autonomous car,
in response to being triggered by a disengagement event, log disengagement related data comprising the sensor data recorded in an analysis time window before the disengagement event,
generate a virtual scenario in a simulator apparatus for the analysis time window based on the disengagement related data, wherein the virtual scenario comprises a virtual counterpart of the autonomous car having virtual counterparts of the basic sensors, and perform a first virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the first state of the control unit in the simulator apparatus, so that a first virtual autonomous run leads to the disengagement event in the virtual scenario,
identify, based on the virtual scenario, an inadequately operating component of the control unit, wherein the disengagement event has been caused by inadequate operation of the inadequately operating component of the control unit, and
generate a second state of the control unit based on the first state of the control unit by modifying the inadequately operating component of the control unit, perform a second virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the second state of the control unit in the simulator apparatus, check whether the disengagement event is avoided in the second virtual autonomous run, and store the modified component of the control unit if the disengagement event is avoided in the second virtual autonomous run.

### Claim 9

A computer program product comprising a non-transitory computer readable medium storing instructions which, when executed by a computer, cause the computer to:
while the control unit is in a first state, record sensor data from basic sensors of the autonomous car during an autonomous test action performed in autonomous driving mode of the autonomous car,
in response to being triggered by a disengagement event, log disengagement related data comprising the sensor data recorded in an analysis time window before the disengagement event,
generate a virtual scenario in a simulator apparatus for the analysis time window based on the disengagement related data, wherein the virtual scenario comprises a virtual counterpart of the autonomous car having virtual counterparts of the basic sensors, and perform a first virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the first state of the control unit in the simulator apparatus, so that a first virtual autonomous run leads to the disengagement event in the virtual scenario,
identify, based on the virtual scenario, an inadequately operating component of the control unit, wherein the disengagement event has been caused by inadequate operation of the inadequately operating component of the control unit, and
generate a second state of the control unit based on the first state of the control unit by modifying the inadequately operating component of the control unit, perform a second virtual autonomous run according to the autonomous test action with the virtual counterpart of the autonomous car using the second state of the control unit in the simulator apparatus, check whether the disengagement event is avoided in the second virtual autonomous run, and store the modified component of the control unit if the disengagement event is avoided in the second virtual autonomous run.