# US 2017/0286570 A1 · Dynamic Virtual Object Generation for Testing Autonomous Vehicles in Simulated Driving Scenarios

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286570 A1
- Document ID: US20170286570A1
- Status: Published application
- Kind: A1
- Application number: 15/085,664
- Filed: 2016-03-30
- Published: 2017-10-05
- Earliest claimed: 2016-03-30
- Assignees: Toyota Jidosha Kabushiki Kaisha
- Applicants: Toyota Jidosha Kabushiki Kaisha
- Inventors: BaekGyu Kim; Shinichi Shiraishi; Jonathan Shum
- Web page: https://patentgrep.com/patent/US20170286570A1

## CPC

- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 11/3696: {Methods or tools to render software testable}
- G06F 30/15: Vehicle, aircraft or watercraft design
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The disclosure includes a system, method and tangible memory for providing a graphical display output including a simulation for testing a design for a vehicle. The method includes extracting object data from accident data describing an object that contributed to a roadway accident occurring in a real world. The method includes extracting behavior data from the accident data describing the erroneous behavior of the object. The method includes constructing an error model that describes a virtual object that represents the object in the simulation based in part on the behavior data, a frequency input and a severity input. The method includes generating the graphical display output including the simulation based on a roadway model, a vehicle model and the error model. The simulation graphically depicts a virtual vehicle present on a virtual roadway system that includes the virtual object behaving dynamically in the simulation based on the error model.

## Claims

### Claim 1

A system to provide a graphical display output including a simulation for testing a design for a vehicle, wherein the simulation includes a virtual roadway system for testing a virtual vehicle that represents the design for the vehicle in the simulation, the system comprising:
a non-transitory memory, a processor, an automated dynamic object generation system and a virtualization application, wherein the automated dynamic object generation system and the virtualization application are stored in the non-transitory memory which is communicatively coupled to the processor;
wherein the non-transitory memory stores accident data that describes a roadway accident that actually occurred in a real world, a roadway model that describes the virtual roadway system for including in the simulation and a vehicle model that describes the virtual vehicle for including in the simulation;
wherein the automated dynamic object generation system includes instructions that, responsive to being executed by the processor, cause the processor to:
extract object data from the accident data, wherein the object data describes an object included in the accident data that contributed to the roadway accident occurring in the real world;
extract behavior data from the accident data, wherein the behavior data describes an erroneous behavior of the object in the real world that contributed to the roadway accident occurring in the real world;
receive user input data that describes a frequency of the object behaving erroneously in the simulation and a severity of the erroneous behavior of the object in the simulation; and
construct an error model that describes a virtual object that represents the object in the simulation, wherein the error model is constructed based on the object data, the behavior data and the user input data so that the virtual object behaves erroneously in the simulation as described by the behavior data and the user input data; and
wherein the virtualization application includes instructions that, responsive to being executed by the processor, cause the processor to generate the graphical display output including the simulation based on the roadway model, the vehicle model and the error model, wherein the simulation includes graphics that graphically depict the virtual vehicle, the virtual roadway system and the virtual object, wherein the virtual object behaves dynamically in the simulation based on the error model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the graphical display output is displayed on a monitor.

### Claim 3 (depends on 1)

The system of claim 1, wherein the accident data is imported from an accident database.

### Claim 4 (depends on 1)

The system of claim 1, wherein the user input data includes a probability distribution that describes the frequency and the severity.

### Claim 5 (depends on 1)

The system of claim 1, wherein the virtual object and the error model are automatically generated based on the user input and no other inputs from a user.

### Claim 6 (depends on 1)

The system of claim 1, wherein the automated dynamic object generation system causes a monitor to display one or more graphical fields that are usable to provide the user input to the automated dynamic object generation system.

### Claim 7 (depends on 1)

The system of claim 1, wherein the automated dynamic object generation system is a plugin for the virtualization application.

### Claim 8

A method to provide a graphical display output including a simulation for testing a design for a vehicle, wherein the simulation includes a virtual roadway system for testing a virtual vehicle that represents the design for the vehicle in the simulation, the method comprising:
extracting object data from a set of accident data stored on a non-transitory memory, wherein the object data describes an object included in the accident data that contributed to a roadway accident occurring in a real world;
extracting behavior data from the accident data, wherein the behavior data describes an erroneous behavior of the object in the real world that contributed to the roadway accident occurring in the real world;
receiving user input data that describes a frequency of the object behaving erroneously in the simulation and a severity of the erroneous behavior of the object in the simulation;
constructing an error model that describes a virtual object that represents the object in the simulation, wherein the error model is constructed based on the object data, the behavior data and the user input data so that the virtual object behaves in the simulation based on the behavior data and the user input data; and
generating the graphical display output including the simulation based on (1) a roadway model that describes the virtual roadway system, (2) a vehicle model that describes the virtual vehicle and (3) the error model, wherein the simulation includes graphics that graphically depict the virtual vehicle present on the virtual roadway system that includes the virtual object behaving dynamically in the simulation based on the error model.

### Claim 9 (depends on 8)

The method of claim 8, wherein the graphical display output is displayed on a monitor.

### Claim 10 (depends on 8)

The method of claim 8, wherein the accident data is imported from an accident database that includes quantitative data describing the erroneous behavior of the object in the real world that contributed to the roadway accident occurring in the real world.

### Claim 11 (depends on 8)

The method of claim 8, wherein the user input data includes a probability distribution that describes the frequency and the severity.

### Claim 12 (depends on 8)

The method of claim 8, wherein the virtual object and the error model are automatically generated based on the user input and no other inputs from a user.

### Claim 13 (depends on 8)

The method of claim 8, wherein the virtual vehicle represents an autonomous vehicle and the virtual object is present in the simulation to test how the design for the autonomous vehicle responds to the behavior of the virtual object in the simulation.

### Claim 14 (depends on 8)

The method of claim 8, wherein the simulation includes a positive whole number of the virtual objects based on the frequency described by the user input data, wherein the positive whole number is proportional to the frequency.

### Claim 15

A non-transitory memory comprising computer code which, when executed by a processor, causes the processor to perform steps to provide a graphical display output including a simulation for testing a design for a vehicle, wherein the simulation includes a virtual roadway system for testing a virtual vehicle that represents the design for the vehicle in the simulation, the steps comprising:
extracting object data from a set of accident data stored on a non-transitory memory, wherein the object data describes an object included in the accident data that contributed to a roadway accident occurring in a real world;
extracting behavior data from the accident data, wherein the behavior data describes an erroneous behavior of the object in the real world that contributed to the roadway accident occurring in the real world;
receiving user input data that describes a frequency of the object behaving erroneously in the simulation and a severity of the erroneous behavior of the object in the simulation;
constructing an error model that describes a virtual object that represents the object in the simulation, wherein the error model is constructed based on the object data, the behavior data and the user input data so that the virtual object behaves in the simulation based on the behavior data and the user input data; and
generating the graphical display output including the simulation based on (1) a roadway model that describes the virtual roadway system, (2) a vehicle model that describes the virtual vehicle and (3) the error model, wherein the simulation includes graphics that graphically depict the virtual vehicle present on the virtual roadway system that includes the virtual object behaving dynamically in the simulation based on the error model.

### Claim 16 (depends on 15)

The non-transitory memory of claim 15, wherein the graphical display output is displayed on a monitor.

### Claim 17 (depends on 15)

The non-transitory memory of claim 15, wherein the accident data is imported from one or more accident databases that includes quantitative data describing the erroneous behavior of the object in the real world that contributed to the roadway accident occurring in the real world.

### Claim 18 (depends on 15)

The non-transitory memory of claim 15, wherein the user input data includes a probability distribution that describes the frequency and the severity.

### Claim 19 (depends on 15)

The non-transitory memory of claim 15, wherein the virtual object and the error model are automatically generated based on the user input and no other inputs from a user.

### Claim 20 (depends on 15)

The non-transitory memory of claim 15, wherein the virtual vehicle represents an autonomous vehicle and the virtual object is present in the simulation to test how the design for the autonomous vehicle responds to the behavior of the virtual object in the simulation.
