# US 10,816,978 B1 · Automated vehicle artificial intelligence training based on simulations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,816,978 B1
- Document ID: US10816978B1
- Status: Granted
- Kind: B1
- Application number: 15/902,894
- Filed: 2018-02-22
- Granted: 2020-10-27
- Earliest claimed: 2018-02-22
- Assignees: MSC.Software Corporation
- Applicants: MSC.Software Corporation
- Inventors: Edward Schwalb
- Web page: https://patentgrep.com/patent/US10816978B1

## CPC

- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G05D 1/0246: {using a video camera in combination with image processing means}
- G06F 30/23: using finite element methods [FEM] or finite difference methods [FDM]
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06T 11/00: Two-dimensional [2D] image generation
- G08G 1/012: {from other sources than vehicle or roadside beacons, e.g. mobile networks}
- G08G 1/0129: {for creating historical data or processing based on historical data}
- G06F 30/15: Vehicle, aircraft or watercraft design
- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Examples described herein relate to apparatuses and methods for or simulating and improving performance of an artificial intelligence (AI) driver, including but not limited to generating sensor data corresponding to a virtual environment, generating a pixelated image corresponding to the virtual environment based on the sensor data, determining actuator commands responsive to pixels in the pixelated image, wherein the decision module determines the actuator commands based on the AI driver, and simulating behaviors of the ego vehicle object using the actuator commands.

## Claims

### Claim 1

A simulation system for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
a memory; and
a processing circuit, wherein the processing circuit is configured to implement a controlled system corresponding to an ego vehicle object associated with the AV, the controlled system comprises, a sensor module, a perception module, a decision module, and an actuator module, wherein:
the sensor module generates sensor data corresponding to a virtual environment;
the perception module generates a pixelated image corresponding to the virtual environment based on the sensor data;
the decision module determines actuator commands responsive to pixels in the pixelated image, wherein the decision module determines the actuator commands based on the AI driver; and
the actuator module simulates behaviors of the ego vehicle object using the actuator commands, wherein the pixelated image comprises a pixelated graphical depiction of metrics, the metrics comprise at least one of a safety quotient, a path tracking parameter, a comfort parameter, a fuel economy parameter, and a duration of drive parameter.

### Claim 2 (depends on 1)

The simulation system of claim 1, further comprises an environment provision module configured to provide a simulated environment based on which the sensor data is generated by the sensor module.

### Claim 3 (depends on 1)

The simulation system of claim 1, wherein the pixelated image is a pixilated dashboard that comprises at least one of:
a pixelated graphical depiction of at least one of a static temporal clearance parameter and a dynamic temporal clearance parameter;
a pixelated graphical depiction of at least one traffic indicator;
a pixelated graphical depiction of a simulated environment, wherein the simulated environment comprises at least one of visibility, traction, and weather information;
a pixelated graphical depiction of a speed of the ego vehicle object; and
a pixelated graphical depiction of alerts.

### Claim 4 (depends on 3)

The simulation system of claim 3, wherein the perception module translates a list of numbers representative of the speed of the ego vehicle object into the pixelated graphical depiction of the speed.

### Claim 5 (depends on 1)

The simulation system of claim 1, wherein the perception module generates the pixelated image by determining, based on the sensor data, at least one of a velocity of the ego vehicle object, distance from the ego vehicle object to at least one traffic participant object, relative velocity to at least one traffic participant object, and traffic indicators.

### Claim 6 (depends on 1)

The simulation system of claim 1, wherein the sensor data is simulated sensor data.

### Claim 7 (depends on 1)

The simulation system of claim 1, wherein the sensor data is physical sensor data received from physical sensors of the AV.

### Claim 8

A simulation system for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
a memory; and
a processing circuit, wherein the processing circuit is configured to implement a controlled system corresponding to an ego vehicle object associated with the AV, the controlled system comprises, a sensor module, a perception module, a decision module, an actuator module, a dynamic simulation module, an environment simulation module, a measurement module, and a modification module, wherein:
the sensor module generates sensor data corresponding to a virtual environment;
the perception module generates a pixelated image corresponding to the virtual environment based on the sensor data;
the decision module determines actuator commands responsive to pixels in the pixelated image, wherein the decision module determines the actuator commands based on the AI driver;
the actuator module simulates behaviors of the ego vehicle object using the actuator commands
the dynamic simulation module determines a first predicted consequence parameter indicative of consequences of the actuator commands on the ego vehicle object;
the environment simulation module determines a second predicted consequence parameter indicative of consequences of the actuator commands on the virtual environment;
the measurement module determines an overall predicted consequence parameter based on the first predicted consequence parameter and the second predicted consequence parameter; and
the modification module modifies the AI driver to improve the overall predicted consequence parameter.

### Claim 9

A method for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
generating sensor data corresponding to a virtual environment;
generating a pixelated image corresponding to the virtual environment based on the sensor data;
determining actuator commands responsive to pixels in the pixelated image, wherein the actuator commands are determined based on the AI driver; and
simulating behaviors of the ego vehicle object using the actuator commands, wherein the pixelated image comprises a pixelated graphical depiction of metrics, the metrics comprise at least one of a safety quotient, a path tracking parameter, a comfort parameter, a fuel economy parameter, and a duration of drive parameter.

### Claim 10

A non-transitory computer-readable medium storing processor-readable instructions such that, when executed, causes a processor to train an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV) by:
generating sensor data corresponding to a virtual environment;
generating a pixelated image corresponding to the virtual environment based on the sensor data;
determining actuator commands responsive to pixels in the pixelated image, wherein the actuator commands are determined based on the AI driver; and
simulating behaviors of the ego vehicle object using the actuator commands, wherein the pixelated image comprises a pixelated graphical depiction of metrics, the metrics comprise at least one of a safety quotient, a path tracking parameter, a comfort parameter, a fuel economy parameter, and a duration of drive parameter.

### Claim 11

A method for improving an AI driver in a simulated virtual environment, the method comprising:
generating pixelated images using sensor data corresponding to a virtual environment;
storing result parameters of simulations involving the AI driver managing an ego vehicle object, wherein the result parameters are generated based on the pixelated images;
organizing the result parameters into inputs and outputs;
deriving a simplified model that reproduces the outputs based on the inputs; and
applying the simplified model to perform subsequent simulations.

### Claim 12 (depends on 11)

The method of claim 11, wherein:
the inputs comprise at least one of steering manipulation, acceleration pedal manipulation, and deceleration pedal manipulation; and
the outputs comprise at least one of chassis acceleration, velocity of the ego vehicle object, and displacement of the ego vehicle object.

### Claim 13 (depends on 11)

The method of claim 11, wherein deriving the simplified model comprises performing a Finite Element Method (FEM) analysis using, as input, discretized material properties of at least one of a vehicle represented by the ego vehicle object and a human passenger, to compute injury parameters to the human passenger as output, wherein the discretized material properties are visualized as images.

### Claim 14 (depends on 11)

The method of claim 11, wherein deriving the simplified model comprises performing a Finite Element Method (FEM) analysis to compute distortion to the ego vehicle object due to additive manufacturing.

### Claim 15 (depends on 11)

The method of claim 11, wherein deriving the simplified model comprises performing a flexible body analysis.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
limiting a number of log files corresponding to the results parameters stored; and
determining an impact to the number of log files due to a change in the simplified model.

### Claim 17 (depends on 11)

The method of claim 11, wherein deriving the simplified model comprises at least one of:
automatically selecting a kernel method from a plurality of kernel methods for deriving the simplified model;
automatically selecting a model from a plurality of models based on simulation criteria; and
deriving the simplified model using Mixture of Experts.

### Claim 18

A simulation system for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
a memory; and
a processing circuit, wherein the processing circuit is configured to:
generate pixelated images using sensor data corresponding to a virtual environment;
store result parameters of simulations involving the AI driver managing an ego vehicle object, wherein the result parameters are generated based on the pixelated images;
organize the result parameters into inputs and outputs;
derive a simplified model that reproduces the outputs based on the inputs; and
apply the simplified model to perform subsequent simulations.

### Claim 19

A non-transitory computer-readable medium storing processor-readable instructions such that, when executed, causes a processor to train an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV) by:
generating pixelated images using sensor data corresponding to a virtual environment;
storing result parameters of simulations involving the AI driver managing an ego vehicle object, wherein the result parameters are generated based on the pixelated images;
organizing the result parameters into inputs and outputs;
deriving a simplified model that reproduces the outputs based on the inputs; and
applying the simplified model to perform subsequent simulations.

### Claim 20

A method for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
generating, by a sensor module, sensor data representing a virtual environment;
generating, by a perception module, a pixelated image corresponding to the virtual environment based on the sensor data;
determining, by a candidate actuator command sequence generator, a plurality of candidate actuator command sequences based on the pixelated image, wherein each of the plurality of candidate actuator command sequences controls an ego vehicle object;
predicting consequences associated with each of the plurality of candidate actuator command sequences through virtual simulations; and
selecting an actuator command sequence from the plurality of candidate actuator command sequences based on the consequences associated with each of the plurality of candidate actuator command sequences.

### Claim 21 (depends on 20)

The method of claim 20, wherein the consequences associated with each of the plurality of candidate comprises:
consequences on the ego vehicle object determined by a control system simulator; and
consequences on the virtual environment determined by an environment simulator.

### Claim 22 (depends on 21)

The method of claim 21, further comprising:
determining a level of confidence for the consequences on the virtual environment based on uncertainties in the virtual environment;
determining factors present in the virtual environment to evaluate accuracy of the sensor data; and
measuring the virtual environment to determine parameterization of the virtual environment.

### Claim 23 (depends on 20)

The method of claim 20, wherein selecting one of the plurality of candidate actuator command sequences comprises:
measuring, by a consequence measurement module, desirability associated with the consequences associated with each of the plurality of candidate actuator command sequences; and
selecting, by a command sequence selection module, one of the plurality of candidate actuator command sequences based on the desirability associated with the consequences associated with each of the plurality of candidate actuator command sequences.

### Claim 24 (depends on 23)

The method of claim 23, wherein the desirability is measured using at least one of a safety quotient, a path tracking parameter, a comfort parameter, a fuel economy parameter, and a duration of drive parameter.

### Claim 25 (depends on 20)

The method of claim 20, selecting one of the plurality of candidate actuator command sequences comprises:
ranking the plurality of candidate actuator command sequences based on the consequences associated with each of the plurality of candidate actuator command sequences;
generating a tree that minimizes backtracking during selecting;
grouping sequences that share a number of subsequences above a threshold; and
select one of the plurality of candidate actuator command sequences using a Monte Carlo Tree Search (MCTS) based on the tree.

### Claim 26 (depends on 20)

The method of claim 20, wherein the virtual simulations are performed by learning reduced order models or performed using error-models associated with physical sensors.

### Claim 27

A simulation system for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
a memory; and
a processing circuit, wherein the processing circuit is configured to:
generate sensor data representing a virtual environment;
generate a pixelated image corresponding to the virtual environment based on the sensor data;
determine a plurality of candidate actuator command sequences based on the pixelated image, wherein each of the plurality of candidate actuator command sequences controls an ego vehicle object;
predict consequences associated with each of the plurality of candidate actuator command sequences through virtual simulations; and
select an actuator command sequence from the plurality of candidate actuator command sequences based on the consequences associated with each of the plurality of candidate actuator command sequences.

### Claim 28

A non-transitory computer-readable medium storing processor-readable instructions such that, when executed, causes a processor to train an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV) by:
generating sensor data representing a virtual environment;
generating a pixelated image corresponding to the virtual environment based on the sensor data;
determining a plurality of candidate actuator command sequences based on the pixelated image, wherein each of the plurality of candidate actuator command sequences controls an ego vehicle object;
predicting consequences associated with each of the plurality of candidate actuator command sequences through virtual simulations; and
selecting an actuator command sequence from the plurality of candidate actuator command sequences based on the consequences associated with each of the plurality of candidate actuator command sequences.

### Claim 29

A method for training an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV), comprising:
generating sensor data corresponding to a virtual environment;
generating a pixelated image corresponding to the virtual environment based on the sensor data;
determining actuator commands responsive to pixels in the pixelated image, wherein the actuator commands are determined based on the AI driver;
simulating behaviors of the ego vehicle object using the actuator commands determining a first predicted consequence parameter indicative of consequences of the actuator commands on the ego vehicle object;
determining a second predicted consequence parameter indicative of consequences of the actuator commands on the virtual environment;
determining an overall predicted consequence parameter based on the first predicted consequence parameter and the second predicted consequence parameter; and
modifying the AI driver to improve the overall predicted consequence parameter.

### Claim 30

A non-transitory computer-readable medium storing processor-readable instructions such that, when executed, causes a processor to train an Artificial Intelligence (AI) driver configured to operate an Automated Vehicle (AV) by:
generating sensor data corresponding to a virtual environment;
generating a pixelated image corresponding to the virtual environment based on the sensor data;
determining actuator commands responsive to pixels in the pixelated image, wherein the actuator commands are determined based on the AI driver;
simulating behaviors of the ego vehicle object using the actuator commands determining a first predicted consequence parameter indicative of consequences of the actuator commands on the ego vehicle object;
determining a second predicted consequence parameter indicative of consequences of the actuator commands on the virtual environment;
determining an overall predicted consequence parameter based on the first predicted consequence parameter and the second predicted consequence parameter; and
modifying the AI driver to improve the overall predicted consequence parameter.