US 11,891,088 B1Granted

Adversarial agent controls generation and problematic scenario forecasting

AssigneeZoox, Inc.

InventorsMarin Kobilarov, Jefferson Bradfield Packer, Gowtham Garimella, Andreas Pasternak, Yiteng Zhang, Ruikun Yu

Life of this patent

Estimating the term…

Loading patent…

Abstract

A reward determined as part of a machine learning technique, such as reinforcement learning, may be used to control an adversarial agent in a simulation such that a component for controlling motion of the adversarial agent is trained to reduce the reward. Training the adversarial agent component may be subject to one or more constraints and/or may be balanced against one or more additional goals. Additionally or alternatively, the reward may be used to alter scenario data so that the scenario data reduces the reward, allowing the discovery of difficult scenarios and/or prospective events.

Claims20

Dependent claims

A method comprising:

receiving scenario data indicating a type of object and an object position relative to an environment and a vehicle;
generating a first simulation based at least in part on the scenario data;
determining a first reward associated with performance of the vehicle in the first simulation over a first period of time;
determining a modification to the scenario data based at least in part on the first reward;
determining an instruction for controlling motion of the object based at least in part on the first reward, wherein at least one of the modification alters the scenario data or the instruction determines an action for controlling the object such that a subsequent reward determined in association with operation of the vehicle during a second simulation is estimated to be less than the first reward;
executing the second simulation based at least in part on the modification to the scenario data, wherein executing the second simulation comprises controlling operation of the vehicle and controlling motion of the object in a simulated environment based at least in part on the instruction;
validating a component of the vehicle based at least in part on at least one of the first simulation or the second simulation;
and
transmitting the component to a real-world vehicle, wherein operation of the real-world vehicle is based at least in part on executing the component.

6 more elements

A system comprising:

one or more processors;
and
a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving scenario data indicating an object and a vehicle;
generating a first simulation based at least in part on the scenario data;
determining a first reward associated with performance of the vehicle in the first simulation over a first period of time;
determining a modification to the scenario data based at least in part on the first reward;
determining an instruction for controlling motion of the object based at least in part on the first reward, wherein at least one of the modification alters the scenario data or the instruction determines an action for controlling the object such that a subsequent reward determined in association with operation of the vehicle during a second simulation is estimated to be less than the first reward;
and
executing the second simulation based at least in part on the modification to the scenario data, wherein executing the second simulation comprises controlling operation of the vehicle and controlling motion of the object in a simulated environment based at least in part on the instruction;
validating a component of the vehicle based at least in part on at least one of the first simulation or the second simulation;
and
transmitting the component to a real-world vehicle, wherein operation of the real-world vehicle is based at least in part on executing the component.

10 more elements

A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving scenario data indicating an object and a vehicle;
generating a first simulation based at least in part on the scenario data;
determining a first reward associated with performance of the vehicle in the first simulation over a first period of time;
determining a modification to the scenario data based at least in part on the first reward;
determining an instruction for controlling motion of the object based at least in part on the first reward, wherein at least one of the modification alters the scenario data or the instruction determines an action for controlling the object such that a subsequent reward determined in association with operation of the vehicle during a second simulation is estimated to be less than the first reward;
executing the second simulation based at least in part on the modification to the scenario data, wherein executing the second simulation comprises controlling operation of the vehicle and controlling motion of the object in a simulated environment based at least in part on the instruction;
validating a component of the vehicle based at least in part on at least one of the first simulation or the second simulation;
and
transmitting the component to a real-world vehicle, wherein operation of the real-world vehicle is based at least in part on executing the component.

6 more elements

Dependencies are inferred from the claim text.