# US 11,440,564 B2 · Method and system for data-driven and modular decision making and trajectory generation of an autonomous agent

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,440,564 B2
- Document ID: US11440564B2
- Status: Granted
- Kind: B2
- Application number: 17/579,862
- Filed: 2022-01-20
- Published: 2022-05-05
- Granted: 2022-09-13
- Earliest claimed: 2020-06-05
- Assignees: Gatik AI Inc.
- Applicants: Gatik AI Inc.
- Inventors: Gautam Narang; Apeksha Kumavat; Arjun Narang; Kinh Tieu; Michael Smart; Marko Ilievski
- Web page: https://patentgrep.com/patent/US11440564B2

## CPC

- B60W 30/18163: {Lane change; Overtaking manoeuvres}
- B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles}
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B60W 30/0956: {the prediction being responsive to traffic or environmental parameters}
- B60W 60/0015: {specially adapted for safety}
- G01C 21/3407: {specially adapted for specific applications}
- G05B 13/027: {using neural networks only}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/092: Reinforcement learning
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2554/20: Static objects
- B60W 2554/40: Dynamic objects, e.g. animals, windblown objects
- B60W 2555/60: Traffic rules, e.g. speed limits or right of way
- B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system for data-driven, modular decision making and trajectory generation includes a computing system. A method for data-driven, modular decision making and trajectory generation includes: receiving a set of inputs; selecting a learning module such as a deep decision network and/or a deep trajectory network from a set of learning modules; producing an output based on the learning module; repeating any or all of the above processes; and/or any other suitable processes. Additionally or alternatively, the method can include training any or all of the learning modules; validating one or more outputs; and/or any other suitable processes and/or combination of processes.

## Claims

### Claim 1

A system for an autonomous agent, comprising:
a set of multiple learned models;
a computing system configured to receive a set of inputs; and
a non-transitory computer readable medium having stored thereon software instructions that, when executed by the computing system, cause the computing system to execute a sequence comprising:
selecting a first learned model from the set of multiple learned models based on the set of inputs;
generating a first output using the first learned model;
based on the first output, selecting a second model from the set of multiple learned models;
determining a trajectory for the autonomous agent based on the second model; and
facilitating operation of the autonomous agent based on the trajectory.

### Claim 2 (depends on 1)

The system of claim 1, wherein the first learned model and second model are selected from a first and second subset of the set of learned models, respectively, wherein the first and second subsets are disjoint.

### Claim 3 (depends on 2)

The system of claim 2, wherein the first output comprises an action, wherein the action is associated with a plurality of learned models of the first subset.

### Claim 4 (depends on 1)

The system of claim 1, further comprising a sensor system communicatively coupled to the computing system, wherein the set of inputs comprises sensor data received from the sensor system.

### Claim 5 (depends on 1)

The system of claim 1, wherein the sequence further comprises: validating the trajectory with a set of rules.

### Claim 6 (depends on 5)

The system of claim 5, wherein the trajectory is validated with the set of rules based on an uncertainty value of the trajectory falling below an uncertainty threshold.

### Claim 7 (depends on 5)

The system of claim 5, wherein the set of rules comprise collision avoidance heuristics.

### Claim 8 (depends on 5)

The system of claim 5, wherein validating the trajectory with the set of rules comprises validating satisfaction of a traffic law.

### Claim 9 (depends on 1)

The system of claim 1, wherein the set of inputs comprises a contextual input.

### Claim 10 (depends on 9)

The system of claim 9, wherein the contextual input is selected from a set of predetermined context assignments.

### Claim 11 (depends on 10)

The system of claim 10, wherein the first learned model and the context contextual input are associated with each other in a one-to-one mapping.

### Claim 12 (depends on 1)

The system of claim 1, wherein the set of inputs comprises a state estimate of the autonomous agent.

### Claim 13 (depends on 1)

The system of claim 1, wherein each of the learned models of the set of multiple learned models is a deep Q-learning network previously trained by reinforcement learning.

### Claim 14 (depends on 1)

The system of claim 1, wherein, when executed by the computing system, the software instructions cause the computing system to repeatedly execute the sequence.

### Claim 15

A non-transitory computer readable medium having stored thereon software instructions that, when executed by a processing system of an autonomous agent, cause the processing system to determine a trajectory for the autonomous agent by:
determining a context of the autonomous agent from a map of labeled contexts;
selecting a first learned model from a first plurality of learned models based on the context;
determining a first output of the first learned model;
based on the first output, selecting a second learned model from a second plurality of learned models; and
determining a trajectory for the autonomous agent based on the second learned model;
wherein the processing system is further configured to facilitate operation of the autonomous agent based on the trajectory.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the second learned model is selected based on a stored mapping of the first output to the second learned model.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the first and second plurality of learned models are distinct.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein each of the learned models of the first and second plurality is a deep Q-learning network previously trained by reinforcement learning.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the learned models of the first plurality are trained with disjoint sets of training data.