# US 2021/0341920 A1 · Methods and systems for trajectory forecasting with recurrent neural networks using inertial behavioral rollout

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0341920 A1
- Document ID: US20210341920A1
- Status: Published application
- Kind: A1
- Application number: 17/373,090
- Filed: 2021-07-12
- Published: 2021-11-04
- Earliest claimed: 2019-05-29
- Assignees: Argo AI, LLC
- Applicants: Argo AI, LLC
- Inventors: Jagjeet Singh; Andrew T. Hartnett; G. Peter K. Carr; Slawomir W. Bak
- Web page: https://patentgrep.com/patent/US20210341920A1

## CPC

- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- B60W 50/0097: {Predicting future conditions}
- B60W 60/0027: {using trajectory prediction for other traffic participants}
- B60W 60/00276: {for two or more other traffic participants}
- G05D 1/0221: {involving a learning process}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- B60W 2554/00: Input parameters relating to objects
- B60W 2554/4041: Position
- B60W 2554/406: Traffic density
- G06N 3/048: Activation functions
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for forecasting trajectories of objects. The method includes obtaining a prediction model trained to predict future trajectories of objects. The prediction model is trained over a first prediction horizon selected to encode inertial constraints in a predicted trajectory and over a second prediction horizon selected to encode behavioral constraints in the predicted trajectory. The method also include generating a planned trajectory of an autonomous vehicle by receiving state data corresponding to the autonomous vehicle, receiving perception data corresponding to an object, predicting a future trajectory of the object based on the perception data and the prediction model, and generating the planned trajectory of the autonomous vehicle based on the future trajectory of the object and the state data.

## Claims

### Claim 1

A method comprising:
obtaining a prediction model trained to predict future trajectories of objects, the prediction model trained over a first prediction horizon selected to encode inertial constraints in a predicted trajectory and over a second prediction horizon selected to encode behavioral constraints in the predicted trajectory; and
generating a planned trajectory of an autonomous vehicle, the generating comprising:
receiving state data corresponding to the autonomous vehicle;
receiving perception data corresponding to an object;
predicting, using the prediction model, a future trajectory of the object based on the perception data; and
generating the planned trajectory of the autonomous vehicle based on the future trajectory of the object and the state data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the prediction model is trained based on a plurality of object trajectory sequences.

### Claim 3 (depends on 1)

The method of claim 1, wherein the second prediction horizon is longer than the first prediction horizon.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first prediction horizon is less than 1 second and the second prediction horizon is greater than about 2 seconds.

### Claim 5 (depends on 1)

The method of claim 1, wherein the perception data comprises one or more of velocity information, acceleration information, location information, heading information, or orientation information of the object.

### Claim 6 (depends on 1)

The method of claim 1, further comprising analyzing the perception data to determine an object trajectory over a time period in a curvilinear coordinate system.

### Claim 7 (depends on 6)

The method of claim 6, wherein predicting the future trajectory of the object comprises predicting the future trajectory based on the object trajectory over the time period in the curvilinear coordinate system.

### Claim 8

A device comprising:
a memory that stores a prediction model trained to predict future trajectories of objects, the prediction model trained over a first prediction horizon selected to encode inertial constraints in a predicted trajectory and over a second prediction horizon selected to encode behavioral constraints in the predicted trajectory; and
a processor coupled to the memory and configured to:
generate a planned trajectory of an autonomous vehicle, the generating comprising:
receiving state data corresponding to the autonomous vehicle;
receiving perception data corresponding to an object;
predicting, using the prediction model, a future trajectory of the object based on the perception data; and
generating the planned trajectory of the autonomous vehicle based on the future trajectory of the object and the state data.

### Claim 9 (depends on 8)

The device of claim 8, wherein the prediction model is trained based on a plurality of object trajectory sequences.

### Claim 10 (depends on 8)

The device of claim 8, wherein the second prediction horizon is longer than the first prediction horizon.

### Claim 11 (depends on 10)

The device of claim 10, wherein the first prediction horizon is less than 1 second and the second prediction horizon is greater than about 2 seconds.

### Claim 12 (depends on 8)

The device of claim 8, wherein the perception data comprises one or more of velocity information, acceleration information, location information, heading information, or orientation information of the object.

### Claim 13 (depends on 8)

The device of claim 8, wherein the processor is further configured to analyze the perception data to determine an object trajectory over a time period in a curvilinear coordinate system.

### Claim 14 (depends on 13)

The device of claim 13, wherein, to predict the future trajectory of the object, the processor is further configured to predict the future trajectory based on the object trajectory over the time period in the curvilinear coordinate system.

### Claim 15

A non-transitory computer readable medium that stores instructions that, when executed by a computing device, cause the computing device to execute a method, the method comprising:
obtaining a prediction model trained to predict future trajectories of objects, the prediction model trained over a first prediction horizon selected to encode inertial constraints in a predicted trajectory and over a second prediction horizon selected to encode behavioral constraints in the predicted trajectory; and
generating a planned trajectory of an autonomous vehicle, the generating comprising:
receiving state data corresponding to the autonomous vehicle;
receiving perception data corresponding to an object;
predicting, using the prediction model, a future trajectory of the object based on the perception data; and
generating the planned trajectory of the autonomous vehicle based on the future trajectory of the object and the state data.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the prediction model is trained based on a plurality of object trajectory sequences.

### Claim 17 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein the second prediction horizon is longer than the first prediction horizon.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the perception data comprises one or more of velocity information, acceleration information, location information, heading information, or orientation information of the object.

### Claim 19 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the method further comprises analyzing the perception data to determine an object trajectory over a time period in a curvilinear coordinate system.

### Claim 20 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein predicting the future trajectory of the object comprises predicting the future trajectory based on the object trajectory over the time period in the curvilinear coordinate system.