# US 2022/0032953 A1 · Systems and methods for controlling vehicles using an amodal cuboid based algorithm

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0032953 A1
- Document ID: US20220032953A1
- Status: Published application
- Kind: A1
- Application number: 17/504,600
- Filed: 2021-10-19
- Published: 2022-02-03
- Earliest claimed: 2020-05-07
- Assignees: Argo AI, LLC
- Applicants: Argo AI, LLC
- Inventors: Kevin James Player
- Web page: https://patentgrep.com/patent/US20220032953A1

## CPC

- B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles}
- B60W 60/0015: {specially adapted for safety}
- B60W 60/0025: {specially adapted for specific operations}
- B60W 60/0027: {using trajectory prediction for other traffic participants}
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/64: Three-dimensional [3D] objects
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed herein are systems, methods and computer program products for object state prediction. For example, the methods include: obtaining, by a computing device, LiDAR datasets from a LiDAR system; and generating, by the computing device, a most likely sequence of states for an object using (i) velocity and acceleration values that are each functions of center point coordinates of amodal cuboids encompassing at least some points in LiDAR point clouds comprising the LiDAR datasets and (ii) a score that is a function of a first geometrical distance of a given point in the LiDAR datasets to an edge of an amodal cuboid and a second geometrical distance from the given point to a center point of the amodal cuboid.

## Claims

### Claim 1

A system, comprising:
a processor; and
a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for object state prediction, wherein the programming instructions comprise instructions to:
obtain LiDAR datasets from a LiDAR system; and
generate a most likely sequence of states for an object using (i) velocity and acceleration values that are each functions of center point coordinates of amodal cuboids encompassing at least some points in LiDAR point clouds comprising the LiDAR datasets and (ii) a score that is a function of a first geometrical distance of a given point in the LiDAR datasets to an edge of an amodal cuboid and a second geometrical distance from the given point to a center point of the amodal cuboid.

### Claim 2 (depends on 1)

The system according to claim 1, wherein the programming instructions comprise instructions to use the most likely sequence of states for the object by an autonomous vehicle to facilitate at least one autonomous driving operation.

### Claim 3 (depends on 2)

The system according to claim 2, wherein the at least one autonomous driving operation comprises object movement tracking, object trajectory predicting, vehicle movement/motion planning, or collision avoidance.

### Claim 4 (depends on 1)

The system according to claim 1, wherein the first or second geometric distance comprises a weighted Euclidean distance of the given point to the edge or center point of the amodal cuboid.

### Claim 5 (depends on 1)

The system according to claim 1, wherein the score is further a function of weighted edges according to a probability of occlusion.

### Claim 6 (depends on 1)

The system according to claim 1, wherein the instructions to generate the most likely sequence of states for the object comprise instructions to do so using a Viterbri algorithm with inputs comprising the velocity values, the acceleration values and the score.

### Claim 7 (depends on 1)

The system according to claim 1, wherein each of the velocity values is further a function of (i) a center point coordinate for a first amodal cuboid in a given position while encompassing data points of a first LiDAR dataset that was acquired at a given time, (ii) a center point coordinate of a second amodal cuboid in the given position while encompassing data points of a second LiDAR dataset that was acquired at immediately subsequent time, and (iii) a LiDAR frame rate.

### Claim 8 (depends on 1)

The system according to claim 1, wherein each of the acceleration values is further a function of (i) a velocity of the object computed using a center point coordinate for a first amodal cuboid in a given position while encompassing data points of a first LiDAR dataset that was acquired at a given time, (ii) a velocity of the object using a center point coordinate for a second amodal cuboid in the given position while encompassing data points of a second LiDAR dataset that was acquired at an immediately subsequent time, and (iii) a LiDAR frame rate.

### Claim 9 (depends on 1)

The system according to claim 1, wherein the programming instructions comprise instructions to:
plot each said LiDAR dataset on a 3D graph;
define an amodal cuboid on the 3D graph; and
obtain a center point coordinate for the amodal cuboid at each of a plurality of different positions within the 3D graph.

### Claim 10

A method for object state prediction, comprising:
obtaining, by a computing device, LiDAR datasets from a LiDAR system; and
generating, by the computing device, a most likely sequence of states for an object using (i) velocity and acceleration values that are each functions of center point coordinates of amodal cuboids encompassing at least some points in LiDAR point clouds comprising the LiDAR datasets and (ii) a score that is a function of a first geometrical distance of a given point in the LiDAR datasets to an edge of an amodal cuboid and a second geometrical distance from the given point to a center point of the amodal cuboid.

### Claim 11 (depends on 10)

The method according to claim 10, further comprising using the most likely sequence of states for the object by an autonomous vehicle to facilitate at least one autonomous driving operation.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the at least one autonomous driving operation comprises object movement tracking, object trajectory predicting, vehicle movement/motion planning, or collision avoidance.

### Claim 13 (depends on 10)

The method according to claim 10, wherein the first or second geometric distance comprises a weighted Euclidean distance of the given point to the edge or center point of the amodal cuboid.

### Claim 14 (depends on 10)

The method according to claim 10, wherein the score is further a function of weighted edges according to a probability of occlusion.

### Claim 15 (depends on 10)

The method according to claim 10, wherein generating the most likely sequence of states for the object comprises using a Viterbri algorithm with inputs comprising the velocity values, the acceleration values and the score.

### Claim 16 (depends on 10)

The method according to claim 10, wherein each said velocity value is further a function of (i) a center point coordinate for a first amodal cuboid in a given position while encompassing data points of a first LiDAR dataset that was acquired at a given time, (ii) a center point coordinate of a second amodal cuboid in the given position while encompassing data points of a second LiDAR dataset that was acquired at immediately subsequent time, and (iii) a LiDAR frame rate.

### Claim 17 (depends on 10)

The method according to claim 10, wherein each said acceleration value is further a function of (i) a velocity of the object computed using a center point coordinate for a first amodal cuboid in a given position while encompassing data points of a first LiDAR dataset that was acquired at a given time, (ii) a velocity of the object using a center point coordinate for a second amodal cuboid in the given position while encompassing data points of a second LiDAR dataset that was acquired at an immediately subsequent time, and (iii) a LiDAR frame rate.

### Claim 18 (depends on 10)

The method according to claim 10, further comprising:
plotting each said LiDAR dataset on a 3D graph;
defining an amodal cuboid on the 3D graph; and
obtaining a center point coordinate for the amodal cuboid at each of a plurality of different positions within the 3D graph.

### Claim 19 (depends on 18)

The method according to claim 18, wherein the plurality of different positions is arbitrarily selected.

### Claim 20

A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
obtaining LiDAR datasets from a LiDAR system; and
generating a most likely sequence of states for an object using (i) velocity and acceleration values that are each functions of center point coordinates of amodal cuboids encompassing at least some points in LiDAR point clouds comprising the LiDAR datasets and (ii) a score that is a function of a first geometrical distance of a given point in the LiDAR datasets to an edge of an amodal cuboid and a second geometrical distance from the given point to a center point of the amodal cuboid.
