US 10,627,818 B2Granted

Temporal prediction model for semantic intent understanding

AssigneeZoox, Inc.

InventorsBenjamin John Sapp, Yilun Wang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A temporal prediction model for semantic intent understanding is described. An agent (e.g., a moving object) in an environment can be detected in sensor data collected from sensors on a vehicle. Computing device(s) associated with the vehicle can determine, based partly on the sensor data, attribute(s) of the agent (e.g., classification, position, velocity, etc.), and can generate, based partly on the attribute(s) and a temporal prediction model, semantic intent(s) of the agent (e.g., crossing a road, staying straight, etc.), which can correspond to candidate trajectory(s) of the agent. The candidate trajectory(s) can be associated with weight(s) representing likelihood(s) that the agent will perform respective intent(s). The computing device(s) can use one (or more) of the candidate trajectory(s) to determine a vehicle trajectory along which a vehicle is to drive.

Claims20

Dependent claims

A system associated with predicting an intent of an object in an environment proximate an autonomous vehicle, the system comprising:

one or more sensors, wherein the one or more sensors are associated with the autonomous vehicle;
one or more processors;
and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform actions comprising:
receiving sensor data from the one or more sensors;
detecting, based at least in part on a portion of the sensor data, the object in the environment;
determining, based at least in part on the sensor data, at least one attribute of the object, the at least one attribute comprising a classification distribution of the object indicating individual probabilities that the object is associated with individual classes of a plurality of classes;
generating, based at least in part on the attribute and using a machine learned model, a set of intents;
associating, based at least in part on the attribute, one or more weights with a subset of intents, the subset of intents corresponding to a subset of candidate trajectories;
determining a weight associated with a candidate trajectory of the subset of candidate trajectories meets or exceeds a threshold, the candidate trajectory corresponding to the intent;
and
determining a vehicle trajectory along which the autonomous vehicle is to drive based at least in part on the intent of the object.

9 more elements

A non-transitory computer-readable medium having a set of instructions that, when executed, cause one or more processors to perform operations comprising:

receiving, from one or more sensors, sensor data associated with an environment;
determining, based at least in part on the sensor data, an object in the environment;
determining, based at least in part on the sensor data, a plurality of attributes of the object or the environment, wherein an attribute of the plurality of attributes comprises a distribution of confidence scores indicating levels of confidence that the object is associated with individual classes of one or more classes of objects;
determining, based at least in part on the plurality of attributes and a machine learned model, a set of intents of the object corresponding to a set of candidate trajectories;
associating a set of weights with the set of candidate trajectories;
selecting, based at least in part on the set of weights associated with the set of candidate trajectories, one or more candidate trajectories of the set of candidate trajectories;
and
determining, based at least in part on the one or more candidate trajectories, a vehicle trajectory along which a vehicle is to drive.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/939,192Filed 28 Mar 2018
Published: US 2019/0302767 A13 Oct 2019
Granted: 21 Apr 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification29 codes

B60W 60/00276
Using trajectory prediction for other traffic participants: for two or more other traffic participants
B60W 50/0098
Details of control systems for road vehicle drive control not related to the control of a particular sub-unit , e.g. process diagnostic or…
B60W 60/00274
Using trajectory prediction for other traffic participants: considering possible movement changes
G01S 17/58
Systems of measurement based on relative movement of target: velocity or trajectory determination systems; Sense-of-movement determination systems