US 12,600,385 B1Granted

Machine-learned model architecture for occluded object spawning, track generation, and/or trajectory prediction

AssigneeZoox, Inc.

InventorsFrancesco Papi, Cong Ding, Mahsa Ghafarianzadeh, Willem Prins, Nicholas George Dilip Roy, Soham Bhave, John Welling Ware

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A machine-learned architecture may use sensor data, data derived from the sensor data, and/or to determine a grid indicating objects that are occluded to one or more sensors of a vehicle and velocities associated therewith. This grid may be used to generate a candidate object detection associated with an occluded object and/or an object track associated with the occluded object. The candidate object detection and/or object track may be used to determine a predicted trajectory of the occluded object and, in cases where the occluded object is affirmed by a perception component of the vehicle to be a true positive, may be used to initialize a track associated with the newly disoccluded object.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
non-transitory memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
determining, based at least in part on at least one of sensor data or map data, a top-down representation of an environment in which a vehicle is operating and a current vehicle state of the vehicle, the top-down representation including the map data and a region of the environment occluded to one or more sensors of the vehicle;
determining, by a first machine-learned model and based at least in part on the top-down representation and the current vehicle state, a virtual object grid, wherein a portion of the virtual object grid indicates:
a likelihood that an occluded object or newly disoccluded object exists at a portion of the environment corresponding with the portion of the virtual object grid, and
a velocity associated with the portion of the virtual object grid;
determining, by a second machine-learned model based at least in part on the virtual object grid, a virtual object track indicating a candidate object detection and a current and historical at least one of position, orientation, velocity, acceleration, region of interest, or classification associated with the candidate object detection;
and
determining, by a third machine-learned model and based at least in part on the virtual object track, a predicted trajectory associated with the candidate object detection;
and
controlling the vehicle based at least in part on the virtual object track and the predicted trajectory, wherein determining the predicted trajectory associated with the candidate object detection comprises resolving the candidate object detection as one of a true positive or a false positive.

9 more elements

One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, perform operations comprising:

receiving at least one of sensor data or map data associated with an environment in which a vehicle is operating, and a current vehicle state of the vehicle;
determining, by a first machine-learned model and based at least in part on the current vehicle state and at least one of the sensor data or the map data, a virtual object grid, wherein a portion of the virtual object grid indicates:
a likelihood that an occluded object or newly disoccluded object exists at a portion of the environment corresponding with the portion of the virtual object grid, and
a velocity associated with the portion of the virtual object grid;
determining, by a second machine-learned model and based at least in part on the virtual object grid, a virtual object track indicating a candidate object detection and a current and historical at least one of position, orientation, velocity, acceleration, a region of interest, or classification associated with the candidate object detection;
and
controlling the vehicle based at least in part on the virtual object track, wherein controlling the vehicle comprises resolving the candidate object detection as one of a true positive or a false positive.

4 more elements

A method comprising:

receiving at least one of sensor data or map data associated with an environment in which a vehicle is operating, and a current vehicle state of the vehicle;
determining, by a first machine-learned model and based at least in part on the current vehicle state and at least one of the sensor data or the map data, a virtual object grid, wherein a portion of the virtual object grid indicates:
a likelihood that an occluded object or newly disoccluded object exists at a portion of the environment corresponding with the portion of the virtual object grid, and
a velocity associated with the portion of the virtual object grid;
determining, by a second machine-learned model and based at least in part on the virtual object grid, a virtual object track indicating a candidate object detection and a current and historical at least one of position, orientation, velocity, acceleration, a region of interest, or classification associated with the candidate object detection;
and
controlling the vehicle based at least in part on the virtual object track, wherein controlling the vehicle comprises resolving the candidate object detection as one of a true positive or a false positive.

4 more elements

Dependencies are inferred from the claim text.