Motion prediction based on appearance
AssigneeZoox, Inc.
InventorsVasiliy Karasev, Tencia Lee, James William Vaisey Philbin, Sarah Tariq, Kai Zhenyu Wang
Life of this patent
Estimating the term…
Abstract
Techniques for determining and/or predicting a trajectory of an object by using the appearance of the object, as captured in an image, are discussed herein. Image data, sensor data, and/or a predicted trajectory of the object (e.g., a pedestrian, animal, and the like) may be used to train a machine learning model that can subsequently be provided to, and used by, an autonomous vehicle for operation and navigation. In some implementations, predicted trajectories may be compared to actual trajectories and such comparisons are used as training data for machine learning.
Claims20
A system comprising:
- one or more processors;
- and
- non-transitory computer-readable media storing instructions that when executed by the one or more processors, cause the system to perform operations comprising:
- receiving first sensor data captured by one or more sensors of an autonomous vehicle at a first time, the first sensor data comprising first image data;
- detecting, based at least in part on the first image data, a pedestrian having a pose in an environment of the autonomous vehicle;
- determining, based at least in part on one or more of the first sensor data or second sensor data captured by the one or more sensors at a second time comprising a second image associated with a representation of the pedestrian, a measured speed of the pedestrian;
- inputting, into a machine learned model, the first image data;
- receiving, from the machine learned model, a predicted speed of the pedestrian;
- determining a difference between the predicted speed of the pedestrian and the measured speed of the pedestrian;
- and
- altering one or more parameters of the machine learned model to minimize the difference to obtain a trained machine learned model.
A method comprising:
- receiving first sensor data captured by one or more sensors at a first time, the first sensor data comprising first image data;
- detecting, based at least in part on the first image data, an object in an environment of the one or more sensors;
- determining, based at least in part on the first sensor data, a measured speed of the object at the first time;
- inputting, into a machine learned model, the first image data;
- receiving, from the machine learned model, a predicted speed of the object;
- determining a difference between the predicted speed of the object and the measured speed of the object;
- and
- altering one or more parameters of the machine learned model to minimize the difference to obtain a trained machine learned model.
A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising:
- receiving first sensor data captured by one or more sensors at a first time, the first sensor data comprising first image data;
- detecting, based at least in part on the first image data, an object having a pose in an environment of the one or more sensors;
- determining a measured speed of the object at the first time;
- inputting, into a machine learned model, the first image data;
- receiving, from the machine learned model, a predicted speed of the object;
- determining a difference between the predicted speed of the object and the measured speed of the object;
- and
- altering one or more parameters of the machine learned model to minimize the difference to obtain a trained machine learned model.
Dependencies are inferred from the claim text.