Detector-tracker architecture
AssigneeZoox, Inc.
InventorsQijun Tan, Sarah Tariq
Life of this patent
Estimating the term…
Abstract
A machine-learning (ML) architecture may comprise a first ML model and/or an optical flow model that receive, as input, a first image and a second image. The first ML model may output a first feature map corresponding to the first image and a second feature map corresponding to the second image. The optical flow model may output an estimated optical flow. A deformation component may modify the second feature map, as a deformed feature map, based at least in part on the estimated optical flow. The deformed feature map and the first feature map may be concatenated together as a concatenated feature map, which may be provided to a second ML model. The second ML model may be trained to output an output ROI and/or a track in association with an object represented in the first image.
Claims20
A method comprising:
- providing, as first input to a first machine-learned model, first image data representing an object;
- generating, by the first machine-learned model and based at least in part on the first image data, first intermediate data;
- providing, as second input to the first machine learned model, second image data;
- generating, by the first machine-learned model and based at least in part on the second image data, second intermediate data;
- determining, based at least in part on the first intermediate data and the second intermediate data, a respective location of the object in the first image data and the second image data;
- and
- controlling an autonomous vehicle based at least in part on the respective location of the object in the first image data and the second image data.
A system comprising:
- one or more processors;
- and
- one or more non-transitory computer-readable media storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
- providing, as first input to a first machine-learned model, first image data representing an object;
- generating, by the first machine-learned model and based at least in part on the first image data, first intermediate data;
- providing, as second input to the first machine-learned model, second image data;
- generating, by the first machine-learned model and based at least in part on the second image data, second intermediate data;
- determining, based at least in part on the first intermediate data and the second intermediate data, a respective location of the object in the first image data and the second image data;
- and
- controlling an autonomous vehicle based at least in part on the respective location of the object in the first image data and the second image data.
One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- providing, as first input to a first machine-learned model, first sensor data representing an object;
- generating, by the first machine-learned model and based at least in part on the first sensor data, first intermediate data;
- providing, as second input to the first machine-learned model, second sensor data;
- generating, by the first machine-learned model and based at least in part on the second sensor data, second intermediate data;
- determining, based at least in part on the first intermediate data and the second intermediate data, a respective location of the object in the first sensor data and the second sensor data;
- and
- controlling an autonomous vehicle based at least in part on the respective location of the object in the first sensor data and the second sensor data.
Dependencies are inferred from the claim text.