Back to results
US 2021/0343022 A1Published application
Multi-modal sensor data association architecture
AssigneeZoox, Inc.
InventorsJoshua Kriser Cohen, Sabeek Mani Pradhan, Balazs Kovacs, Cooper Stokes Sloan
Life of this patent
Estimating the term…
Abstract
A machine-learning architecture may be trained to determine point cloud data associated with different types of sensors with an object detected in an image and/or generate a three-dimensional region of interest (ROI) associated with the object. In some examples, the point cloud data may be associated with sensors such as, for example, a lidar device, radar device, etc.
Claims1
A method comprising:
- receiving first sensor data associated with a first type of sensor, the first sensor data representing a portion of an environment surrounding an autonomous vehicle;
- receiving second sensor data associated with a second type of sensor, the second sensor data representing a same portion or different portion of the environment as the portion represented by the first sensor data;
- receiving an object detection, wherein the object detection identifies an object in one or more images;
- determining, based at least in part on the object detection, a first subset of the first sensor data and a second subset of the second sensor data;
- inputting the first subset of the first sensor data into a first subnetwork;
- inputting the second subset of the second sensor data into a second subnetwork;
- receiving a first output from the first subnetwork and a second output from the second subnetwork;
- combining, as a combined output, the first output and the second output;
- inputting a first portion of the combined output into a third subnetwork and a second portion of the combined output into a fourth subnetwork;
- and
- receiving a first map from the third subnetwork and a second map from the fourth subnetwork, wherein:
- the first map indicates at least a first probability that a first point of the first sensor data is associated with the object, and
- the second map indicates at least a second probability that a second point of the second sensor data is associated with the object.
10 more elements
Dependencies are inferred from the claim text.