Back to results
US 12,475,588 B2Granted
Systems and methods for object detection including pose and size estimation
AssigneeGoogle Inc.
InventorsTingbo Hou, Adel Ahmadyan, Jianing Wei, Matthias Grundmann
Life of this patent
Estimating the term…
Abstract
The present disclosure is directed to systems and methods for performing object detection and pose estimation in 3D from 2D images. Object detection can be performed by a machine-learned model configured to determine various object properties. Implementations according to the disclosure can use these properties to estimate object pose and size.
Claims20
A computer-implemented method for three-dimensional (3D) object detection in two-dimensional (2D) images, the method comprising:
- obtaining, by one or more computing devices, a 2D image that includes an object, wherein the 2D image comprises a plurality of pixels;
- inputting, by the one or more computing devices, the 2D image into a machine-learned object detection model comprising a plurality of heads, wherein the plurality of heads comprise:
- a first head configured to generate a centroid heat map, wherein the centroid heat map provides a respective heat value for each of the plurality of pixels, and wherein the respective heat value for each pixel describes a probability that a centroid of the object is depicted by such pixel;
- and
- a second head configured to generate a plurality of distance fields, wherein each distance field provides a respective distance value for each of contributing pixels of the plurality of pixels, and wherein the respective distance value for each pixel of the contributing pixels of the plurality of pixels indicates a distance to one of a plurality of vertices associated with a bounding box for the object, wherein each distance field provides the respective distance value for each of the contributing pixels of the plurality of pixels that has a heat value above a predetermined value;
- generating, by the one or more computing devices, the centroid heat map with the first head of the machine-learned object detection model;
- generating, by the one or more computing devices, the plurality of distance fields with the second head of the machine-learned object detection model;
- and
- determining, by the one or more computing devices and based at least in part on the centroid heat map and the plurality of distance fields, a dataset comprising 3D coordinates for the bounding box associated with the object in the 2D image.
6 more elements
A computing system comprising:
- one or more processors;
- and
- one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
- obtaining a 2D image that includes an object, wherein the 2D image comprises a plurality of pixels;
- inputting the 2D image into a machine-learned object detection model comprising a plurality of heads, wherein the plurality of heads comprise:
- a first head configured to generate a centroid heat map, wherein the centroid heat map provides a respective heat value for each of the plurality of pixels, and wherein the respective heat value for each pixel describes a probability that a centroid of the object is depicted by such pixel;
- and
- a second head configured to generate a plurality of distance fields, wherein each distance field provides a respective distance value for each of contributing pixels of the plurality of pixels, wherein the respective distance value for each pixel of the contributing pixels of the plurality of pixels indicates a distance to one of a plurality of vertices associated with a bounding box for the object, wherein each distance field provides the respective distance value for each of the contributing pixels of the plurality of pixels that has a heat value above a predetermined value;
- generating the centroid heat map with the first head of the machine-learned object detection model;
- generating the plurality of distance fields with the second head of the machine-learned object detection model;
- and
- determining, based at least in part on the centroid heat map and the plurality of distance fields, a dataset comprising 3D coordinates for the bounding box associated with the object in the 2D image.
9 more elements
Dependencies are inferred from the claim text.