Generating object representations using neural networks for autonomous systems and applications
AssigneeNVIDIA Corporation
InventorsAbhishek Bajpayee, Sai Krishnan Chandrasekar, Xudong Chen, Hae Jong Seo, Siddharth Kothiyal
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed that relate to object detection and to generating detected object representations. Sensor data corresponding to a scene may be obtained that may represent one or more objects. A tensor may be generated based at least on the sensor data, where the tensor may represent the one or more objects and may include respective predicted 3D characteristics of the one or more objects. The tensor may be represented in 2D space and may be decoded to generate 3D representations of objects using, for example, one or more curve fitting algorithms.
Claims19
A method comprising:
- computing, based at least on processing sensor data generated using a sensor, an output corresponding to one or more objects, the output being representative of:
- two-dimensional (2D) locations of points corresponding to the one or more objects,
- one or more semantic classifiers associated with the one or more objects, and
- depth information corresponding to the points;
- determining three-dimensional (3D) locations of the points based at least on the 2D locations and the depth information included in the output;
- and
- generating one or more 3D representations of the one or more objects based at least on the 3D locations and the one or more semantic classifiers included in the output.
A system comprising:
- one or more processing units to perform one or more operations using a machine based at least on data encoded in a tensor output using one or more deep neural networks (DNNs), the one or more DNNs trained, at least, by:
- generating, using the one or more DNNs and based at least on training sensor data, a predicted tensor representing predicted three-dimensional (3D) characteristics of one or more objects represented by the training sensor data;
- comparing, using one or more loss functions, values of the predicted tensor to values of a ground truth tensor, the ground truth tensor representing known 3D characteristics of the one or more objects;
- and
- modifying one or more parameters of the one or more DNNs based at least on the comparing.
A processor comprising:
- one or more processing units to perform one or more operations based at least on a three-dimensional (3D) representation of a detected landmark, the 3D representation determined based at least on two-dimensional (2D) location information, semantic classifier information, and depth information corresponding to two or more points of a curve corresponding to the detected landmark.
Dependencies are inferred from the claim text.