Augmenting lane-topology reasoning with a standard definition navigation map
AssigneeNVIDIA Corporation
InventorsKatie Z. Luo, Yue Wang, Xinshuo Weng, Marco Pavone, Yan Wang
Life of this patent
Estimating the term…
Abstract
In the context of autonomous driving, the recognition of lane topologies is required for the vehicle to make well-informed and prudent decisions such as lane changes, navigation through intricate intersections, and smooth merging. Current autonomous driving systems rely solely on sensor (e.g. camera) inputs to recognize lane topology. As a result, poor sensor data will have a direct negative impact on lane topology recognition. The present disclosure augments lane topology reasoning with a standard definition navigation map for use in autonomous driving applications.
Claims27
A method, comprising:
- at a device:
- encoding a representation of a map defining road geometry and road connectivity in a geographical region to form a first feature representation of the geographical region;
- generating a second feature representation of the geographical region from at least one image of the geographical region captured by at least one camera installed on an autonomous driving vehicle;
- applying cross-attention between the first feature representation and the second feature representation to construct a combined feature representation for the geographical region that includes information derived from the at least one image of the geographical region and information from the map;
- performing lane-topology reasoning by a lane-topology model, using the combined feature representation, to detect one or more road-level features within the geographical region;
- using the one or more road-level features to make one or more autonomous driving decisions for the autonomous driving vehicle;
- and
- controlling operation of the autonomous driving vehicle based on the one or more autonomous driving decisions.
A system, comprising:
- a non-transitory memory storage comprising instructions;
- and
- one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:
- encode a representation of a map defining road geometry and road connectivity in a geographical region to form a first feature representation of the geographical region;
- generate a second feature representation of the geographical region from at least one image of the geographical region captured by at least one camera installed on an autonomous driving vehicle;
- apply cross-attention between the first feature representation and the second feature representation to construct a combined feature representation for the geographical region that includes information derived from the at least one image of the geographical region and information from the map;
- perform lane-topology reasoning by a lane-topology model, using the combined feature representation, to detect one or more road-level features within the geographical region;
- use the one or more road-level features to make one or more autonomous driving decisions for the autonomous driving vehicle;
- and
- control operation of the autonomous driving vehicle based on the one or more autonomous driving decisions.
A non-transitory computer-readable medium comprising computer instructions executable by one or more processors of a device to cause the device to:
- encode a representation of a map defining road geometry and road connectivity in a geographical region to form a first feature representation of the geographical region;
- generate a second feature representation of the geographical region from at least one image of the geographical region captured by at least one camera installed on an autonomous driving vehicle;
- apply cross-attention between the first feature representation and the second feature representation to construct a combined feature representation for the geographical region that includes information derived from the at least one image of the geographical region and information from the map;
- perform lane-topology reasoning by a lane-topology model, using the combined feature representation, to detect one or more road-level features within the geographical region;
- use the one or more road-level features to make one or more autonomous driving decisions for the autonomous driving vehicle;
- and
- control operation of the autonomous driving vehicle based on the one or more autonomous driving decisions.
Dependencies are inferred from the claim text.