# US 11,897,471 B2 · Intersection detection and classification in autonomous machine applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,471 B2 - Document ID: US11897471B2 - Status: Granted - Kind: B2 - Application number: 18/162,576 - Filed: 2023-01-31 - Published: 2023-06-01 - Granted: 2024-02-13 - Earliest claimed: 2019-03-11 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Sayed Mehdi Sajjadi Mohammadabadi; Berta Rodriguez Hervas; Hang Dou; Igor Tryndin; David Nister; Minwoo Park; Neda Cvijetic; Junghyun Kwon; Trung Pham - Web page: https://patentgrep.com/patent/US11897471B2 ## CPC - B60W 30/18154: {Approaching an intersection} - B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering - B60W 30/095: Predicting travel path or likelihood of collision - B60W 60/001: {Planning or execution of driving tasks} - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - G06F 18/251: {of input or preprocessed data} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/764: using classification, e.g. of video objects - G06V 10/803: {of input or preprocessed data} - G06V 10/82: using neural networks - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G08G 1/0125: {Traffic data processing} - B60W 2420/403: Image sensing, e.g. optical camera - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract In various examples, live perception from sensors of a vehicle may be leveraged to detect and classify intersections in an environment of a vehicle in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as bounding box coordinates for intersections, intersection coverage maps corresponding to the bounding boxes, intersection attributes, distances to intersections, and/or distance coverage maps associated with the intersections. The outputs may be decoded and/or post-processed to determine final locations of, distances to, and/or attributes of the detected intersections. ## Claims ### Claim 1 A method comprising: determining, using one or more neural networks and based at least on sensor data representative of at least a portion of an intersection, location information associated with the intersection and distance information associated with the intersection; determining, based at least on the location information and the distance information, a location of the intersection within an environment; and causing a vehicle to perform one or more operations based at least on the location of the intersection. ### Claim 2 (depends on 1) The method of claim 1, wherein the location information includes a bounding shape associated with a portion of an image represented by the sensor data, the portion of the image depicting the at least the portion of the intersection. ### Claim 3 (depends on 1) The method of claim 1, further comprising: determining, using the one or more neural networks and based at least on the sensor data, a confidence that the location information corresponds to the intersection, wherein the determining of the location of the intersection within the environment is further based at least on the confidence. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining, using the one or more neural networks and based at least on the sensor data, one or more attributes associated with the intersection, wherein the causing the vehicle to perform the one or more operations is further based at least on the one or more attributes. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining, using the one or more neural networks and based at least on second sensor data representative of at least a portion of the intersection, second location information associated with the intersection, wherein the determining the location of the intersection within the environment is further based at least on the second location information. ### Claim 6 (depends on 1) The method of claim 1, wherein: the location information is associated with two-dimensional (2D) coordinates corresponding to an image represented by the sensor data; and the determining the location of the intersection within the environment comprises determining, based at least on the location information, the location of the intersection within the environment by at least converting the 2D coordinates to three-dimensional (3D) coordinates associated with the environment. ### Claim 7 (depends on 1) The method of claim 1, wherein: the sensor data is representative of a sensor representation; and the distance information indicates at least one or more distances to one or more points of the sensor representation. ### Claim 8 A system comprising: one or more processing units to: compute, based at least on one or more neural networks processing sensor data representative of at least a portion of an intersection located within an environment, data indicative of distance information associated with an intersection; determine, based at least on the data, a distance to the intersection within the environment; and cause a vehicle to perform one or more operations based at least on the distance. ### Claim 9 (depends on 8) The system of claim 8, wherein the one or more processing units are further to: determine, based at least on the one or more neural networks processing the sensor data, a bounding shape associated with at least a portion of an image represented by the sensor data, the portion of the image depicting the at least the portion of the intersection; and determine, based at least on the bounding shape, a location of the intersection within the environment, wherein the vehicle is further caused to perform the one or more operations based at least on the location of the intersection within the environment. ### Claim 10 (depends on 8) The system of claim 8, wherein: the sensor data represents an image depicting the at least the portion of the intersection; and the distance information indicates one or more distances to one or more points of the images that are associated with the at least the portion of the intersection. ### Claim 11 (depends on 8) The system of claim 8, wherein the one or more processing units are further to: determine a confidence associated with the intersection; and determine, based at least on the confidence, a location of the intersection within environment, wherein the vehicle is further caused to perform the one or more operations based at least on the location of the intersection within the environment. ### Claim 12 (depends on 8) The system of claim 8, wherein the one or more processing units are further to: compute, based at least on the one or more neural networks processing the sensor data, one or more attributes associated with the intersection, wherein the vehicle is further caused to perform the one or more operations based at least on the one or more attributes. ### Claim 13 (depends on 8) The system of claim 8, wherein the one or more processing units are further to: compute, based at least on the one or more neural networks processing second sensor data representative of at least another portion of the intersection, second data indicative of second information associated with the intersection, wherein the vehicle is further caused to perform the one or more operations based at least on the second information. ### Claim 14 (depends on 8) The system of claim 8, wherein: the distance information is associated with one or more two-dimensional (2D) coordinates corresponding to an image represented by the sensor data; the distance to the intersection within the environment is determined by at least converting the one or more 2D coordinates to one or more three-dimensional (3D) coordinates associated with the environment; and the vehicle is caused to perform the one or more operations based at least on the one or more 3D coordinates. ### Claim 15 (depends on 8) The system of claim 8, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing light transport simulation; a system for performing deep learning operations; a system implemented using a robot; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. ### Claim 16 A processor comprising: one or more processing units cause a vehicle to perform one or more operations based at least on one or more three-dimensional (3D) coordinates corresponding to an intersection within an environment, the one or more 3D coordinates being determined by converting one or more two-dimensional (2D) coordinates associated with location information that is computed based at least on one or more neural networks processing sensor data corresponding to at least a portion of the intersection. ### Claim 17 (depends on 16) The processor of claim 16, wherein the one or more 2D coordinates are associated with a bounding shape associated with a portion of an image represented by the sensor data, the portion of the image depicting the at least the portion of the intersection. ### Claim 18 (depends on 16) The processor of claim 16, wherein the one or more 2D coordinates are associated with a distance to the intersection within the environment. ### Claim 19 (depends on 16) The processor of claim 16, wherein the one or more processing units are further to: compute, based at least on the one or more neural networks processing the sensor data, one or more attributes associated with the intersection, wherein the vehicle is further caused to perform the one or more operations based at least on the one or more attributes. ### Claim 20 (depends on 16) The processor of claim 16, wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing light transport simulation; a system for performing deep learning operations; a system implemented using a robot; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.