Particle-Based Hazard Detection for Autonomous Machine Applications

A system comprising:

• an image sensor;
• a depth sensor;
• one or more processing units;
• and
• one or more memory units storing instructions that, when executed by the one or more processing units, cause the one or more processing units to execute operations comprising:
• determining, using one or more first particle locations of one or more particles corresponding to an object at a first time and motion of an ego machine between the first time and a second time, one or more second particle locations of the particles corresponding to an object at the second time, the particles having one or more associated first confidences indicative of whether the object is located at the one or more second particle locations, the one or more first particle locations being determined using first sensor data generated using the image sensor;
• correlating the first sensor data generated using the image sensor with second sensor data generated using the depth sensor;
• based at least in part on the correlating, comparing one or more depth detections from the second sensor data to a bounding shape corresponding to the object as represented by the first sensor data;
• determining a set of the one or more depth detections that is within the bounding shape;
• updating the one or more first confidences to one or more second confidences based at least in part on the set of the one or more depth detections;
• and
• determining a location of the object using the one or more second confidences.

A method comprising:

• detecting an object in an environment based at least on first sensor data representative of an image generated at a first time using an image sensor of an ego-machine;
• based at least in part on the detecting the object, determining a first probability distribution corresponding to a location of the object in three-dimensional (3D) world space;
• generating a set of particles at one or more first locations within the first probability distribution having one or more first confidence values indicative of whether the object exists at the one or more first locations;
• generating, at a second time after the first time, the set of particles at one or more second locations based at least in part on the one or more first locations and motion of the ego-machine between the first time and the second time, the set of particles at the one or more second locations including one or more second confidence values indicative of whether the object exists at the one or more second locations;
• correlating the first sensor data with second sensor data representative of one or more depth detections generated using a depth sensor of the ego-machine at a third time;
• based at least in part on the correlating, updating the one or more second confidence values of the set of particles to one or more third confidence values indicative of whether the object exists at the one or more second locations;
• and
• filtering out one or more of the set of particles having one or more associated third confidence values that are less than a threshold confidence value to determine a subset of the particles.