Top-down view object detection and tracking

A system comprising:

• one or more processors;
• and
• a memory storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• receiving first sensor data and second sensor data;
• inputting the first sensor data to a first perception pipeline and inputting the second sensor data to a second perception pipeline;
• receiving a first output from the first perception pipeline based at least in part on the first sensor data and a second output from the second perception pipeline, the first output and the second output identifying an object in an environment;
• receiving a previous track associated with the object in the environment, the previous track identifying at least one of an estimated previous position of the object, a previous region of interest, or a previous velocity of the object;
• inputting the first output, the second output, and at least part of the previous track into a machine-learning (ML) model;
• receiving, from the ML model, a data structure comprising a region of interest, object classification, and a pose associated with the object, the pose indicating at least one of a position or a yaw associated with the object;
• determining an updated track associated with the object based at least in part on the data structure, a current position, and at least one of the region of interest or the yaw associated with the object;
• and
• updating, based at least in part on the data structure, one or more previous tracks by retiring the one or more previous tracks, wherein retiring the one or more previous tracks comprises indicating that the object associated with the one or more previous tracks has been occluded for a threshold amount of time.

A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving a first output associated with a first process and a second output associated with a second process;
• receiving a previous track associated with an object in an environment, the previous track identifying at least one of an estimated previous position of the object, a previous region of interest, or a previous velocity of the object;
• generating a multi-channel data structure based at least in part on the first output, the second output, and at least part of the previous track;
• inputting the multi-channel data structure into a machine-learning (ML) model;
• receiving, from the ML model, a data structure comprising a region of interest, object classification, and a pose associated with the object, the pose indicating at least one of a position or a yaw associated with the object;
• determining an updated track associated with the object based at least in part on the data structure, the updated track comprising at least a current position, and at least one of the region of interest or the yaw associated with the object;
• and
• updating, based at least in part on the data structure, one or more previous tracks by retiring the one or more previous tracks, wherein retiring the one or more previous tracks comprises indicating that the object associated with the one or more previous tracks has been occluded for a threshold amount of time.