Vision-based machine learning model for aggregation of static objects and systems for autonomous driving

A system comprising one or more processors and non-transitory computer storage media storing instructions that when executed by the one or more processors, cause the processors to perform operations, wherein the system is included in an autonomous or semi-autonomous vehicle, and wherein the operations comprise:

• obtaining images from a multitude of image sensors positioned about a vehicle;
• determining features associated with the images, wherein the features are output based on a forward pass through a machine learning model;
• projecting, based on the machine learning model, the features into a vector space associated with a birds-eye view;
• aggregating, based on a video module, the projected features with other projected features associated with prior images;
• and
• outputting, based on a plurality of heads of the machine learning model, images depicting static objects in the birds-eye view.

Non-transitory computer storage media storing instructions that when executed by a system of one or more processors which are included in an autonomous or semi-autonomous vehicle, cause the system to perform operations comprising:

• obtaining images from a multitude of image sensors positioned about a vehicle;
• determining features associated with the images, wherein the features are output based on a forward pass through a machine learning model;
• projecting, based on the machine learning model, the features into a vector space associated with a birds-eye view;
• aggregating, based on a video module, the projected features with other projected features associated with prior images;
• and
• outputting, based on a plurality of heads of the machine learning model, images depicting static objects in the birds-eye view.