Machine-learned model training for pedestrian attribute and gesture detection

A system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
• receiving first image data captured by an image sensor associated with a vehicle, the first image data representing a pedestrian that is proximate the vehicle at a first time;
• receiving second image data captured by the image sensor, the second image data representing the pedestrian at a second time after the first time;
• receiving a first indication of a gesture performed by a body part of the pedestrian, the gesture determined based at least in part on the first image data and the second image data;
• receiving a second indication of an attribute associated with the pedestrian, the attribute determined based at least in part on the first image data or the second image data;
• inputting the first image data and the second image data into a machine-learned model;
• receiving, from the machine-learned model, an output indicating a predicted gesture performed by the body part of the pedestrian and a predicted attribute associated with the pedestrian;
• determining a first difference between the gesture and the predicted gesture;
• determining a second difference between the attribute and the predicted attribute;
• and
• altering a parameter of the machine-learned model to minimize at least one of the first difference or the second difference to obtain a trained machine-learned model that is configured to determine the gesture of the pedestrian and the attribute associated with the pedestrian.