Data augmentation using computer simulated objects for autonomous control systems

A computer-implemented method of training an object detection model configured to receive an image of a vehicle and output a set of 3-D bounding box coordinates of the vehicle in the image, comprising:

• obtaining a training dataset of images that represent a set of computer simulated vehicles with a known set of characteristics, the characteristics including geometric information of the vehicles in the images;
• obtaining a training set of coordinates for the images that represent 3-D bounding boxes of the computer simulated vehicles, the 3-D bounding boxes enclosing an outer boundary of the computer simulated vehicles, and the training set of coordinates for the 3-D bounding boxes determined based on the geometric information of the computer simulated vehicles;
• and
• training a set of parameters of the object detection model using the training dataset of images, the training comprising repeatedly performing iterations of:
• generating a set of estimated coordinates by applying the object detection model with an estimated set of parameters to the training dataset of images,
• determining a loss function indicating a difference between the set of estimated coordinates and the training set of coordinates for the computer simulated vehicles, and
• updating the set of parameters of the object detection model to reduce the loss function.

A non-transitory computer-readable storage medium storing computer program instructions executable to perform operations for training a machine-learned transformation model configured to receive an image of a vehicle from a reference viewpoint and output a target image of the vehicle from a target viewpoint different from the reference viewpoint, the operations comprising:

• obtaining a training dataset of images that represent a set of computer simulated vehicles with a known set of characteristics, the training dataset including a training set of reference images that capture the computer simulated vehicles from one or more reference viewpoints and a corresponding training set of target images that capture the computer simulated vehicles from one or more target viewpoints different from the one or more reference viewpoints;
• and
• training a set of parameters of the transformation model using the training dataset of images, the training comprising repeatedly performing iterations of:
• generating a set of estimated target images by applying the transformation model with an estimated set of parameters to the training set of reference images,
• determining a loss function indicating a difference between the set of estimated target images and the training set of target images, and
• updating the set of parameters of the transformation model to reduce the loss function.

A non-transitory computer-readable storage medium storing computer program instructions executable to perform operations for training an object detection model configured to receive an image of a vehicle and output a set of 3-D bounding box coordinates of the vehicle in the image, comprising:

• obtaining a training dataset of images that represent a set of computer simulated vehicles with a known set of characteristics, the characteristics including geometric information of the vehicles in the images;
• obtaining a training set of coordinates for the images that represent 3-D bounding boxes of the computer simulated vehicles, the 3-D bounding boxes enclosing an outer boundary of the computer simulated vehicles, and the training set of coordinates for the 3-D bounding boxes determined based on the geometric information of the computer simulated vehicles;
• and
• training a set of parameters of the object detection model using the training dataset of images, the training comprising repeatedly performing iterations of:
• generating a set of estimated coordinates by applying the object detection model with an estimated set of parameters to the training dataset of images,
• determining a loss function indicating a difference between the set of estimated coordinates and the training set of coordinates for the computer simulated vehicles, and
• updating the set of parameters of the object detection model to reduce the loss function.

An autonomous control vehicle including a machine-learned transformation model stored on a computer-readable medium manufactured by the process of:

• obtaining a training dataset of images that represent a set of computer simulated vehicles with a known set of characteristics, the training dataset including a training set of reference images that capture the computer simulated vehicles from one or more reference viewpoints and a corresponding training set of target images that capture the computer simulated vehicles from one or more target viewpoints different from the one or more reference viewpoints;
• and
• training a set of parameters of the transformation model using the training dataset of images, the training comprising repeatedly performing iterations of:
• generating a set of estimated target images by applying the transformation model with an estimated set of parameters to the training set of reference images,
• determining a loss function indicating a difference between the set of estimated target images and the training set of target images, and
• updating the set of parameters of the transformation model to reduce the loss function.

An autonomous control vehicle including a machine-learned a machine-learned object detection model stored on a computer-readable medium manufactured by the process of:

• obtaining a training dataset of images that represent a set of computer simulated vehicles with a known set of characteristics, the characteristics including geometric information of the vehicles in the images;
• obtaining a training set of coordinates for the images that represent 3-D bounding boxes of the computer simulated vehicles, the 3-D bounding boxes enclosing an outer boundary of the computer simulated vehicles, and the training set of coordinates for the 3-D bounding boxes determined based on the geometric information of the computer simulated vehicles;
• and
• training a set of parameters of the object detection model using the training dataset of images, the training comprising repeatedly performing iterations of:
• generating a set of estimated coordinates by applying the object detection model with an estimated set of parameters to the training dataset of images,
• determining a loss function indicating a difference between the set of estimated coordinates and the training set of coordinates for the computer simulated vehicles, and
• updating the set of parameters of the object detection model to reduce the loss function.