Neural Network Training Method Using Semi-Pseudo-Labels
AssigneeAImotive Kft.
InventorsDániel Ákos Kozma, Tamás Matuszka, Ákos Utasi
Life of this patent
Estimating the term…
Abstract
The present invention provides a method for training a complex machine learning model based on a dataset that comprises datapoints and ground truth labels for at least some of the datapoints of the dataset, wherein the datapoints comprise images, the method comprising: using a reduced-complexity machine learning model to predict pseudo labels for the dataset, and training the complex machine learning model using an extended dataset that comprises the ground truth labels and the pseudo labels as output.
Claims15
A method for training a complex machine learning model based on a dataset that comprises datapoints and ground truth labels for at least some of the datapoints of the dataset, wherein the datapoints comprise images, the method comprising:
- using (210) a reduced-complexity machine learning model to predict pseudo labels for the dataset, and
- training (220) the complex machine learning model using an extended dataset that comprises the ground truth labels and the pseudo labels as output.
The method of one of the previous claims, wherein the ground truth labels are labels of objects within an annotated area (110a, 110b, 110c) of the dataset and wherein the dataset comprises unlabelled datapoints for objects that are outside the annotated area of the dataset, wherein preferably the annotated area corresponds to a near-field (130) of a camera and a non-annotated area (112a, 112b, 112c) corresponds to a far-field (132) of the camera.
The method of one of the previous claims, further comprising:
- obtaining 3D data, and
- extending the dataset with virtual images based on a virtual camera virtually capturing the 3D data,
- wherein the obtaining the images comprises moving a position of the virtual camera to obtain shifted and/or zoomed images from a moved camera position.
The method of one of the previous claims, further comprising
- an initial step of detecting, based on the images, objects for which pseudo labels need to be predicted, and/or
- performing deduplication to remove pseudo labels for which the dataset already comprises ground truth labels.
The method of one of the previous claims, wherein the training the complex machine learning model comprises using a loss function that does not penalize errors contribu-tions that correspond to 3D properties where no ground truth label is available, wherein preferably a Boolean flag is used to indicate in the extended dataset whether a label is a ground truth label or a predicted pseudo label.
The method of one of the previous claims, further comprising using the complex machine learning model for detecting objects in autonomous driving, wherein the labels comprise object classes that preferably comprise one or more of vehicles, two-wheelers, pedestrians, traffic signs, and traffic lights.
The method of one of the previous claims, wherein the complex machine learning model is used to predict a center point of a 3D cuboid based on a prediction of the 2D projection of the 3D cuboid and a predicted depth.
The method of one of the previous claims, wherein the complex machine learning model is a multi task machine learning model that is configured to solve a first task and a second task in parallel, and wherein the reduced-complexity machine learning model is configured to solve the first task, wherein preferably the first task comprises a prediction of a 2D bounding box and the second task comprises a prediction of a 3D bounding box.
The method of one of the previous claims, wherein training of the complex machine learning model comprises using a loss terms that comprises one or more of:
- a projection loss term that is based on a 3D projection of 3D cuboid,
- a disparity loss term that is based on a disparity between a centre point of a 2D projected 3D bounding box on left and right stereo images, and
- a width loss term that is based on a difference between a width of a 2D projection of 3D bounding boxes on left and right stereo images.
An apparatus, configured to carry out the method of one of the previous claims.
Dependencies are inferred from the claim text.