Deep learning model embodiments and training embodiments for faster training
AssigneeBaidu USA LLC
InventorsBaopu Li, Zhiyu Cheng, Yingze Bao
Life of this patent
Estimating the term…
Abstract
Presented herein are embodiments of a training deep learning models. In one or more embodiments, a compact deep learning model comprises fewer layers, which require fewer floating-point operations (FLOPs). Presented herein are also embodiments of a new learning rate function, which can adaptively change the learning rate between two linear functions. In one or more embodiments, combinations of half-precision floating point format training together with larger batch size in the training process may also be employed to aid the training process.
Claims20
A computer-implemented method for training an image classification model, the method comprising:
- forming one or more batches comprising images and their corresponding labels, the images and their corresponding labels being selected from one or more training datasets in which each image has a corresponding label:
- repeating, for each training epoch until a stop condition is reached, a set of steps comprising:
- inputting a batch into the image classification model, the image classification model comprising:
- a convolution module comprising a convolution with a set of filters, a batch normalization operation, and an activation operation;
- a first residual module comprising at least two convolution modules separated by a max pooling layer, in which each convolution module has its own set of filters;
- a second residual module comprising at least two convolution modules separated by a max pooling layer, in which each convolution module has its own set of filters;
- and
- a fully connected layer that receives an input obtained from an output of the second residual module;
- determining a loss for the image classification model given the predicted output for the batch;
- and
- updating one or more parameters of the image classification model using the loss.
A system for training an image classification model, the system comprising:
- one or more processors;
- and
- a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
- forming one or more batches comprising images and their corresponding labels, the images and their corresponding labels being selected from one or more training datasets in which each image has a corresponding label:
- repeating, for each training epoch until a stop condition is reached, a set of steps comprising:
- inputting a batch into the image classification model, the image classification model comprising:
- a convolution module comprising a convolution with a set of filters, a batch normalization operation, and an activation operation;
- a first residual module comprising at least two convolution modules separated by a max pooling layer, in which each convolution module has its own set of filters;
- a second residual module comprising at least two convolution modules separated by a max pooling layer, in which each convolution module has its own set of filters;
- and
- a fully connected layer that receives an input obtained from an output of the second residual module;
- determining a loss for the image classification model given the predicted output for the batch;
- and
- updating one or more parameters of the image classification model using the loss.
A computer-implemented method for classifying an image, the method comprising:
- inputting an input image into a classification model, the classification model comprising:
- a convolution module comprising a convolution with a set of filters, a batch normalization operation, and an activation operation;
- a first residual module comprising at least two convolution modules each with its own set of filters separated by a max pooling layer;
- a second residual module comprising at least two convolution modules separated by a max pooling layer;
- and
- a fully connected layer;
- and
- outputting a classification label for the input image.
Dependencies are inferred from the claim text.