Offline combination of convolutional/deconvolutional and batch-norm layers of convolutional neural network models for autonomous driving vehicles

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• receiving, at a neural network model generator, a first convolutional neural network (CNN) model as input to the neural network model generator, wherein the first CNN model is a fully trained CNN model;
• extracting, by the neural network model generator, a plurality of first groups of layers and a plurality of second groups of layers from the first convolutional neural network (CNN) model, each group of the plurality of first groups having a first convolutional layer, a first batch-norm layer, and a first scaling layer that is separate from the first batch-norm layer, and each group of the plurality of second groups having a first deconvolutional layer and a second batch-norm layer;
• for each group of the plurality of first groups,
• calculating, by the neural network model generator, a first scale vector and a first shift vector based on the first batch-norm layer, wherein the first scale vector and the first shift vector are calculated scalarly for each dimension of a multi-dimensional input of the first convolutional layer,
• calculating a second scale vector and a second shift vector based on the first scaling layer,
• combining the first scale vector and the second scale vector into a combined scale vector, and the first shift vector and the second shift vector into a combined shift vector, wherein each of the first scale vector and the first shift vector is calculated from the first batch-norm layer, and wherein each of the second scale vector and the second shift vector is calculated from the first scaling layer, and
• generating, by the neural network model generator, a second convolutional layer representing the corresponding group of the plurality of first groups based on the first convolutional layer, the combined scale vector, and the combined shift vector;
• for each group of the plurality of second groups,
• calculating a third scale vector and a third shift vector based on the second batch-norm layer, and
• generating a second deconvolutional layer representing the corresponding group of the plurality of second groups based on the third scale vector, the third shift vector, and the first deconvolutional layer;
• and
• generating, by the neural network model generator, a second CNN model based on the second convolutional layer corresponding to each of the plurality of first groups and the deconvolutional layer corresponding to each of the plurality of second groups, wherein the second CNN model is utilized subsequently to classify an object perceived by an autonomous driving vehicle (ADV).

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:
• receiving, at a neural network model generator, a first convolutional neural network (CNN) model as input to the neural network model generator, wherein the first CNN model is a fully trained CNN model,
• extracting a plurality of first groups of layers and a plurality of second groups of layers from the first convolutional neural network (CNN) model, each group of the plurality of first groups having a first convolutional layer, a first batch-norm layer, and a first scaling layer that is separate from the first batch-norm layer, and each group of the plurality of second groups having a first deconvolutional layer and a second batch-norm layer,
• for each group of the plurality of first groups,
• calculating a first scale vector and a first shift vector based on the first batch-norm layer, wherein the first scale vector and the first shift vector are calculated scalarly for each dimension of a multi-dimensional input of the first convolutional layer,
• calculating a second scale vector and a second shift vector based on the first scaling layer,
• combining the first scale vector and the second scale vector into a combined scale vector, and the first shift vector and the second shift vector into a combined shift vector, wherein each of the first scale vector and the first shift vector is calculated from the first batch-norm layer, and wherein each of the second scale vector and the second shift vector is calculated from the first scaling layer, and
• generating a second convolutional layer representing the corresponding group of the plurality of first groups based on the first convolutional layer, the combined scale vector, and the combined shift vector,
• for each group of the plurality of second groups,
• calculating a third scale vector and a third shift vector based on the second batch-norm layer, and
• generating a second deconvolutional layer representing the corresponding group of the plurality of second groups based on the third scale vector, the third shift vector, and the first deconvolutional layer,
• generating a second CNN model based on the second convolutional layer corresponding to each of the plurality of first groups and the deconvolutional layer corresponding to each of the plurality of second groups, and
• classifying an object perceived by an autonomous driving vehicle (ADV) using the second CNN model.