Batch normalization layer fusion and quantization method for model inference in ai neural network engine
AssigneeBaidu USA LLC
InventorMIN GUO
Life of this patent
Estimating the term…
Abstract
Batch normalization (BN) layer fusion and quantization method for model inference in artificial intelligence (AI) network engine are disclosed. A method for a neural network (NN) includes merging batch normalization (BN) layer parameters with NN layer parameters and computing merged BN layer and NN layer functions using the merged BN and NN layer parameters. A rectified linear unit (RELU) function can be merged with the BN and NN layer functions.
Claims20
A method for training a neural network (NN), the method comprising:
- for each of a plurality of NN layers of the NN,
- merging batch normalization (BN) layer parameters with convolutional (CONV) layer parameters, wherein the BN layer parameters are associated with a BN layer of a corresponding NN layer and the NN layer parameters are associated with a CONV layer of the corresponding NN payer;
- and
- forming a merged BN and CONV (BN/CONV) layer to compute merged BN layer and CONV layer functions using the merged BN and CONV layer parameters.
A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for training a neural network (NN), the operations comprising:
- for each of a plurality of NN layers of the NN,
- merging batch normalization (BN) layer parameters with convolutional (CONV) layer parameters, wherein the BN layer parameters are associated with a BN layer of a corresponding NN layer and the NN layer parameters are associated with a CONV layer of the corresponding NN payer;
- and
- forming a merged BN and CONV (BN/CONV) layer to compute merged BN layer and CONV layer functions using the merged BN and CONV layer parameters.
A data processor, comprising:
- one or more memories to receive and store input data;
- and
- a processing core coupled to the one or more memories to classify the input data using a neural network (NN) having a plurality of NN layers, wherein each of the NN layers includes a merged batch normalization (BN) transform and convolutional (CONV) kernel computation layer using a set of merged BN and CONV parameters.
Dependencies are inferred from the claim text.