# US 2020/0401884 A1 · Batch normalization layer fusion and quantization method for model inference in ai neural network engine

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401884 A1
- Document ID: US20200401884A1
- Status: Published application
- Kind: A1
- Application number: 16/450,716
- Filed: 2019-06-24
- Published: 2020-12-24
- Earliest claimed: 2019-06-24
- Assignees: Baidu USA LLC
- Applicants: Baidu USA LLC
- Inventors: MIN GUO
- Web page: https://patentgrep.com/patent/US20200401884A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/09: Supervised learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 3/045: Combinations of networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Batch normalization (BN) layer fusion and quantization method for model inference in artificial intelligence (AI) network engine are disclosed. A method for a neural network (NN) includes merging batch normalization (BN) layer parameters with NN layer parameters and computing merged BN layer and NN layer functions using the merged BN and NN layer parameters. A rectified linear unit (RELU) function can be merged with the BN and NN layer functions.

## Claims

### Claim 1

A method for training a neural network (NN), the method comprising:
for each of a plurality of NN layers of the NN,
merging batch normalization (BN) layer parameters with convolutional (CONV) layer parameters, wherein the BN layer parameters are associated with a BN layer of a corresponding NN layer and the NN layer parameters are associated with a CONV layer of the corresponding NN payer; and
forming a merged BN and CONV (BN/CONV) layer to compute merged BN layer and CONV layer functions using the merged BN and CONV layer parameters.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
merging a batch normalization transform function with a convolutional kernel computation function.

### Claim 3 (depends on 2)

The method of claim 2, wherein computing the merged BN layer and NN layer functions includes computing the merged batch normalization transform function and convolutional kernel computation function.

### Claim 4 (depends on 2)

The method of claim 2, wherein merging BN layer parameters with CONV layer parameters includes merging a batch normalization constant with weights of the convolutional kernel computation function.

### Claim 5 (depends on 2)

The method of claim 2, wherein merging BN layer parameters with CONV layer parameters includes merging batch normalization bias with a bias of the convolutional kernel computation function.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
merging a rectified linear unit (RELU) layer function with the merged BN layer and CNN layer functions to form a merged BN/CONV/RELU layer.

### Claim 7 (depends on 1)

The method of claim 1, further comprising storing the merged BN and CONV layer parameters as metadata in a metafile, which can be utilized subsequently by the merged BN/CONV layer during inference.

### Claim 8

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for training a neural network (NN), the operations comprising:
for each of a plurality of NN layers of the NN,
merging batch normalization (BN) layer parameters with convolutional (CONV) layer parameters, wherein the BN layer parameters are associated with a BN layer of a corresponding NN layer and the NN layer parameters are associated with a CONV layer of the corresponding NN payer; and
forming a merged BN and CONV (BN/CONV) layer to compute merged BN layer and CONV layer functions using the merged BN and CONV layer parameters.

### Claim 9 (depends on 8)

The machine-readable medium of claim 8, wherein the operations further comprise:
merging a batch normalization transform function with a convolutional kernel computation function.

### Claim 10 (depends on 9)

The machine-readable medium of claim 9, wherein computing the merged BN layer and NN layer functions includes computing the merged batch normalization transform function and convolutional kernel computation function.

### Claim 11 (depends on 9)

The machine-readable medium of claim 9, wherein merging BN layer parameters with CONV layer parameters includes merging a batch normalization constant with weights of the convolutional kernel computation function.

### Claim 12 (depends on 9)

The machine-readable medium of claim 9, wherein merging BN layer parameters with CONV layer parameters includes merging batch normalization bias with a bias of the convolutional kernel computation function.

### Claim 13 (depends on 8)

The machine-readable medium of claim 8, wherein the operations further comprise:
merging a rectified linear unit (RELU) layer function with the merged BN layer and CNN layer functions to form a merged BN/CONV/RELU layer.

### Claim 14 (depends on 8)

The machine-readable medium of claim 8, wherein the operations further comprise storing the merged BN and CONV layer parameters as metadata in a metafile, which can be utilized subsequently by the merged BN/CONV layer during inference.

### Claim 15

A data processor, comprising:
one or more memories to receive and store input data; and
a processing core coupled to the one or more memories to classify the input data using a neural network (NN) having a plurality of NN layers, wherein each of the NN layers includes a merged batch normalization (BN) transform and convolutional (CONV) kernel computation layer using a set of merged BN and CONV parameters.

### Claim 16 (depends on 15)

The data processor of claim 15, wherein the one or more memories store the merged BN and CONV parameters.

### Claim 17 (depends on 15)

The data processor of claim 15, wherein the processing core is configured to compute merged BN transform and CONV kernel computation functions.

### Claim 18 (depends on 17)

The data processor of claim 17, wherein the merged BN and CONV parameters include a merged BN constant with weights of a CONV kernel computation.

### Claim 19 (depends on 15)

The data processor of claim 15, wherein the merged BN and CONV parameters include a merged BN bias with a bias of a CONV kernel computation.

### Claim 20 (depends on 15)

The data processor of claim 15, wherein the processing core is trained to implement the NN with merged BN transform, CONV kernel computation, and rectified linear unit (RELU) layers.