# US 2021/0174175 A1 · Building of Custom Convolution Filter for a Neural Network Using an Automated Evolutionary Process

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174175 A1
- Document ID: US20210174175A1
- Status: Published application
- Kind: A1
- Application number: 16/705,488
- Filed: 2019-12-06
- Published: 2021-06-10
- Earliest claimed: 2019-12-06
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Mihir Choudhury; Atin Sood; Ruchir Puri
- Web page: https://patentgrep.com/patent/US20210174175A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/063: using electronic means
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Mechanisms are provided for synthesizing a computer implemented neural network. An initially trained neural network is received and modified by introducing a new hidden layer of neurons and new connections that connect the new hidden layer of neurons to an output layer and a previous layer of neurons previously directly connected to the output layer of neurons to generate a modified neural network. The modified neural network is trained through one or more epochs of machine learning to generate modified weight values for the new connections and the new connections are pruned based on the modified weight values to remove a subset of the new connections and leaving remaining connections in the modified neural network. A merge operation is performed on the remaining connections in the modified neural network to generate a custom convolution filter and modified neural network. The modified neural network is then retrained for deployment.

## Claims

### Claim 1

A method, in a data processing system, for synthesizing a computer implemented neural network with a custom convolution filter, the method comprising:
receiving an initially trained neural network comprising an initial set of weight values associated with connections in the initially trained neural network;
modifying the initially trained neural network by introducing a first new hidden layer of neurons and new connections that connect the first new hidden layer of neurons to an output layer of neurons of the initially trained neural network and a previous layer of neurons previously directly connected to the output layer of neurons to generate a modified neural network;
training the modified neural network through one or more epochs of machine learning to generate modified weight values for the new connections;
pruning the new connections based on the modified weight values to remove a subset of the new connections and leaving remaining connections in the modified neural network;
performing a merge operation on the remaining connections in the modified neural network to generate a custom convolution filter and modified neural network comprising the custom convolution filter; and
retraining the modified neural network with one or more additional epochs of machine learning to generate a trained neural network, having the custom convolution filter, for deployment.

### Claim 2 (depends on 1)

The method of claim 1, further comprising repeating the modifying of the initially trained neural network, training the modified neural network, pruning the new connections based on the modified weight values, and performing the merge operation for a second new hidden layer of neurons connected to the first new hidden layer.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
selecting a subset of neurons of the initially trained neural network as the previous layer of neurons.

### Claim 4 (depends on 1)

The method of claim 1, wherein modifying the initially trained neural network further comprises setting weights of the new connections to values which do not perturb output values of output neurons of the output layer of neurons more than a predetermined threshold amount.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first hidden layer of neurons comprises a same number of neurons as neurons in the previous layer of neurons, and wherein weights and biases for the new connections are set to be the same as the weights and biases of connections between the previous layer of neurons and the output layer of neurons.

### Claim 6 (depends on 1)

The method of claim 1, wherein pruning the new connections comprises selecting the remaining connections as a predetermined number of the new connections that have a relatively larger amount of change in associated weights, as a result of the training, than other ones of the new connections.

### Claim 7 (depends on 1)

The method of claim 1, wherein pruning the new connections comprises selecting the remaining connections as a predetermined number of the new connections having a relatively larger associated weight, as a result of the training, than other ones of the new connections.

### Claim 8 (depends on 1)

The method of claim 1, wherein pruning the new connections is performed in an iterative manner wherein in each iteration of the pruning, a fixed number of lowest weighted new connections are removed and the training is repeated for a fixed number of epochs.

### Claim 9 (depends on 1)

The method of claim 1, wherein the merge operation performed on the remaining connections in the modified neural network comprises:
generating a predetermined number of weight buckets;
determining a measure of similarity or dissimilarity between weight vectors of neurons in the first new hidden layer;
binning neurons into weight buckets of the predetermined number of weight buckets based on the measure of similarity or dissimilarity; and
for each weight bucket, merging connections associated with neurons in the weight bucket together to generate a single merged neuron and corresponding merged set of connections.

### Claim 10 (depends on 1)

The method of claim 1, wherein the method is executed by a graphics processing unit (GPU) of the data processing system utilizing acceleration capabilities of the GPU.

### Claim 11

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to synthesize a computer implemented neural network with a custom convolution filter, at least by:
receiving an initially trained neural network comprising an initial set of weight values associated with connections in the initially trained neural network;
modifying the initially trained neural network by introducing a first new hidden layer of neurons and new connections that connect the first new hidden layer of neurons to an output layer of neurons of the initially trained neural network and a previous layer of neurons previously directly connected to the output layer of neurons to generate a modified neural network;
training the modified neural network through one or more epochs of machine learning to generate modified weight values for the new connections;
pruning the new connections based on the modified weight values to remove a subset of the new connections and leaving remaining connections in the modified neural network;
performing a merge operation on the remaining connections in the modified neural network to generate a custom convolution filter and modified neural network comprising the custom convolution filter; and
retraining the modified neural network with one or more additional epochs of machine learning to generate a trained neural network, having the custom convolution filter, for deployment.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the computing device to synthesize the computer implemented neural network with a custom convolution filter, at least by repeating the modifying of the initially trained neural network, training the modified neural network, pruning the new connections based on the modified weight values, and performing the merge operation for a second new hidden layer of neurons connected to the first new hidden layer.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the computing device to synthesize the computer implemented neural network with a custom convolution filter, at least by selecting a subset of neurons of the initially trained neural network as the previous layer of neurons.

### Claim 14 (depends on 11)

The computer program product of claim 11, wherein modifying the initially trained neural network further comprises setting weights of the new connections to values which do not perturb output values of output neurons of the output layer of neurons more than a predetermined threshold amount.

### Claim 15 (depends on 11)

The computer program product of claim 11, wherein the first hidden layer of neurons comprises a same number of neurons as neurons in the previous layer of neurons, and wherein weights and biases for the new connections are set to be the same as the weights and biases of connections between the previous layer of neurons and the output layer of neurons.

### Claim 16 (depends on 11)

The computer program product of claim 11, wherein pruning the new connections comprises selecting the remaining connections as a predetermined number of the new connections that have a relatively larger amount of change in associated weights, as a result of the training, than other ones of the new connections.

### Claim 17 (depends on 11)

The computer program product of claim 11, wherein pruning the new connections comprises selecting the remaining connections as a predetermined number of the new connections having a relatively larger associated weight, as a result of the training, than other ones of the new connections.

### Claim 18 (depends on 11)

The computer program product of claim 11, wherein pruning the new connections is performed in an iterative manner wherein in each iteration of the pruning, a fixed number of lowest weighted new connections are removed and the training is repeated for a fixed number of epochs.

### Claim 19 (depends on 11)

The computer program product of claim 11, wherein the merge operation performed on the remaining connections in the modified neural network comprises:
generating a predetermined number of weight buckets;
determining a measure of similarity or dissimilarity between weight vectors of neurons in the first new hidden layer;
binning neurons into weight buckets of the predetermined number of weight buckets based on the measure of similarity or dissimilarity; and
for each weight bucket, merging connections associated with neurons in the weight bucket together to generate a single merged neuron and corresponding merged set of connections.

### Claim 20

An apparatus comprising:
a processor; and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:
synthesize a computer implemented neural network with a custom convolution filter, at least by:
receiving an initially trained neural network comprising an initial set of weight values associated with connections in the initially trained neural network;
modifying the initially trained neural network by introducing a first new hidden layer of neurons and new connections that connect the first new hidden layer of neurons to an output layer of neurons of the initially trained neural network and a previous layer of neurons previously directly connected to the output layer of neurons to generate a modified neural network;
training the modified neural network through one or more epochs of machine learning to generate modified weight values for the new connections;
pruning the new connections based on the modified weight values to remove a subset of the new connections and leaving remaining connections in the modified neural network;
performing a merge operation on the remaining connections in the modified neural network to generate a custom convolution filter and modified neural network comprising the custom convolution filter; and
retraining the modified neural network with one or more additional epochs of machine learning to generate a trained neural network, having the custom convolution filter, for deployment.
