# US 2019/0171936 A1 · Progressive Modification of Neural Networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171936 A1
- Document ID: US20190171936A1
- Status: Published application
- Kind: A1
- Application number: 16/252,336
- Filed: 2019-01-18
- Published: 2019-06-06
- Earliest claimed: 2017-10-26
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Tero Tapani Karras; Timo Oskari Aila; Samuli Matias Laine; Jaakko T. Lehtinen; Janne Hellsten
- Web page: https://patentgrep.com/patent/US20190171936A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/048: Activation functions
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A neural network learns a particular task by being shown many examples. In one scenario, a neural network may be trained to label an image, such as cat, dog, bicycle, chair, etc. In other scenario, a neural network may be trained to remove noise from videos or identify specific objects within images, such as human faces, bicycles, etc. Rather than training a complex neural network having a predetermined topology of features and interconnections between the features to learn the task, the topology of the neural network is modified as the neural network is trained for the task, eventually evolving to match the predetermined topology of the complex neural network. In the beginning the neural network learns large-scale details for the task (bicycles have two wheels) and later, as the neural network becomes more complex, learns smaller details (the wheels have spokes).

## Claims

### Claim 1

A computer-implemented method, comprising:
training, for a first time duration, a neural network using a training dataset including training input data and training output data, wherein a topology of the neural network comprises features and interconnections between the features, the training comprising:
modifying at least one of the training input data and the training output data to produce first modified training data;
processing the training input data or the first modified training input data by the neural network according to parameters of the neural network to produce network output data; and
updating the parameters to reduce differences between the network output data and the training output data or the first modified training output data;
modifying the topology of the neural network to produce a modified neural network;
modifying at least one of the training input data and the training output data to be consistent with the modified neural network to produce second modified training data that is different compared with the first modified training data; and
training the modified neural network for a second time duration using the second modified training data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein modifying the topology changes a processing capacity of the neural network.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the topology is modified by adding at least one layer in the neural network.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the topology is modified by removing at least one layer from the neural network.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein at least one of the training input data and the training output data is downsampled to produce the first modified training data.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein modifying the training output data comprises increasing or decreasing a density of the training output data.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the training output data is image data and modifying the training output data comprises decreasing a pixel resolution of the training output data.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein modifying the training input data comprises increasing or decreasing a density of the training input data.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the training input data is image data and modifying the training input data comprises decreasing a pixel resolution of the training input data.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising, during the second time duration, smoothly modifying the topology.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein first intermediate values generated by the neural network using the topology are interpolated with second intermediate values generated by the neural network using the modified topology.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, further comprising, during the second time duration, interpolating between the first modified training data and the second modified training data.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is a classification neural network.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is an autoencoder neural network.

### Claim 15 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is an object detection neural network and the network output data comprises zero or more labels for zero or more locations in the input data.

### Claim 16 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is a segmentation neural network.

### Claim 17 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is a recurrent neural network.

### Claim 18 (depends on 1)

The computer-implemented method of claim 1, wherein the topology is modified by adding one layer to an encoder portion of the neural network and one layer to a decoder portion of the neural network.

### Claim 19

A system, comprising:
a neural network comprising features and interconnections between the features, wherein
the neural network is trained using a training dataset including training input data and training output data, the training comprising:
modifying at least one of the training input data and the training output data to produce first modified training data for a first time duration;
processing the training input data or the first modified training input data by the neural network according to parameters of the neural network to produce network output data for the first time duration;
updating the parameters to reduce differences between the network output data and the training output data or the first modified training output data; for the first time duration;
after the first time duration, modifying the topology of the neural network to produce a modified neural network;
modifying at least one of the training input data and the training output data to be consistent with the modified neural network to produce second modified training data that is different compared with the first modified training data; and
training the modified neural network for a second time duration using the second modified training data.

### Claim 20

A non-transitory computer-readable media storing computer instructions for training a neural network that, when executed by one or more processors, cause the one or more processors to perform the steps of:
training, for a first time duration, the neural network using a training dataset including training input data and training output data, wherein a topology of the neural network comprises features and interconnections between the features, the training comprising:
modifying at least one of the training input data and the training output data to produce first modified training data;
processing the training input data or the first modified training input data by the neural network according to parameters of the neural network to produce network output data; and
updating the parameters to reduce differences between the network output data and the training output data or the first modified training output data;
modifying the topology of the neural network to produce a modified neural network;
modifying at least one of the training input data and the training output data to be consistent with the modified neural network to produce second modified training data that is different compared with the first modified training data; and
training the modified neural network for a second time duration using the second modified training data.
