# US 12,182,711 B2 · Generation of neural network containing middle layer background

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,711 B2
- Document ID: US12182711B2
- Status: Granted
- Kind: B2
- Application number: 16/440,064
- Filed: 2019-06-13
- Published: 2019-09-26
- Granted: 2024-12-31
- Earliest claimed: 2018-01-19
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Takeshi Inagaki
- Web page: https://patentgrep.com/patent/US12182711B2

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24133: {Distances to prototypes}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/0499: Feedforward networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06V 10/82: using neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method, computer program product and system for generating a neural network. Initial neural networks are prepared, each of which includes an input layer containing one or more input nodes, a middle layer containing one or more middle nodes, and an output layer containing one or more output nodes. A new neural network is generated that includes a new middle layer containing one or more middle nodes based on the middle nodes of the middle layers of the initial neural networks.

## Claims

### Claim 1

A method for generating a neural network, the method comprising:
preparing a plurality of initial neural networks, each of which comprises an input layer containing one or more input nodes, a middle layer containing one or more middle nodes, and an output layer containing one or more output nodes;
performing supervised training of the output layer of each of the plurality of initial neural networks using a set of training data to prepare the plurality of initial neural networks; and
generating a new neural network comprising a new middle layer containing one or more middle nodes based on the middle nodes of the middle layers of the plurality of initial neural networks.

### Claim 2 (depends on 1)

The method as recited in claim 1, wherein the plurality of initial neural networks comprises N initial neural networks, N being an integer larger than 1, and wherein the generating of the new neural network comprises:
selecting one or more of the middle nodes of the N initial neural networks; and
including the selected one or more middle nodes in the new middle layer of the new neural network.

### Claim 3 (depends on 2)

The method as recited in claim 2, wherein the selecting of the one or more of the middle nodes of the N initial neural networks comprises:
obtaining K different sets of training data, K being an integer more than 1;
performing supervised training on the N initial neural networks with each of the K different sets of training data to obtain K training results for each of the N initial neural networks; and
selecting at least one of the middle nodes in the middle layer of the N initial neural networks using the K training results, such that selected middle nodes contribute to an output from the output layer to a greater degree than non-selected middle nodes.

### Claim 4 (depends on 2)

The method as recited in claim 2, wherein the middle layer of each of the plurality of initial neural network comprises L middle nodes, L being an integer larger than 2, and wherein the number of the middle nodes in the new middle layer is equal to or less than L.

### Claim 5 (depends on 2)

The method as recited in claim 2, wherein the generating of the new neural network further comprises:
performing unsupervised training on the selected middle nodes, the unsupervised training comprising biasing the middle nodes such that certain middle nodes are avoided.

### Claim 6 (depends on 1)

The method as recited in claim 1, wherein the preparing of the plurality of initial neural networks comprises:
obtaining N initial conditions, N being an integer larger than 1, each condition corresponding to one of the initial neural networks; and
performing unsupervised training of the middle layer of each initial neural network using the corresponding initial condition.

### Claim 7 (depends on 1)

The method as recited in claim 1, wherein the preparing of the plurality of initial neural networks comprises:
obtaining M initial conditions, M being an integer larger than 2, each condition corresponding to one of M candidate neural networks;
performing unsupervised training of the middle layer of each candidate neural network using the corresponding initial condition;
performing supervised training of the output layer of each candidate neural network using a set of training data;
evaluating a performance of each candidate neural network; and
selecting N initial neural networks from among the M candidate neural networks using the performances, N being an integer larger than 1 and smaller than M.