# US 7,613,665 B2 · Ensembles of neural networks with different input sets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,613,665 B2
- Document ID: US7613665B2
- Status: Granted
- Kind: B2
- Application number: 11/165,892
- Filed: 2005-06-24
- Published: 2007-01-11
- Granted: 2009-11-03
- Earliest claimed: 2005-06-24
- Assignees: Halliburton Energy Services, Inc.
- Inventors: Dingding Chen; John A. Quirein; Harry D. Smith, Jr.; Syed Hamid; Jeffery L. Grable
- Web page: https://patentgrep.com/patent/US7613665B2

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods of creating and using robust neural network ensembles are disclosed. Some embodiments take the form of computer-based methods that comprise receiving a set of available inputs; receiving training data; training at least one neural network for each of at least two different subsets of the set of available inputs; and providing at least two trained neural networks having different subsets of the available inputs as components of a neural network ensemble configured to transform the available inputs into at least one output. The neural network ensemble may be applied as a log synthesis method that comprises: receiving a set of downhole logs; applying a first subset of downhole logs to a first neural network to obtain an estimated log; applying a second, different subset of the downhole logs to a second neural network to obtain an estimated log; and combining the estimated logs to obtain a synthetic log.

## Claims

### Claim 1

A method of synthesizing a well log, the method comprising:
receiving a set of input signals that represent measurements of downhole formation characteristics;
applying a first subset of the set of input signals to a first neural network to obtain one or more estimated logs;
applying a second subset of the set of input signals to a second neural network to obtain one or more estimated logs, wherein the first and second subsets are distinct in that one subset has at least one input signal not included by the other; and
combining corresponding ones of the one or more estimated logs from the first and second neural networks to output one or more synthetic logs to display to a user.

### Claim 2 (depends on 1)

The method of claim 1, wherein the set of input signals represents a set of cased hole logs derived from measurements by a pulsed neutron tool.

### Claim 3 (depends on 2)

The method of claim 2, wherein the pulsed neutron tool is a pulsed neutron capture tool.

### Claim 4 (depends on 2)

The method of claim 2, wherein the set of input signals is derived from measurements by at least one of the following: full wave sonic logging tools, natural gamma ray spectroscopy tools, cased hole resistivity logging tools, cased hole neutron logging tools, and production logging tools.

### Claim 5 (depends on 1)

The method of claim 1, wherein the set of input signals is derived from measurements by at least one of the following: full wave sonic logging tools, natural gamma ray spectroscopy tools, cased hole resistivity logging tools, cased hole neutron logging tools, and production logging tools.

### Claim 6 (depends on 1)

The method of claim 1, wherein the one or more synthetic logs comprise at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first and second neural networks have different values of an architectural parameter.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first and second neural networks are backpropagation networks with different numbers of hidden nodes.

### Claim 9 (depends on 1)

The method of claim 1, wherein the second subset is a proper subset of the first subset.

### Claim 10 (depends on 1)

The method of claim 1, wherein the first subset equals the set of input signals.

### Claim 11

A computer-based method that generates a trained neural network ensemble, the method comprising:
receiving a set of available inputs;
receiving training data comprising values for the available inputs and corresponding values for at least one output;
training at least one neural network for each of at least two different subsets of the set of available inputs, wherein the at least two different subsets are distinct in that one subset has at least one input signal from a sensor type not included in the other subset;
providing at least two trained neural networks having different subsets of the available inputs as components of a neural network ensemble that transforms said available inputs into said at least one output, wherein said components are selected at least in part based on a measure of negative correlation between the components; and
plotting said at least one output from the neural network ensemble as a function of at least one of time, depth, and position.

### Claim 12 (depends on 11)

The computer-based method of claim 11, wherein the set of available inputs is a set of cased hole logs that can be derived from measurements by a pulsed neutron tool.

### Claim 13 (depends on 12)

The computer-based method of claim 12, wherein the set of available input signals further includes cased hole logs that can be derived from measurements by at least one of the following: a full wave sonic logging tool, a natural gamma ray spectroscopy tool, a cased hole resistivity logging tool, a cased hole neutron logging tool, and a production logging tool.

### Claim 14 (depends on 11)

The computer-based method of claim 11, wherein at least one output comprises at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

### Claim 15 (depends on 11)

The computer-based method of claim 11, wherein said training results in a pool of trained neural networks, and wherein said providing includes:
forming an initial population of neural network ensembles, said ensembles comprising neural networks from the pool;
evolving the population of neural network ensembles to improve a fitness measure; and selecting an ensemble from the evolved population.

### Claim 16 (depends on 15)

The computer-based method of claim 15, wherein said evolving is based on a genetic algorithm.

### Claim 17 (depends on 15)

The computer-based method of claim 15, wherein the fitness measure is determined from at least one of the following: a combination of a performance measure, a complexity measure, and a negative correlation measure.

### Claim 18 (depends on 11)

The computer-based method of claim 11, wherein said training results in a pool of trained neural networks, said pool having diversity in neural network inputs and neural network architectures.

### Claim 19 (depends on 18)

The computer-based method of claim 18, wherein the neural networks in the pool are backpropagation networks, and wherein the architectural diversity resides in a hidden layer size.

### Claim 20 (depends on 11)

The computer-based method of claim 11, wherein the training includes training at least one neural network for each of a series of different input subsets, the input subsets each being a proper subset of all subsequent input subsets in the series.

### Claim 21 (depends on 20)

The computer-based method of claim 20, wherein the number of input subsets in the series is at least equal to the number of available inputs.

### Claim 22 (depends on 11)

The computer-based method of claim 11, wherein the training includes determining for each input subset size a best subset of available inputs.

### Claim 23 (depends on 22)

The computer-based method of claim 22, wherein the training includes constraining the input subsets tested for a best subset to those subsets that include the best subsets of smaller size.

### Claim 24 (depends on 22)

The computer-based method of claim 22, wherein the training includes constraining the input subsets tested for a best subset to those subsets that are proper subsets of the best subsets of larger size.

### Claim 25 (depends on 11)

The computer-based method of claim 11, wherein said training results in a locally ranked set of trained candidate neural networks.

### Claim 26 (depends on 11)

The computer-based method of claim 11, wherein said training results in a globally ranked set of trained candidate neural networks.

### Claim 27 (depends on 11)

The computer-based method of claim 11, wherein said training results in a set of trained neural networks, and wherein the method further comprises setting a candidate neural network pool size based on a number of trained neural networks in the set satisfying a performance measure.

### Claim 28 (depends on 27)

The computer-based method of claim 27, wherein the performance measure is a prediction error threshold determined from a minimum prediction error of a set of trained neural networks for a best input subset.

### Claim 29

An information storage medium that, when placed in operable relation to a computer, provides the computer with software that generates a trained neural network ensemble, the software comprising:
a training process that generates a set of neural networks having diversity in inputs and in complexity, wherein diversity in inputs requires that at least one of the neural networks in the set operates on a different combination of input signals than another of the neural networks in the set;
a selection process that identifies a combination of neural networks from the set having a desirable fitness measure, said fitness measure being based at least in part on a measure of negative correlation for each neural network in the combination;
a transform process that applies the combination of neural networks in ensemble fashion to a set of inputs to synthesize at least one output; and
a process to display said at least one output.

### Claim 30 (depends on 29)

The medium of claim 29, wherein said at least one output comprises at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

### Claim 31 (depends on 29)

The medium of claim 29, wherein the set of neural networks is locally ranked.

### Claim 32 (depends on 29)

The medium of claim 29, wherein the set of neural networks is globally ranked.

### Claim 33 (depends on 29)

The medium of claim 29, wherein the selection process forms a pool of candidate neural networks from best-performing neural networks in the set of neural networks after determining a pool size, and wherein the selection process determines a pool size based on a number of neural networks in the set that satisfy a performance measure.

### Claim 34 (depends on 33)

The medium of claim 33, wherein the performance measure is a prediction error threshold determined from a minimum prediction error by a set of neural networks trained from a best input subset.