Optimal filtering by recurrent neural networks

A method for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, said method comprising the steps of:

• (1) selecting an ancillary signal process on the basis of said estimation error statistic process;
• (2) selecting an ancillary estimation error criterion on the basis of said estimation error statistic process and said ancillary signal process;
• (3) selecting an ancillary recurrent neural network paradigm;
• (4) generating realizations of said measurement process and corresponding realizations of said ancillary signal process;
• (5) constructing an ancillary training criterion;
• (6) synthesizing said realizations of said ancillary signal process and corresponding said realizations of said measurement process into an ancillary filter, which is an ancillary recurrent neural network of said ancillary recurrent neural network paradigm;
• (7) implementing said ancillary filter, specified by said ancillary recurrent neural network's architecture, weights and initial dynamic state;
• and
• (8) receiving one measurement vector of said measurement process at a time at at least one input terminal of the implementation of said ancillary filter and producing an estimate of one ancillary signal vector of said ancillary signal process at a time at at least one output terminal of said implementation of said ancillary filter.

A method for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, said method comprising the steps of:

• (1) selecting an ancillary signal process on the basis of said estimation error statistic process;
• (2) selecting an ancillary estimation error criterion on the basis of said estimation error statistic process and said ancillary signal process;
• (3) selecting an ancillary recurrent neural network paradigm;
• (4) generating realizations of said measurement process, corresponding realizations of said primary filter's output process and corresponding realizations of said ancillary signal process;
• (5) constructing an ancillary training criterion;
• (6) synthesizing said realizations of said primary filter's output process, corresponding said realizations of said ancillary signal process and corresponding said realizations of said measurement process into an ancillary filter, which is an ancillary recurrent neural network of said ancillary recurrent neural network paradigm;
• (7) implementing said ancillary filter, specified by said ancillary recurrent neural network's architecture, weights and initial dynamic state;
• and
• (8) receiving one measurement vector of said measurement process and one output vector of said primary filter at a time at at least one input terminal of the implementation of said ancillary filter and producing an estimate of one ancillary signal vector of said ancillary signal process at a time at at least one output terminal of said implementation of said ancillary filter.

A method for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said method comprising the steps of:

• (1) selecting a recurrent neural network paradigm;
• (2) selecting an estimation error criterion;
• (3) generating augmented training data comprising a set of exemplary time functions, corresponding realizations of said signal process and corresponding realizations of said measurement process;
• (4) constructing a training criterion;
• (5) synthesizing said augmented training data into a primary filter with augmented exogenous input nodes, which is a recurrent neural network of said recurrent neural network paradigm;
• (6) implementing said primary filter with augmented exogenous input nodes, specified by said recurrent neural network's architecture, weights and initial dynamic state;
• and
• (7) receiving one measurement vector of said measurement process and one vector of said p-dimensional vector-valued time function at a time at at least one input terminal of the implementation of said primary filter with augmented exogenous input nodes, and producing an estimate of one signal vector of said signal process at a time at at least one output terminal of the implementation of said primary filter with augmented exogenous input nodes.

A method for evaluating a q-dimensional vector-value estimation error statistic process, that is required for a primary filter with augmented exogenous input nodes for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said method comprising the steps of:

• (1) selecting an ancillary signal process on the basis of said estimation error statistic process;
• (2) selecting an ancillary estimation error criterion on the basis of said estimation error statistic process and said ancillary signal process;
• (3) selecting an ancillary recurrent neural network paradigm;
• (4) generating augmented ancillary training data comprising a set of exemplary time functions, corresponding realizations of said measurement process and corresponding realizations of said ancillary signal process;
• (5) constructing an ancillary training criterion;
• (6) synthesizing said augmented ancillary training data into an ancillary filter with augmented exogenous input nodes, which is an ancillary recurrent neural network of said ancillary recurrent neural network paradigm;
• (7) implementing said ancillary filter with augmented exogenous input nodes, specified by said ancillary recurrent neural network's architecture, weights and initial dynamic state;
• and
• (8) receiving one measurement vector of said measurement process and one vector of said p-dimensional vector-valued time function at a time at least one input terminal of the implementation of said ancillary filter with augmented exogenous input nodes and producing an estimate of one signal vector of said ancillary signal process at a time at least one output terminal of said implementation of said ancillary filter with augmented exogenous input nodes.

A method for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter with augmented exogenous input nodes for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said method comprising the steps of:

• (1) selecting an ancillary signal process on the basis of said estimation error statistic process;
• (2) selecting an ancillary estimation error criterion on the basis of said estimation error statistic process and said ancillary signal process;
• (3) selecting an ancillary recurrent neural network paradigm;
• (4) generating augmented ancillary training data comprising a set of exemplary time functions, corresponding realizations of said measurement process, corresponding realizations of the output process of said primary filter with augmented exogenous input nodes, and corresponding realizations of said ancillary signal process;
• (5) constructing an ancillary training criterion;
• (6) synthesizing said augmented ancillary training data into an ancillary filter with augmented exogenous input nodes, which is an ancillary recurrent neural network of said ancillary recurrent neural network paradigm;
• (7) implementing said ancillary filter with augmented exogenous input nodes, specified by said ancillary recurrent neural network's architecture, weights and initial dynamic state;
• and
• (8) receiving one vector of said p-dimensional vector-valued time function, one measurement vector of said measurement process and one output vector of said primary filter at a time at least one input terminal of the implementation of said ancillary filter with augmented exogenous input nodes and producing an estimate of one signal vector of said ancillary signal process at a time at least one output terminal of said implementation of said ancillary filter with augmented exogenous input nodes.

A primary filter for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process with respect to a selected estimation error criterion, said primary filter being an implementation of a recurrent neural network of a selected recurrent neural network paradigm, and said primary filter comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing an estimate of one signal vector of said n-dimensional vector-valued signal process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.

An ancillary filter for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, said ancillary filter being an implementation of an ancillary recurrent neural network of a selected ancillary recurrent neural network paradigm, and said ancillary filter comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing one estimation error statistic vector of said q-dimensional vector-valued estimation error statistic process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.

An ancillary filter for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, said ancillary filter being an implementation of an ancillary recurrent neural network of a selected ancillary recurrent neural network paradigm, and said ancillary filter comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process and one output vector of said primary filter at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing one estimation error statistic vector of said q-dimensional vector-valued estimation error statistic process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.

A primary filter with augmented exogenous input nodes for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process with respect to a selected estimation error criterion, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said primary filter with augmented exogenous input nodes being an implementation of a recurrent neural network paradigm, and said primary filter with augmented exogenous input nodes comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process and one vector of said p-dimensional vector-valued time function at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing an estimate of one signal vector of said n-dimensional vector-valued signal process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.

An ancillary filter with augmented exogenous input nodes for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter with augmented exogenous input nodes for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said ancillary filter with augmented exogenous input nodes being an implementation of an ancillary recurrent neural network of a selected ancillary recurrent neural network paradigm, and said ancillary filter with augmented exogenous input nodes comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process and one vector of said p-dimensional vector-valued time function at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing one estimation error statistic vector of said q-dimensional vector-valued estimation error statistic process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.

An ancillary filter with augmented exogenous input nodes for evaluating a q-dimensional vector-valued estimation error statistic process, that is required for a primary filter with augmented exogenous input nodes for processing an m-dimensional vector-valued measurement process to estimate an n-dimensional vector-valued signal process, which signal and measurement processes are time-variant with said signal and measurement processes' time-variant property described by a p-dimensional vector-valued time function, said ancillary filter with augmented exogenous input nodes being an implementation of an ancillary recurrent neural network of a selected ancillary recurrent neural network paradigm, and said ancillary filter with augmented exogenous input nodes comprising

• (1) a plurality of neurons, each neuron being means for receiving said neuron's input values, processing said input values and producing an output value, that is called the activation level of said neuron;
• (2) a plurality of connections, each connection being means for multiplying a first neuron's activation level by said connection's weight and sending the resulting multiple to a second neuron as one of said second neuron's input values;
• (3) a plurality of feedback connections, each feedback connection being means for holding a third neuron's activation level for one unit of time, multiplying said third neuron's activation level by said feedback connection's weight, and sending the resulting multiple to a fourth neuron as one of said fourth neuron's input values;
• (4) at least one input terminal, being means for inputing one measurement vector of said m-dimensional vector-valued measurement process, one output vector of said primary filter and one vector of said p-dimensional vector-valued time function at a time, said at least one input terminal being connected to at least one exogenous input node included in said plurality of neurons;
• and
• (5) at least one output terminal, being means for outputing one estimation error statistic vector of said q-dimensional vector-valued estimation error statistic process at a time, said at least one output terminal being connected to at least one outward output node included in said plurality of neurons.