Neural network learning system inferring an input-output relationship from a set of given input and output samples

A data analyzing device used in the neural network learning system according to claim 22 for performing a clustering of a set of given data, said device comprising:

• classifying means for classifying data, stored in a first storage part, into a set of clusters in accordance with a prescribed probability distribution with respect to data included in each cluster so as to allocate the given data to the clusters;
• initializing means for determining initial values of parameters from the data in the first storage part, each of said parameters including an average, a standard deviation, and a coefficient to define the probability distribution of the data for each cluster, and said initial values of the parameters being stored in a second storage part;
• and
• parameter updating means for updating said parameters in said second storage part for each cluster each time the given data is allocated by said classifying means to the clusters,
• wherein the data is repeatedly classified by said classifying means into a set of clusters in accordance with a probability distribution based on said parameters in said second storage part each time said parameters are updated by said parameter updating means.

A neural network learning system for inferring an input-output relationship from a set of given input and output samples, comprising:

• a) probability density means for determining a probability density on a sum space of an input space and an output space from the set of given input and output samples by learning a value of a parameter through a prescribed maximum likelihood method;
• b) inference means for determining a probability density function based on the probability density from said probability density means, to infer the input-output relationship of the samples from the probability density function;
• wherein the learning a value of the parameter is repeated by said inference means until a value of a predefined parameter differential function using the prescribed maximum likelihood method is smaller than a prescribed value, thereby determining said parameter value;
• c) conditional probability distribution means for computing a conditional probability distribution from either the given input samples or the given output samples in accordance with the probability density function determined by said inference means;
• and
• d) output means for obtaining an inference value based on the conditional probability distribution from said conditional probability distribution means and for outputting said inference value;
• wherein:
• 1) said conditional probability distribution computed in accordance with the probability density function is an input distribution function P(w;
• x.vertline.y),
• 2) x.vertline.y indicates a relationship between x and y, and
• 3) said output means determines and outputs a set of inputs x in accordance with said function P(w;
• x.vertline.y) for a given output y, and a set of values of said function P(w;
• x.vertline.y) corresponding to the respective inputs x.

A neural network learning system for inferring an input-output relationship from a set of given input and output samples, comprising:

• a) probability density means for determining a probability density on a sum space of an input space and an output space from the set of given input and output samples by learning a value of a parameter through a prescribed maximum likelihood method;
• b) inference means for determining a probability density function based on the probability density from said probability density means, to infer the input-output relationship of the samples from the probability density function;
• wherein the learning a value of the parameter is repeated by said inference means until a value of a predefined parameter differential function using the prescribed maximum likelihood method is smaller than a prescribed value, thereby determining said parameter value;
• c) conditional probability distribution means for computing a conditional probability distribution from either the given input samples or the given output samples in accordance with the probability density function determined by said inference means;
• and
• d) output means for obtaining an inference value based on the conditional probability distribution from said conditional probability distribution means and for outputting said inference value;
• wherein:
• 1) a set of outputs and a probability of occurrence for each one of the outputs are determined for a given input, a probability density to determine the probability of occurrence being approximated by a linear combination of predetermined exponential functions according to a prescribed probability distribution function and being learned repeatedly using the prescribed maximum likelihood method;
• and
• 2) when the probability of occurrence for the given input is obtained, a determination is made whether the given input is known or unknown based on said probability of occurrence for the given input.

A neural network learning system for inferring an input-output relationship from a set of given input and output samples, comprising:

• a) probability density means for determining a probability density on a sum space of an input space and an output space from the set of given input and output samples by learning a value of a parameter through a prescribed maximum likelihood method;
• b) inference means for determining a probability density function based on the probability density from said probability density means, to infer the input-output relationship of the samples from the probability density function;
• wherein the learning a value of the parameter is repeated by said inference means until a value of a predefined parameter differential function using the prescribed maximum likelihood method is smaller than a prescribed value, thereby determining said parameter value;
• c) conditional probability distribution means for computing a conditional probability distribution from either the given input samples or the given output samples in accordance with the probability density function determined by said inference means;
• and
• d) output means for obtaining an inference value based on the conditional probability distribution from said conditional probability distribution means and for outputting said inference value;
• wherein an inference value for each of a set of outputs and a variance of the inference values are determined by said learning performed based on the probability density on the sum space for the given input and output samples.

A neural network learning system in which an input-output relationship for a set of input and output samples is predetermined, said system comprising:

• a) output probability means for determining output probabilities for given inputs in accordance with probability distributions on a sum space of an input space and an output space;
• and
• b) parameter learning means for carrying out a learning of parameters, the parameters respectively defining the probability distributions of said output probability means, wherein said parameter learning means includes:
• 1) clustering means for classifying the given data on the sum space into a set of clusters to determine statistical quantities of data in each of the clusters, and
• 2) parameter computing means for determining values of the parameters for the probability distributions of the output probability means based on the statistical quantities of the data from the clustering means.