Support vector method for function estimation
AssigneeAT&T Corp.
InventorVladimir Vapnik
Life of this patent
Estimating the term…
Abstract
A method for estimating a real function that describes a phenomenon occurring in a space of any dimensionality. The function is estimated by taking a series of measurements of the phenomenon being described and using those measurements to construct an expansion that has a manageable number of terms. A reduction in the number of terms is achieved by using an approximation that is defined as an expansion on kernel functions, the kernel functions forming an inner product in Hilbert space. By finding the support vectors for the measurements one specifies the expansion functions. The number of terms in an estimation according to the present invention is generally much less than the number of observations of the real world phenomenon that is being estimated.
Claims44
Apparatus for forming a function estimate that characterizes behavior of a system, comprising:
- a) an input device for receiving a plurality of measurements of said behavior of said system;
- b) one or more stored programs that include instructions for
- (i) using said measurements, a kernel function that defines an inner product in Hilbert space, and a chosen error tolerance to form a functional;
- (ii) minimizing said functional to find a plurality of coefficients and a plurality of support vectors, said coefficients and said support vectors corresponding on a one to one basis;
- and
- (iii) forming said function estimate in a form of an expansion from said kernel, said coefficients, and said support vectors, said expansion having a plurality of terms, each said term constituting a separate dimension of said function estimate and including a first multiplier that is said kernel function as defined by one of said support vectors and a second multiplier that is said coefficient corresponding to said one of said support vectors;
- c) a processor for executing said instructions;
- and
- d) an output module that develops output signals based on said function estimate for interacting with said system.
A function estimator for estimating a real function that describes a phenomenon in a space of any dimension, comprising:
- a) an input device for forming signals from measurements of the phenomenon;
- b) a processing unit for
- (i) using said measurements, a kernel function that defines an inner product in Hilbert space, and a chosen error tolerance to form a functional;
- (ii) minimizing said functional to find a plurality of coefficients and a plurality of support vectors, said coefficients and said support vectors corresponding on a one to one basis;
- and
- (iii) forming an expansion from said kernel, said coefficients, and said support vectors, said expansion forming said estimate of the real function and having a plurality of terns, each said term including a first multiplier that is said kernel function as defined by one of said support vectors and a second multiplier that is said coefficient corresponding to said one of said support vectors;
- and
- c) an output processing unit, responsive to said expansion for developing output signals for forming decisions about said phenomenon.
A method for forming a function estimate that characterizes behavior or a system describable by a function in a multi-dimensional space, comprising the steps of:
- a) taking a plurality of measurements of said behavior;
- b) using said measurements, a kernel function that defines an inner product in Hilbert space, and a chosen error tolerance to form a functional;
- c) minimizing said functional to find a plurality of coefficients and a plurality of support vectors, said coefficients and said support vectors corresponding on a one to one basis;
- and
- d) forming an expansion from said kernel, said coefficients, and said support vectors, said expansion forming said function estimate and having a plurality of terms, each said term including a first multiplier that is said kernel function as defined by one of said support vectors and a second multiplier that is said coefficient corresponding to said one of said support vectors;
- and
- e) developing an output signal for interacting with said system.
A method for forming a function estimate that characterizes behavior or a system describable by a function in a multi-dimensional space, comprising the steps of:
- performing measurements of activities of said system;
- forming a functional from a kernel function that defines an inner product in Hilbert space and a chosen error tolerance criterion that is insensitive to the error in the neighborhood of -.epsilon. to +.epsilon., for small values of error .epsilon.;
- minimizing said functional for an applied plurality of measurement signals to find a plurality of coefficients and a plurality of support vectors;
- forming an expansion from said kernel, said coefficients, and said support vectors, said expansion forming said function estimate and having a plurality of terms, each said term including a first multiplier that is said kernel function as defined by one of said support vectors and a second multiplier that is said coefficient corresponding to said one of said support vectors;
- and
- developing an output signal for interacting with said system.
Dependencies are inferred from the claim text.