Gradient based training method for a support vector machine
AssigneeTelestra New Wave Pty Ltd
InventorsAdam Kowalczyk, Trevor Bruce Anderson
Life of this patent
Estimating the term…
Abstract
A training method for a support vector machine, including executing an iterative process on a training set of data to determine parameters defining the machine, the iterative process being executed on the basis of a differentiable form of a primal optimisation problem for the parameters, the problem being defined on the basis of the parameters and the data set.
Claims16
A training method for a support vector machine, including executing an iterative process on a training set of data to determine parameters defining said machine, said iterative process being executed on the basis of a differentiable form of a primal optimisation problem for said parameters, said problem being defined on the basis of said parameters and said data set.
A support vector machine for a classification task having an output y given by
- 39 y = y ( x ) = ∑ i = 1 n y i α i k ( x i , x j ) + β b
- where xεRm is a data point to be classified and xi are training data points, k is a kernel function, and αi are coefficients determined by
- αi=CL′(1−yiηi−βb)
- where L′(ξ) is the derivative of the loss and the values ηi are determined by iteratively executing
- 40 η j t + 1 = η j t - δ ( η j t - C ∑ i = 1 n L ′ ( 1 - y i η i t - y i β b t ) y i k ( x i , x j ) ) , b t + 1 = β b t + δ β C ∑ i = 1 n L ′ ( 1 - y i η j t - y i β b t ) y i .
- where ε>0 is a free parameter representing a learning rate and/or, by iteratively executing in the homogeneous case (β=0):
- 41 η j t + 1 = C ∑ i = 1 n L ′ ( 1 - y i η j t ) y i k ( x i , x j ) .
- where i, j=1, . . . , n, n are the number of data points, t represents an iteration and L′ is the derivative of a loss function L.
A support vector machine for F-regression having output y given by
- 42 y ( x ) = ∑ i = I n β i k ( x , x i ) + β b
- where xεRm is a data point to be evaluated and xi are training data points, k is a kernel function, β=0 or 1, and βi and bias b are coefficients determined by
- βi=CL′(|yi−ηi−βb|−ε)sgn(yi−ηi−βb)
- where ε is a free parameter and the values ηi and b are determined by iteratively executing
- 43 η j t + 1 = η j t - δ ( η j t - C ∑ i = 1 n L ′ ( [LeftBracketingBar] y i - η i t - β b [RightBracketingBar] - ϵ ) sgn ( y i - η i t - β b ) k ( x i , x j ) ) b t + 1 = b t + δ β C ∑ i = 1 n L ′ ( [LeftBracketingBar] y i - η i t - β b [RightBracketingBar] - ϵ ) sgn ( y i - η i t - β b )
- where δ>0 is a free parameter representing a learning rate and/or, by iteratively executing in the homogeneous case (β=0):
- 44 η j t + 1 = C ∑ i = 1 n L ′ ( [LeftBracketingBar] y i - η j t [RightBracketingBar] - ϵ ) sgn ( y i - η i t ) k ( x i , x j ) .
- where i, j=1, . . . , n, n being the number of data points and t represents an iteration and L′ is the derivative of a loss function L.
A regularisation network having output y given by
- 45 y ( x ) = ∑ i = 1 n β i k ( x , x i ) + β b
- where xεRm is a data point to be evaluated and xi are training data points, k is a kernel function, β=0 or 1, and βi and bias b are coefficients determined by
- βi=CL′(|yi−ηi−βb|−ε)
- where ε is a free parameter and the values ηi and b are determined by iteratively executing
- 46 η j t + 1 = η j t - δ ( η j t - C ∑ i = 1 n L ′ ( y i - η i t - β b ) k ( x ~ i , x ~ j ) ) b t + 1 = b t + δ β C ∑ i = 1 n L ′ ( y i - η i t - β b )
- where δ>0 is a free parameter representing a learning rate and/or, by iteratively executing in the homogeneous case (δ=0):
- 47 η j t + 1 = C ∑ i = 1 n L ′ ( y i - η i t - β b ) k ( x ~ i , x ~ j )
- where i j 1, . . . , n, n being the number of data points and t represents an iteration and L′ is the derivative of a loss function L.
A method of generating a support vector machine with a differentiable penalty by direct minimisation of a primal problem.
A method of generating support vector machine with a linear penalty by executing a process to solve differentiable loss as a smoothed approximation of a linear loss, L(ξ)−max(0,ξ).
A method of generating a decision surface for a learning machine by iteratively executing a solution for a vector representing said decision surface using a training data set, said solution being based on a gradient of a differentiable form of an optimisation function for said surface.
Dependencies are inferred from the claim text.