US 2019/0156243 A1Published application

Efficient Large-Scale Kernel Learning Using a Distributed Processing Architecture

AssigneeInternational Business Machines Corporation

InventorsShen Li, Xiang Ni, Michael Witbrock, Lingfei Wu

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and system of creating a model for large scale data analytics is provided. Training data is received in a form of a data matrix X and partitioned into a plurality of partitions. A random matrix T is generated. A feature matrix is determined based on multiplying the partitioned training data by the random matrix T. A predicted data {tilde over (y)} is determined for each partition via a stochastic average gradient (SAG) of each partition. A number of SAG values is reduced based on a number of rows n in the data matrix X. For each iteration, a sum of the reduced SAG values is determined, as well as a full gradient based on the sum of the reduced SAG values from all rows n, by distributed parallel processing. The model parameters w are updated based on the full gradient for each partition.

Claims20

Dependent claims

A computing device comprising:

a processor;
a network interface coupled to the processor to enable communication over a network;
a storage device coupled to the processor;
a learning engine stored in the storage device, wherein an execution of the learning engine by the processor configures the computing device to perform acts comprising:
receiving training data in a form of a data matrix X;
partitioning the training data into a plurality of partitions;
generating a random matrix T having a size d*r, where r is a number of rows in the matrix T;
determining a feature matrix based on multiplying the partitioned training data by the random matrix T;
iteratively determining a predicted data {tilde over (y)} for each partition via a stochastic average gradient (SAG) of each partition and the feature matrix;
reducing a number of SAG values based on a number of rows n in the data matrix X;
for each iteration:
determining a sum of the reduced SAG values;
determining a full gradient based on the sum of the reduced SAG values from all rows n in the data matrix X, by distributed parallel processing via a distribution and reduction function;
and
updating model parameters w based on the full gradient for each partition.

12 more elements

A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computer device to carry out a method of creating a model for large scale data analytics via a distributed processing approach, the method comprising:

receiving training data in a form of a data matrix X;
partitioning the training data into a plurality of partitions;
generating a random matrix T having a size d*r, where r is a number of rows in the matrix T;
determining a feature matrix based on multiplying the partitioned training data by the random matrix T;
iteratively determining a predicted data {tilde over (y)} for each partition via a stochastic average gradient (SAG) of each partition;
reducing a number of SAG values based on a number of rows n in the data matrix X;
for each iteration:
determining a sum of the reduced SAG values;
determining a full gradient based on the sum of the reduced SAG values from all rows n in the data matrix X, by distributed parallel processing via AllReduce;
and
updating model parameters w based on the full gradient for each partition.

8 more elements

Dependencies are inferred from the claim text.