Minibatch Parallel Machine Learning System Design
AssigneeInternational Business Machines Corporation
InventorsChanghoan Kim, Michael P. Perrone
Life of this patent
Estimating the term…
Abstract
The disclosure is directed to optimizing parallel machine learning system design and performance using minibatch. A system for allocating data center resources according to embodiments includes: a machine learning process; a machine learning data set; a processing system including a P parallel processing elements for training the machine learning process using the machine learning data set, wherein the machine learning data set is split into a plurality of batches with a batch size M; and a resource manager for (1) minimizing a training time T=T(M,P) of the machine learning process over M for each value of P, and (2) efficient system design.
Claims20
A system for allocating data center resources, comprising:
- a machine learning process;
- a machine learning data set;
- a processing system including P parallel processing elements for training the machine learning process using the machine learning data set, wherein the machine learning data set is split into a plurality of batches with a batch size M;
- and
- a resource manager for minimizing a training time T=T(M,P) of the machine learning process over the batch size M for each value of P.
An optimization system, comprising:
- a machine learning process;
- a machine learning data set;
- a processing system for training the machine learning process using the machine learning data set, wherein the machine learning data set is split into a plurality of batches with a batch size M;
- and
- a resource manager for determining a number P of parallel processing elements in the processing system such that a training time T=T(M,P) of the machine learning process is minimized for the batch size M and a cost constraint is met.
An optimization method, comprising:
- training a machine learning process on a processing system using a machine learning data set, wherein the machine learning data set is split into a plurality of batches with a batch size M;
- and
- optimizing the processing system by:
- minimizing, using P parallel processing elements in the processing system, a training time T=T(M,P) of the machine learning process over the batch size M for each value of P;
- or
- determining a number P of parallel processing elements in the processing system, such that a training time T=T(M,P) of the machine learning process is minimized for the batch size M.
Dependencies are inferred from the claim text.