Computer system for distributed machine learning

A method for distributed training of a machine learning model by implementing a plurality of software modules executed by computer hardware, the modules comprising a central (bulk synchronous parallel) BSP control module, a local BSP module, and a machine learning module, the method comprising:

• instructing, by the central BSP control module, the local BSP module to store a local model in a shared memory corresponding to the local BSP module and the machine learning module;
• reading, by the machine learning module, the local model from the shared memory corresponding to the local BSP module and the machine learning module;
• computing, by the machine learning module, a gradient based on the local model;
• aggregating, by the machine learning module, the gradient into an aggregated gradient stored in the shared memory;
• and
• instructing, by the central BSP module, the local BSP module to periodically read out the aggregated gradient from the shared memory.

A non-transitory computer program product storing program code for performing, when running on a computer, a method for distributed training of a machine learning model by implementing a plurality of software modules executed by computer hardware, the modules comprising a central (bulk synchronous parallel) BSP control module, a local BSP module, and a machine learning module, the method comprising:

• instructing, by the central BSP control module, the local BSP module to store a local model in a shared memory corresponding to the local BSP module and the machine learning module;
• reading, by the machine learning module, the local model from the shared memory corresponding to the local BSP module and the machine learning module;
• computing, by the machine learning module, a gradient based on the local model;
• aggregating, by the machine learning module, the gradient into an aggregated gradient stored in the shared memory;
• and
• instructing, by the central BSP module, the local BSP module to periodically read out the aggregated gradient from the shared memory.