Dynamic computation rates for distributed deep learning

A computer program product comprising program instructions stored on a computer readable storage medium, the program instructions executable by a processor to cause the processor to perform a method comprising:

• performing distributed deep learning training on a plurality of batches of training data using a corresponding plurality of learners;
• determining a plurality of training times wherein the plurality of learners performs the distributed deep learning training on the plurality of batches of training data;
• and
• modifying a processing aspect of a plurality of remaining learners to increase a future training time of the plurality of remaining learners for performing the distributed deep learning training on a new batch of training data in response to identifying a straggler of the plurality of learners by a centralized control.

A system comprising:

• a computer processing circuit;
• and
• a computer-readable storage medium storing instructions, which, when executed by the computer processing circuit, are configured to cause the computer processing circuit to perform a method comprising:
• performing distributed deep learning training on a plurality of batches of training data using a corresponding plurality of learners;
• determining a plurality of training times wherein the learners performs the distributed deep learning training on the plurality of batches of training data;
• generating a table comprising the training times;
• identifying a straggler of the plurality of learners based on the table, wherein a training time of the straggler exceeds a threshold training time for the plurality of training times;
• and
• modifying a processing aspect of the straggler to reduce a future training time of the straggler for performing the distributed deep learning training on a new batch of training data in response to identifying a straggler of the plurality of learners by a centralized control.