Method, apparatus and computer program to carry out a training procedure in a convolutional neural network

An apparatus to communicationally couple to a number of processing nodes 1 to X to carry out a training procedure of a Convolutional Neural Network (CNN), the apparatus comprising:

• a processor;
• and
• a memory having instructions stored thereon, the instructions when executed by the apparatus implementing a node among the number of nodes, causing the node to control operations including,
• dividing neurons of the CNN between the number of nodes;
• allocating a mini-batch of input data from among mini-batches of input data to a node of the number of nodes;
• splitting from among the number of nodes, the mini-batch into a number of mini-batch sections X corresponding and equal to the number of nodes;
• retaining a mini-batch section from among the split mini-batch sections which has a same number as the node and sending other mini-batch sections of the split mini-batch sections to corresponding other nodes according to a number of the split mini-batch sections;
• collating the split mini-batch sections at the node into a single matrix and multiplying the collated matrix by the neurons to provide output data sections having one section of output data per each mini-batch section of the split mini-batch sections;
• and
• sending the output data sections corresponding to the other corresponding nodes to the corresponding nodes and combining the output data sections in the node so that the node has output data for entire of the split mini-batch sections.

A non-transitory computer-readable storage means storing a computer program which when executed by a number of processing nodes 1 to X, causes the number of processing nodes to perform operations comprising:

• dividing neurons of a Convolutional Neural Network (CNN) between processing nodes;
• allocating a mini-batch of input data from among mini-batches of input data to a node of the number of nodes;
• splitting for the node, from among the number of nodes, the mini-batch into a number of mini-batch sections X corresponding and equal to the number of nodes;
• at the node retaining a mini-batch section from among the split mini-batch sections which has a same number as the node and sending other mini-batch sections of the split mini-batch sections to corresponding other nodes according to a number of the split mini-batch sections;
• collating at the node the split mini-batch sections at the node into a single matrix and multiplying the collated matrix by the neurons to provide output data sections having one section of output data per each mini-batch section of the split mini-batch sections;
• at the node sending the output data sections corresponding to the other corresponding nodes to the corresponding nodes and combining the output data sections in the node so that the node has output data for entire of the split mini-batch sections.