Load distribution for a distributed neural network

An apparatus for dynamic load distribution for a distributed neural network wherein processing by the distributed neural network comprises processing a plurality of layers, comprising:

• a memory comprising executable instructions;
• and
• one or more processors configured to communicate with the memory, wherein the one or more processors are configured to cause the apparatus to:
• estimate, by a device of the neural network, an energy usage for processing at least one non-processed layer in the device;
• estimate, by the device of the neural network, an energy usage for transmitting layer output of at least one processed layer to a cloud service of the neural network for processing;
• compare, by the device of the neural network, the estimated energy usage for processing the at least one non-processed layer in the device with the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service;
• determine to process the at least one non-processed layer in the device when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is equal or greater than the estimated energy usage for processing the at least one non-processed layer;
• and
• determine to transmit the layer output of the at least one processed layer to the cloud service for processing subsequent layers when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is less than the estimated energy usage for processing the at least one non-processed layer in the device.

A cloud service assembly for dynamic load distribution for a distributed neural network wherein processing by the distributed neural network comprises processing a plurality of layers and wherein the cloud service comprises controlling circuitry configured to:

• receive layer output of at least one processed layer of the neural network from a device of the neural network;
• process subsequent layers of the neural network in response to receiving the layer output of the at least one processed layer from the device of the neural network;
• and
• transmit layer output of the processed subsequent layers to the device of the neural network.

A system for dynamic load distribution for a distributed neural network wherein processing by the distributed neural network comprises processing a plurality of layers, comprising:

• an estimating module configured to estimate, by a device of the neural network, an energy usage for processing at least one non-processed layer in the device;
• an estimating module configured to estimate, by the device of the neural network, an energy usage for transmitting layer output of at least one processed layer to a cloud service of the neural network for processing;
• a comparing module configured to compare, by the device of the neural network, the estimated energy usage for processing the at least one non-processed layer in the device with the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service;
• a determining module configured to determine to process the at least one non-processed layer in the device when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is equal or greater than the estimated energy usage for processing the at least one non-processed layer;
• and
• a determining module configured to determine to transmit the layer output of the at least one processed layer to the cloud service for processing subsequent layers when the estimated energy usage for transmitting the layer output of the at least one processed layer to the cloud service is less than the estimated energy usage for processing the at least one non-processed layer in the device.