Splitting neural networks on multiple edge devices to train on vertically distributed data
AssigneeEMC IP Holding Company LLC
InventorsMohamed Abouzeid, Osama Taha Mohamed, AbdulRahman Diaa
Life of this patent
Estimating the term…
Abstract
One example method includes a pipeline for a distributed neural network. The pipeline includes a first phase that identifies intersecting identifiers across datasets of multiple clients in a privacy preserving manner. The second phase includes a distributed neural network that includes a data receiving portion at each of the clients and an orchestrator portion at an orchestrator. The data receiving portions and the orchestrator portions communicate forward and backward passes to perform training without revealing the raw training data.
Claims20
A method for training a distributed neural network, comprising:
- identifying intersecting identifiers for datasets across clients, each of the clients including a data receiving portion of the distributed neural network, the datasets including training data for the distributed neural network;
- receiving the training data from the clients as input into corresponding data receiving portions;
- generating a forward pass at each of the data receiving portions;
- receiving the forward passes at an orchestrator that includes an orchestrator portion of the distributed neural network;
- calculating a loss at the orchestrator portion and propagating the loss through the distributed neural network using a backwards pass from the orchestrator portion to the data receiving portions;
- and
- adjusting the neural network based on the backwards pass.
A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
- identifying intersecting identifiers for datasets across clients, each of the clients including a data receiving portion of the distributed neural network, the datasets including training data for the distributed neural network;
- receiving the training data from the clients as input into corresponding data receiving portions;
- generating a forward pass at each of the data receiving portions;
- receiving the forward passes at an orchestrator that includes an orchestrator portion of the distributed neural network;
- calculating a loss at the orchestrator portion and propagating the loss through the distributed neural network using a backwards pass from the orchestrator portion to the data receiving portions;
- and
- adjusting the neural network based on the backwards pass.
A method for training a distributed neural network, comprising:
- identifying intersecting identifiers for datasets across clients, each of the clients including a data receiving portion of the distributed neural network, the datasets including training data for the distributed neural network;
- receiving the training data from the clients as input into corresponding data receiving portions;
- generating a forward pass at each of the data receiving portions;
- receiving the forward passes at an orchestrator that includes an orchestrator portion of the distributed neural network;
- sending an output of the orchestrator portion to the data receiving portions;
- determining a loss at each of the data receiving portions;
- sending the losses to the orchestrator portion;
- propagating the loss through the distributed neural network using a backwards pass from the orchestrator portion to the data receiving portions;
- and
- adjusting the neural network based on the backwards pass.
Dependencies are inferred from the claim text.