US 2017/0372201 A1Published application

Secure Training of Multi-Party Deep Neural Network

AssigneeMassachusetts Institute of Technology

InventorsOtkrist Gupta, Ramesh Raskar

Life of this patent

Estimating the term…

Loading patent…

Abstract

A deep neural network may be trained on the data of one or more entities, also know as Alices. An outside computing entity, also known as a Bob, may assist in these computations, without receiving access to Alices' data. Data privacy may be preserved by employing a “split” neural network. The network may comprise an Alice part and a Bob part. The Alice part may comprise at least three neural layers, and the Bob part may comprise at least two neural layers. When training on data of an Alice, that Alice may input her data into the Alice part, perform forward propagation though the Alice part, and then pass output activations for the final layer of the Alice part to Bob. Bob may then forward propagate through the Bob part. Similarly, backpropagation may proceed backwards through the Bob part, and then through the Alice part of the network.

Claims20

Dependent claims

A method of training a neural network, partially on a first set of one or more computers and partially on a second set of one or more computers, wherein:

(a) the network comprises a first part and a second part, the first and second parts of the network being denoted in this claim as the “Bob part” and “Alice part”, respectively, and the first and second sets of computers being denoted in this claim as the “Bob computer” and “Alice computer”, respectively;
(b) the Alice part of the network comprises three or more neural layers;
(c) the Bob part of the network comprises two or more neural layers;
(d) a first dataset is inputted into an input layer of the Alice part of the network;
(e) the Alice computer performs forward propagation through the Alice part of the network;
(f) output activations of an output layer of the Alice part of the network are sent to the Bob computer and are inputted into an input layer of the Bob part of the network;
(g) the Bob computer performs forward propagation through the Bob part of the network;
(h) the Bob computer calculate losses and gradients;
(i) the Bob computer performs backpropagation through the Bob part of the network;
(j) gradients of the input layer of the Bob part of the network are sent to the Alice computer;
(k) the Alice computer performs backpropagation through the Alice part of the network;
and
(l) the Bob computer does not have access to the first dataset.

10 more elements

A method of training a neural network, partially on a first set of one or more computers and partially on other sets of one or more computers each, wherein:

(a) the network comprises a first part and a second part, the first and second parts of the network being denoted in this claim as the “Bob part” and “Alice part”, respectively, the first set of computers being denoted in this claim as the “Bob computer”, and each of the other sets of computers, respectively, being denoted in this claim as an “Alice computer”;
(b) the Alice part of the network comprises three or more neural layers;
(c) the Bob part of the network comprises two or more neural layers;
(d) for each respective Alice computer
(i) a dataset is inputted by the respective Alice computer into an input layer of the Alice part of the network;
(ii) the respective Alice computer performs forward propagation through the Alice part of the network;
(iii) output activations of an output layer of the Alice part of the network are sent to the Bob computer and inputted into an input layer of the Bob part of the network;
(iv) the Bob computer performs forward propagation through the Bob part of the network;
(v) the Bob computer calculates losses and gradients;
(vi) the Bob computer performs backpropagation through the Bob part of the network;
(vii) gradients of the input layer of the Bob part of the network are sent to the respective Alice computer, and
(viii) the respective Alice computer performs backpropagation through the Alice part of the network;
and
(e) the Bob computer has access to no database inputted in clause (d)(i) of this claim.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/630,944Filed 22 Jun 2017
Priority: 22 Jun 2016
Published: 28 Dec 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06N 3/098
Learning methods: distributed learning, e.g. federated learning
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks
G06N 3/0455
Combinations of networks: auto-encoder networks; Encoder-decoder networks
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]