US 2022/0029971 A1Published application

Systems and Methods for Providing a Modified Loss Function in Federated-Split Learning

AssigneeTripleBlind, Inc.

InventorsGharib GHARIBI, Ravi PATEL, Babak Poorebrahim GILKALAYE, Praneeth VEPAKOMMA, Greg STORM, Riddhiman DAS

Life of this patent

Estimating the term…

Loading patent…

Abstract

Disclosed is a method that includes training, at a client, a part of a deep learning network up to a split layer of the client. Based on an output of the split layer, the method includes completing, at a server, training of the deep learning network by forward propagating the output received at a split layer of the server to a last layer of the server. The server calculates a weighted loss function for the client at the last layer and stores the calculated loss function. After each respective client of a plurality of clients has a respective loss function stored, the server averages the plurality of respective weighted client loss functions and back propagates gradients based on the average loss value from the last layer of the server to the split layer of the server and transmits just the server split layer gradients to the respective clients.

Claims20

Dependent claims

A method comprising:

training, at a client system of a plurality of client systems, a part of a deep learning network up to a split layer of the client system;
based on an output of the split layer of the client system, completing, at a server system, training of the deep learning network by asynchronously forward propagating the output received at a split layer of the server system to a last layer of the server system;
calculating a weighted loss function for the client system at the last layer of the server system to yield a calculated loss function for the client system;
storing the calculated loss function for the client system in a queue;
after each respective client system of the plurality of client systems has a respective loss function stored in the queue to yield a plurality of respective weighted client loss functions, averaging, at the server system, the plurality of respective weighted client loss functions to yield an average loss value;
back propagating gradients based on the average loss value from the last layer of the server system to the split layer of the server system to yield server system split layer gradients;
and
transmitting just the server system split layer gradients to the plurality of client systems, wherein no weights are shared across different client systems of the plurality of client systems.

5 more elements

A system comprising:

a storage configured to store instructions;
one or more processors configured to execute the instructions and cause the one or more processors to perform operations comprising:
training, at a client system of a plurality of client systems, a part of a deep learning network up to a split layer of the client system;
based on an output of the split layer of the client system, completing, at a server system, training of the deep learning network by asynchronously forward propagate the output received at a split layer of the server system to a last layer of the server system;
calculating a weighted loss function for the client system at the last layer of the server system to yield a calculated loss function for the client system;
storing the calculated loss function for the client system in a queue;
after each respective client system of the plurality of client systems has a respective loss function stored in the queue to yield a plurality of respective weighted client loss functions, averaging, at the server system, the plurality of respective weighted client loss functions to yield an average loss value;
back propagating gradients based on the average loss value from the last layer of the server system to the split layer of the server system to yield server system split layer gradients;
and
transmitting just the server system split layer gradients to the plurality of client systems, wherein no weights are shared across different client systems of the plurality of client systems.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/499,153Filed 12 Oct 2021
Priority: 13 Dec 2019
Published: 27 Jan 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification27 codes

H04L 63/0428
For providing a confidential data exchange among entities communicating through data packet networks: wherein the data content is protected, e.g.…
G06F 17/16
Complex mathematical operations: matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
G06F 18/2113
Selection of the most significant subset of features: by ranking or filtering the set of features, e.g. using a measure of variance or of feature…
G06F 18/24
Analysing: classification techniques

Family

Application62/948,105Us provisional application · filed 13 Dec 2019
Application16/828,085Continuation in part · filed 24 Mar 2020
Application16/828,216Continuation in part · filed 24 Mar 2020
Application16/828,354Continuation · filed 24 Mar 2020
Application16/828,420Continuation in part · filed 24 Mar 2020
Application63/090,904Us provisional application · filed 13 Oct 2020
Application17/176,530Continuation in part · filed 16 Feb 2021
Application63/226,135Us provisional application · filed 27 Jul 2021
Application17/499,153Continuation in part
Application17/176,530Continuation
Application16/828,354Continuation in part
US 10,924,460 B2Related patent
US 11,363,002 B2Related patent
US 11,431,688 B2Related patent
US 11,582,203 B2Related patent
US 11,895,220 B2Related patent
US 12,026,219 B2Related patent
US 2020/0220851 A1Published application
US 2020/0226470 A1Published application
US 2020/0228313 A1Published application
US 2020/0286145 A1Published application
US 2021/0194858 A1Published application
US 2022/0029971 A1This patent

From the US filing history. Foreign family members aren’t listed.