US 11,238,849 B1Granted

System and method for federated, context-sensitive, adaptive language models

AssigneeRockspoon, Inc.

InventorNagib Georges Mimassi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system and method for federated context-sensitive language models comprising a federated language model server and a plurality of edge devices. The federated language model server may comprise one or more machine learning models trained and developed centrally on the server, and distribute these one or more machine learning models to edge devices wherein they may be operated locally on the edge devices. The edge devices may gather or generate context data that can be used by a speech recognition engine, and the local language models contained therein, to develop adaptive, context-sensitive, user-specific language models. Periodically, the federated language model server may select a subset of edge devices from which to receive uploaded local model parameters, that may be aggregated to perform central model updates wherein the updated model parameters may then be sent back to edge devices in order to update the local model parameters.

Claims14

Dependent claims

A system for federated adaptive language models, comprising:

one or more edge devices, comprising at least a plurality of programming instructions stored in the memory of, and operating on at least one processor of, a computing device, wherein the plurality of programming instructions, when operating on the at least one processor, cause the computing device to:
receive context-based model adjustments;
receive updated machine and deep learning central model parameters;
apply the context-based model adjustments to the local models stored in the edge device;
apply the updated machine and deep learning central model parameters to the local models stored in the edge device;
receive local data from edge device sensors and other input mechanisms;
feed the received local data as input into one or more of the updated local machine and deep learning models to generate text-based output responsive to an edge device user query;
forward text-based output to an endpoint device and set up processes necessary for fulfillment of the edge device user query;
train and update local machine and deep learning models using the received local data;
and
upload trained and updated local machine and deep learning models parameters to a federated language model server;
and
a federated language model server, comprising at least a plurality of programming instructions stored in the memory of, and operating on at least one processor of, a computing device, wherein the plurality of programming instructions, when operating on the processor, causes the computer system to:
select a subset of edge devices to upload trained and updated local model parameters;
aggregate the received updated local model parameters form the subset of edge devices and compute the average value of the local model parameters;
update the machine and deep learning central models using the computed average values of the local model parameters as new central model parameters;
send the updated machine and deep learning central models to the one or more edge devices;
receive and store central data from the edge devices and external sources;
feed the central data into the machine and deep learning central models to generate as output context-based model adjustments for the local machine and deep learning models;
and
send the context-based adjustments to the one or more edge devices.

18 more elements

A method for federated adaptive language models, comprising the steps of:

receiving context-based model adjustments;
receiving updated machine and deep learning central model parameters;
applying the context-based model adjustments to the local models stored in the edge device;
applying the updated machine and deep learning central model parameters to the local models stored in the edge device;
receiving local data from edge device sensors and other input mechanisms;
feeding the received local data as input into one or more of the updated local machine and deep learning models to generate text-based output responsive to an edge device user query;
forwarding text-based output to an endpoint device and set up processes necessary for fulfillment of the edge device user query;
training and update local machine and deep learning models using the received local data;
uploading trained and updated local machine and deep learning models parameters to a federated language model server;
selecting a subset of edge devices to upload trained and updated local model parameters;
aggregating the received updated local model parameters form the subset of edge devices and compute the average value of the local model parameters;
updating the machine and deep learning central models using the computed average values of the local model parameters as new central model parameters;
sending the updated machine and deep learning central models to the one or more edge devices;
receiving and store central data from the edge devices and external sources;
feeding the central data into the machine and deep learning central models to generate as output context-based model adjustments for the local machine and deep learning models;
and
sending the context-based adjustments to the one or more edge devices.

14 more elements

Dependencies are inferred from the claim text.