Back to results
US 12,585,938 B2Granted
Consensus driven learning
AssigneeThe Government of the United States of America, as represented by the Secretary of the Navy
InventorsKyle Crandall, Dustin Webb
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for consensus driven learning (CDL) using machine learning (ML) to enable devices to learn a model on a data set that is distributed over several computational nodes in a decentralized manner. In an embodiment, local models are trained on local data and share model parameters in an asynchronous, decentralized, and distributed manner that imposes minimal restrictions on the topology of a communications network. Systems and methods using CDL in accordance with embodiments of the present disclosure do not require a central server to coordinate models like most conventional technologies, high bandwidth, or highly robust communication architecture between nodes.
Claims20
An unmanned platform device for gathering oceanographic data, the unmanned platform comprising:
- a consensus driven learning (CDL) device, wherein the CDL device is configured to:
- cause a processor to perform operations corresponding to a training phase, wherein the operations comprise:
- gathering the oceanographic data in a surrounding environment of the unmanned platform device, thereby generating a first observation of the surrounding environment, and
- training a local model using a local data set and a machine learning technique, wherein the local model includes a plurality of parameters, and wherein the local data set includes the first observation;
- and
- cause the processor to perform operations corresponding to an update phase, wherein the operations comprise:
- selecting a first portion of the plurality of parameters to be sent in a first partial update,
- selecting respective subsequent portions of parameters in the plurality of parameters to be sent in respective subsequent partial updates until all parameters in the plurality of parameters have been sent in the respective subsequent partial updates,
- generating a consensus for the oceanographic data based on the local model and input received from a second device in communication with the unmanned platform, wherein the input includes a second observation generated by the second device based on second oceanographic data in the surrounding environment;
- and
- a transceiver device configured to:
- send the first portion of the plurality of parameters in the first partial update to the second device,
- send the respective subsequent portions of parameters in the respective subsequent partial updates to the second device, wherein the first partial update and the respective subsequent partial updates are sent asynchronously, and
- receive the input from the second device, wherein the CDL device is further configured to cause the processor to perform operations comprising:
- pulling a packet of data from a communications buffer, and
- in response to a determination that the packet is an update packet:
- calculating a first quantity based on the packet and respective parameters of the local model corresponding to the packet, and
- updating the parameters of the local model based on the first quantity.
15 more elements
A system, comprising:
- an unmanned platform device for gathering oceanographic data, the unmanned platform configured to cause a processor to perform operations comprising:
- gathering the oceanographic data in a surrounding environment of the unmanned platform device, thereby generating a first observation of the surrounding environment;
- training a first local model using a local data set including the first observation and a machine learning technique, wherein the first local model includes a neural network containing a plurality of neurons,
- selecting a first portion of the plurality of neurons to be sent in a first partial update,
- creating a first neural sub network using the first portion of the plurality of neurons, and
- sending the first neural sub network in the first partial update, wherein the first partial update is sent asynchronously with respect to subsequent partial updates;
- and
- a second device in communication with the unmanned platform, wherein the second device is configured to cause the processor to perform operations comprising:
- receiving the first partial update,
- calculating a first quantity based on the first partial update and respective parameters of a second local model in the second device that correspond to the first partial update, wherein the second local model includes a second observation of the oceanographic data in the surrounding environment,
- updating the parameters of the second local model based on the first quantity, thereby generating an updated second local model, and
- generating a consensus for the oceanographic data based on the updated second local model.
9 more elements
A method, comprising:
- gathering, using a first unmanned platform device for gathering oceanographic data, the oceanographic data in a surrounding environment of the unmanned platform device, thereby generating a first observation of the surrounding environment;
- training, using the first unmanned platform device, a local model of the first unmanned platform device using a local data set including the first observation and a machine learning technique;
- selecting, using the first unmanned platform device, a first portion of the local model to be sent in a first partial update;
- selecting, using the first unmanned platform device, respective subsequent portions of the local model to be sent in respective subsequent partial updates until all portions of the local model have been sent in the respective subsequent partial updates, wherein the first partial update and the subsequent partial updates are sent asynchronously;
- sending, using a transceiver device of the first unmanned platform device, the first portion of the plurality of parameters in the first partial update to a second device in communication with the first device;
- sending, from the transceiver device, the respective subsequent portions of parameters in the respective subsequent partial updates to the second device;
- receiving, from the transceiver device, input from a second device in communication with the unmanned platform device, wherein the input includes a second observation generated by the second device of the oceanographic data in the surrounding environment;
- generating, using the first unmanned platform device, a consensus for the oceanographic data based on the local model and the input received from the second device;
- pulling, using the first unmanned platform device, a packet of data from a communications buffer;
- and
- in response to a determination that the packet is an update packet:
- calculating, using the first unmanned platform device, a first quantity based on the packet and respective parameters of the local model corresponding to the packet, and
- updating, using the first unmanned platform device, the parameters of the local model based on the first quantity.
10 more elements
Dependencies are inferred from the claim text.