Parallelized Machine Learning With Distributed Lockless Training
AssigneeNEC Laboratories America, Inc.
InventorsAsim Kadav, Erik Kruus, Hao Li
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed for providing distributed learning over a plurality of parallel machine network nodes by allocating a per-sender receive queue at every machine network node and performing distributed in-memory training; and training each unit replica and maintaining multiple copies of the unit replica being trained, wherein all unit replicas train, receive unit updates and merge in parallel in a peer-to-peer fashion, wherein each receiving machine network node merges updates at later point in time without interruption and wherein the propagating and synchronizing unit replica updates are lockless and asynchronous.
Claims21
A method for providing distributed learning over a plurality of parallel machine network nodes, comprising:
- training the parallel machine network nodes in a peer-to-peer fashion and allocating a per-sender receive queue at every machine network node and performing distributed in-memory training;
- and
- training each unit replica and maintaining multiple copies of the unit replica being trained, wherein all unit replicas train, receive unit updates and merge in parallel in a peer-to-peer fashion, wherein each receiving machine network node merges updates at a later point in time without interruption and wherein the propagating and synchronizing unit replica updates are lockless and asynchronous.
The method of claim 3, wherein no replicas wait for one-another to merge or receive units.
A plurality of learning machine network nodes, comprising:
- a processor in communication with a plurality of parallel machine network nodes;
- computer readable code for allocating a per-sender receive queue at every machine network node and performing distributed in-memory training;
- and
- computer readable code for training a unit replica and maintaining multiple copies of the unit replica being trained, wherein all unit replicas train, receive unit updates and merge in parallel in a peer-to-peer fashion, wherein each receiving machine network node merges updates at later point in time without interruption and wherein the propagating and synchronizing unit replica updates are lockless and asynchronous.
Dependencies are inferred from the claim text.