Back to results
US 12,725,024 B2Granted
Accelerator system for training deep neural network model using NAND flash memory and operating method thereof
AssigneeSeoul National University R&Db Foundation
InventorsJae W. Lee, Yunho Jin, Jong Hyun Bae, Gin A Sohn, Tae Jun Ham
Life of this patent
Estimating the term…
Abstract
A DNN accelerator system includes a plurality of accelerator nodes each including a plurality of NAND flash memories, a flash memory system (FMS) controller for controlling the plurality of NAND flash memories, and a tensor buffer, and a processor configured to generate an operation sequence of the plurality of accelerator nodes, in which a DNN model is trained in a data parallel manner using the plurality of accelerator nodes.
Claims20
A deep neural network (DNN) accelerator system comprising:
- a plurality of accelerator nodes each including a plurality of NAND flash memories, a flash memory system (FMS) controller for controlling the plurality of NAND flash memories, and a tensor buffer;
- and
- a processor configured to generate an operation sequence of the plurality of accelerator nodes,
- wherein a DNN model is trained in a data parallel manner using the plurality of accelerator nodes,
- wherein the plurality of accelerator nodes are configured to participate in a multi-phase training process including generation of intermediate training data and updating of model parameters across multiple layers of the DNN model,
- wherein, during training of the DNN model:
- intermediate training data generated by execution of one or more layers of the DNN model is stored in the plurality of NAND flash memories and retained for reuse during subsequent training operations;
- different classes of training data generated during training are stored and managed differently within the plurality of NAND flash memories based on a role of the training data in the training process;
- and
- the FMS controller manages allocation and access to the plurality of NAND flash memories based on a training phase associated with the stored training data.
7 more elements
A method of training a DNN model, the method comprising:
- a forward propagation step in which, while iterative training is performed for one or more layers of the DNN model, one or more activation nodes and a weight node perform an operation in each of the one or more layers, wherein intermediate training data generated during the forward propagation step is stored in non-volatile memory and retained for reuse during subsequent training operations;
- a back propagation step in which the one or more activation nodes and the weight node generate gradient data according to the operation in response to each forward propagation step, wherein gradient data generated during the back propagation step is stored separately from the intermediate training data in the non-volatile memory;
- and
- a step of updating, by the weight node, a final weight based on final gradient data in response to completion of operations of all the layers, wherein storage and access to the non-volatile memory are managed based on a training phase associated with the stored training data.
1 more element
Dependencies are inferred from the claim text.