Apparatus and method with compressed neural network computation
AssigneesSamsung Electronics Co., Ltd., Seoul National University R&Db Foundation
InventorsSeon Min RHEE, Jaekyeom KIM, Gunhee KIM, Minjung KIM, Dongyeon WOO, Seungju HAN
Life of this patent
Estimating the term…
Abstract
A neural network includes a drop layer configured to drops feature values. A method of computation using the neural network includes extracting feature data from input data using a first portion of a neural network, generating compressed representation data of the extracted feature data by dropping a feature value from the extracted feature data at a drop layer of the neural network based on a drop probability corresponding to the feature value, and indicating an inference result from the compressed representation data using a second portion of the neural network.
Claims25
A processor-implemented method, the method comprising:
- extracting feature data from input data using a first portion of a neural network;
- generating compressed representation data of the extracted feature data by dropping a feature value from the extracted feature data at a drop layer of the neural network based on a drop probability corresponding to the feature value;
- and
- indicating an inference result from the compressed representation data using a second portion of the neural network.
A processor-implemented method of training a neural network, the method comprising:
- generating temporary compressed representation data by dropping a feature value of feature data extracted from a training input based on a drop probability corresponding to the feature value using a drop layer of the neural network;
- calculating an objective function value using a prediction term based on the temporary compressed representation data and a target task and a compression term based on the feature data and the temporary compressed representation data;
- and
- updating a parameter of the drop layer based on the objective function value.
A computing apparatus comprising:
- a memory configured to store a neural network comprising a drop layer;
- and
- a processor configured to extract feature data from input data using a first portion of the neural network, to generate compressed representation data of the extracted feature data by dropping a feature value from the extracted feature data at the drop layer based on a drop probability corresponding to the feature value, and to indicate an inference result from the compressed representation data using a second portion of the neural network.
Dependencies are inferred from the claim text.