Method and apparatus for compressing neural network model by using device characteristics

A method of compressing a neural network model which is performed by a computing device, by using device characteristics, the method comprising:

• obtaining the neural network model that is executed by a device;
• adjusting a target number of output channels of a target sub-layer of a layer to be compressed from among layers included in the neural network model, based on an arithmetic intensity obtained from a roofline model and a latency characteristic of a staircase pattern of the device;
• and
• compressing the neural network model by decomposing the layer to be compressed such that the number of output channels of the target sub-layer is equal to the adjusted target number of output channels;
• wherein the target sub-layer is one of a plurality of sub-layers generated by decomposing the layer to be compressed;
• wherein the adjusting comprises:
• comparing an arithmetic intensity of a target sub-layer with an arithmetic intensity of a next sub-layer which is connected to the target sub-layer;
• selecting, as a reference sub-layer, a sub-layer having a higher arithmetic intensity based on a result of the comparison;
• and
• adjusting the target number of output channels based on a latency characteristic of the selected reference sub-layer, wherein in a case in which the selected reference sub-layer is the target sub-layer, the adjusting is based on a first latency characteristic for the output channels of the target sub-layer and in a case in which the reference sub-layer is the next sub-layer, the adjusting is based on a second latency characteristic for input channels of the next sub-layer.

An apparatus for performing, by a computing device, a compression of a neural network model, by using device characteristics, the apparatus comprising:

• a memory storing at least one program;
• and
• at least one processor configured to execute the neural network model by executing the at least one program, wherein the at least one processor is further configured to:
• obtain the neural network model that is executed by a device;
• adjust a target number of output channels of a target layer included in the neural network model, based on an arithmetic intensity obtained from a roofline model and a latency characteristic of a staircase pattern of the device;
• compare an arithmetic intensity of the target layer with an arithmetic intensity of a next layer which is connected to the target layer;
• select, as a reference layer, a layer having a higher arithmetic intensity based on a result of the comparison;
• adjust the target number of output channels of the target layer based on a first latency characteristic of the selected reference layer, wherein in a case in which the selected reference layer is the target layer, the adjusting is based on a first latency characteristic for the output channels of the target layer and in a case in which the reference layer is the next layer, the adjusting is based on a second latency characteristic for input channels of the next layer;
• and
• compress the neural network model by using at least one of channel pruning and filter decomposition on the target layer to be compressed such that a number of output channels of the target layer is equal to the adjusted target number of output channels.