# US 2023/0068381 A1 · Method and electronic device for quantizing dnn model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0068381 A1
- Document ID: US20230068381A1
- Status: Published application
- Kind: A1
- Application number: 17/961,453
- Filed: 2022-10-06
- Published: 2023-03-02
- Earliest claimed: 2021-04-09
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Tejpratap Venkata Subbu Lakshmi GOLLANAPALLI; Arun ABRAHAM; Raja KUMAR; Pradeep NELAHONNE SHIVAMURTHAPPA; Vikram Nelvoy RAJENDIRAN; Prasen Kumar SHARMA
- Web page: https://patentgrep.com/patent/US20230068381A1

## CPC

- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06V 10/82: using neural networks
- G06N 3/063: using electronic means
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Various embodiments of the disclosure disclose a method for quantizing a Deep Neural Network (DNN) model in an electronic device. The method includes: estimating, by the electronic device, an activation range of each layer of the DNN model using self-generated data (e.g. retro image, audio, video, etc.) and/or a sensitive index of each layer of the DNN model; quantizing, by the electronic device, the DNN model based on the activation range and/or the sensitive index; and allocating, by the electronic device, a dynamic bit precision for each channel of each layer of the DNN model to quantize the DNN model.

## Claims

### Claim 1

A method for quantizing a Deep Neural Network (DNN) model in an electronic device, the method comprising:
estimating, by the electronic device, at least one of an activation range of each layer of the DNN model using self-generated data and a sensitive index of each layer of the DNN model; and
quantizing, by the electronic device, the DNN model based on the at least one of the activation range and the sensitive index.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein the self-generated data is generated based on at least one of layer statistics of the DNN model and weight distributions of the DNN model.

### Claim 3 (depends on 1)

The method as claimed in claim 1, wherein the DNN model quantizes weights and activation at lower bit precision without access to at least one of training dataset and validation dataset to obtain a compression of the DNN model and a fast inference of the DNN model.

### Claim 4 (depends on 1)

The method as claimed in claim 1, wherein estimating, by the electronic device, the activation range of each layer of the DNN model using the self-generated data comprises:
determining, by the electronic device, a plurality of random images, wherein each random image of the plurality of random images comprises uniform distribution data across the images;
passing, by the electronic device, each random image into the DNN model;
determining, by the electronic device, weight distributions of the DNN model for each random image after each layer of the DNN model;
determining, by the electronic device, layer statistics of the DNN model for each random image after each layer of the DNN model, wherein the layer statistics of the DNN model comprises at least one of a mean and a variance;
determining, by the electronic device, a difference between pre-stored layer statistics of the DNN model and the determined layer statistics of the DNN model;
determining, by the electronic device, whether the difference is less than a threshold; and
performing, by the electronic device, one of:
generating the data using at least one of the layer statistics of the DNN model and the weight distributions of the DNN model in response to determining that the difference is less than the threshold, or
executing back propagation in the DNN model in response to determining that the difference is greater than or equal to the threshold.

### Claim 5 (depends on 4)

The method as claimed in claim 4, wherein the self-generated data is a plurality of retro data images, wherein the plurality of retro data images are equivalent or represent all features of the DNN model.

### Claim 6 (depends on 1)

The method as claimed in claim 1, wherein estimating, by the electronic device, the activation range of each layer of the DNN model using the self-generated data comprises:
determining, by the electronic device, a plurality of random images, wherein each random image of the plurality of random images comprises uniform distribution data across the images;
passing, by the electronic device, each random image into the DNN model;
determining, by the electronic device, weight distributions of the DNN model for each random image after each layer of the DNN model, wherein the weight distributions at each layer of the DNN model comprises a Gaussian distribution;
determining, by the electronic device, layer statistics of the DNN model for each random image after each layer of the DNN model using a Z-score, wherein the layer statistics of the DNN model comprises at least one of a mean and a variance;
determining, by the electronic device\, a difference in pre-stored layer statistics of the DNN model and the determined layer statistics of the DNN model;
determining, by the electronic device, whether the difference is less than a threshold; and
performing, by the electronic device, one of:
generating the data using the at least one of the layer statistics of the DNN model and the weight distributions of the DNN model in response to determining that the difference is less than the threshold, or
executing back propagation in the DNN model in response to determining that the difference is greater than equal to the threshold.

### Claim 7 (depends on 6)

The method as claimed in claim 6, wherein the Z-score provides a difference between two distributions.

### Claim 8 (depends on 1)

The method as claimed in claim 1, wherein the self-generated data is a plurality of retro data images, wherein the plurality of retro data images are equivalent or represent all features of the DNN model.

### Claim 9 (depends on 1)

The method as claimed in claim 1, wherein estimating, by the electronic device, the sensitive index of each layer of the DNN model comprises:
determining, by the electronic device, an accuracy using per-channel quantization and per-tensor quantization schemes at each layer of the DNN model; and
determining, by the electronic device, a sensitive index of each layer corresponding to per-channel quantization and per-tensor quantization.

### Claim 10 (depends on 1)

The method as claimed in claim 1, wherein quantizing, by the electronic device, the DNN model based the sensitive index comprises:
determining, by the electronic device, an optimal quantization for each layer of the DNN model based on the sensitive index to quantize the DNN model, wherein the sensitive index comprises a minimum sensitivity; and
applying, by the electronic device, the optimal quantization for each layer of the DNN model.

### Claim 11 (depends on 1)

The method as claimed in claim 1, wherein the sensitive index is determined a using a Kullback-Leibler Divergence.

### Claim 12 (depends on 10)

The method as claimed in claim 10, wherein the optimal quantization comprises at least one of per-channel quantization, and per-tensor quantization for the DNN model.

### Claim 13 (depends on 1)

The method as claimed in claim 1, wherein the sensitive index is used to reduce a search space of each layer of the DNN model from exponential to linear, wherein n is a number of layers in the DNN model.

### Claim 14 (depends on 1)

The method as claimed in claim 1, wherein the method comprises:
allocating, by the electronic device, a dynamic bit precision for each channel of each layer of the DNN model, wherein the allocated dynamic bit precision minimizes and/or reduces overall quantization noise of each layer.

### Claim 15

An electronic device configured to quantize a Deep Neural Network (DNN) model, the electronic device comprising:
a memory;
a processor; and
a quantization engine comprising processing circuitry and/or executable program instructions, operably connected to the memory and the processor, configured to:
estimate at least one of an activation range of each layer of the DNN model using self-generated data and a sensitive index of each layer of the DNN model; and
quantize the DNN model based on the at least one of the activation range and the sensitive index.
