# US 2023/0401834 A1 · Image processing method, apparatus and device, and readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0401834 A1
- Document ID: US20230401834A1
- Status: Published application
- Kind: A1
- Application number: 18/032,125
- Filed: 2021-07-29
- Published: 2023-12-14
- Earliest claimed: 2020-10-16
- Assignees: Inspur (Beijing) Electronic Information Industry Co., Ltd.
- Applicants: Inspur (Beijing) Electronic Information Industry Co., Ltd
- Inventors: Lingyan LIANG; Dong GANG; Yaqian ZHAO; Qichun CAO; Wenfeng YIN
- Web page: https://patentgrep.com/patent/US20230401834A1

## CPC

- G06V 10/82: using neural networks
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06V 10/40: Extraction of image or video features
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/98: Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

An image processing method, apparatus and device, and a readable storage medium are disclosed, including: obtaining a target image; inputting the target image into a quantized target deep neural network model for classification/detection to obtain an output result; and processing the target image according to a policy corresponding to the output result. A process of performing quantization to obtain the target deep neural network model includes: obtaining a pre-trained floating point type deep neural network model; extracting weight features of a deep neural network model; determining a quantization policy using the weight features; and quantizing the deep neural network model according to the quantization policy to obtain the target deep neural network model.

## Claims

### Claim 1

An image processing method, comprising:
obtaining a target image;
inputting the target image into a quantized target deep neural network model for at least one of classification or detection to obtain an output result; and
processing the target image according to a policy corresponding to the output result; and
performing quantization to obtain the target deep neural network model by:
obtaining a pre-trained floating point type deep neural network model;
extracting weight features of a deep neural network model;
determining a quantization policy using the weight features; and
quantizing the deep neural network model according to the quantization policy to obtain the target deep neural network model.

### Claim 2 (depends on 1)

The image processing method according to claim 1, wherein under a circumstance that the weight features are weight parameter features, extracting the weight features of the deep neural network model comprises:
calculating a per-layer quantization error and a per-channel quantization error corresponding to the deep neural network model separately; and
determining the per-layer quantization error and the per-channel quantization error as the weight parameter features.

### Claim 3 (depends on 2)

The image processing method according to claim 2, wherein determining the quantization policy using the weight features comprises:
calculating a difference value between the per-layer quantization error and the per-channel quantization error;
determining whether the difference value is within an error range;
in response to the difference value being within the error range, determining that the quantization policy is the per-layer quantization; and
in response to the difference value being not within the error range, determining that the quantization policy is the per-channel quantization, or performing quantization according to a certain quantization unit.

### Claim 4 (depends on 3)

The image processing method according to claim 3, wherein before determining that the quantization policy is the per-channel quantization, or performing quantization according to the certain quantization unit, the method further comprises:
extracting weight channel features of the deep neural network model; and
taking similar weight channels in the deep neural network model as the certain quantization unit by using the weight channel features, wherein the certain quantization unit comprises at least one weight channel.

### Claim 5 (depends on 1)

The image processing method according to claim 1, wherein under a circumstance that the weight features are weight channel features, correspondingly, extracting the weight features of the deep neural network model comprises:
clustering various weight channels in the deep neural network model to obtain clustering results; and
determining the clustering results as the weight channel features.

### Claim 6 (depends on 5)

The image processing method according to claim 5, wherein clustering the various weight channels in the deep neural network model to obtain the clustering results comprises:
obtaining weight value ranges of the various weight channels in the deep neural network model, and a statistical distribution chart of weight parameter values of the various weight channels; and
clustering the various weight channels based on the weight value ranges and the statistical distribution chart of the weight parameter values, to obtain the clustering results.

### Claim 7 (depends on 5)

The image processing method according to claim 5, wherein the determining a quantization policy using the weight features comprises:
in response to the number of clusters of the clustering results being 1, determining that the quantization policy is the per-layer quantization;
in response to the number of clusters being equal to a number of the weight channels, determining that the quantization policy is the per-channel quantization; and
in response to the number of clusters being greater than 1 and less than the number of the weight channels, taking all the weight channels corresponding to each cluster as an entirety for quantization.

### Claim 8

(canceled)

### Claim 9

An image processing device, comprising:
a memory storing a computer program; and
a processor configured to execute the computer program wherein the processor, upon execution of the computer program, is configured to:
obtain a target image;
input the target image into a quantized target deep neural network model for at least one of classification or detection to obtain an output result;
process the target image according to a policy corresponding to the output result; and
perform quantization to obtain the target deep neural network model by:
obtaining a pre-trained floating point type deep neural network model;
extracting weight features of a deep neural network model;
determining a quantization policy using the weight features; and
quantizing the deep neural network model according to the quantization policy to obtain the target deep neural network model.

### Claim 10

A non-transitory computer-readable storage medium storing a computer program; and the computer program, when executed by a processor, is configured to cause the processor to:
obtain a target image;
input the target image into a quantized target deep neural network model for at least one of classification or detection to obtain an output result;
process the target image according to a policy corresponding to the output result; and
perform quantization to obtain the target deep neural network model by:
obtaining a pre-trained floating point type deep neural network model;
extracting weight features of a deep neural network model;
determining a quantization policy using the weight features; and
quantizing the deep neural network model according to the quantization policy to obtain the target deep neural network model.

### Claim 11 (depends on 9)

The image processing device according to claim 9, wherein the processor, upon execution of the computer program, is further configured to:
calculate a per-layer quantization error and a per-channel quantization error corresponding to the deep neural network model separately; and
determine the per-layer quantization error and the per-channel quantization error as the weight parameter features.

### Claim 12 (depends on 11)

The image processing device according to claim 11, wherein the processor, upon execution of the computer program, is further configured to:
calculate a difference value between the per-layer quantization error and the per-channel quantization error;
determine whether the difference value is within an error range;
in response to the difference value being within the error range, determine that the quantization policy is the per-layer quantization; and
in response to the difference value being not within the error range, determine that the quantization policy is the per-channel quantization, or perform quantization according to a certain quantization unit.

### Claim 13 (depends on 12)

The image processing device according to claim 12, wherein the processor, upon execution of the computer program, is further configured to:
extract weight channel features of the deep neural network model; and
take similar weight channels in the deep neural network model as the certain quantization unit by using the weight channel features, wherein the certain quantization unit comprises at least one weight channel.

### Claim 14 (depends on 9)

The image processing device according to claim 9, wherein the processor, upon execution of the computer program, is further configured to:
cluster various weight channels in the deep neural network model to obtain clustering results; and
determine the clustering results as the weight channel features.

### Claim 15 (depends on 14)

The image processing device according to claim 14, wherein the processor, upon execution of the computer program, is further configured to:
obtain weight value ranges of the various weight channels in the deep neural network model, and a statistical distribution chart of weight parameter values of the various weight channels; and
cluster the various weight channels based on the weight value ranges and the statistical distribution chart of the weight parameter values, to obtain the clustering results.

### Claim 16 (depends on 14)

The image processing device according to claim 14, wherein the processor, upon execution of the computer program, is further configured to:
in response to the number of clusters of the clustering results being 1, determine that the quantization policy is the per-layer quantization;
in response to the number of clusters being equal to a number of the weight channels, determine that the quantization policy is the per-channel quantization; and
in response to the number of clusters being greater than 1 and less than the number of the weight channels, take all the weight channels corresponding to each cluster as an entirety for quantization.

### Claim 17 (depends on 10)

The non-transitory computer-readable storage medium according to claim 10, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
calculate a per-layer quantization error and a per-channel quantization error corresponding to the deep neural network model separately; and
determine the per-layer quantization error and the per-channel quantization error as the weight parameter features.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
calculate a difference value between the per-layer quantization error and the per-channel quantization error;
determine whether the difference value is within an error range;
in response to the difference value being within the error range, determine that the quantization policy is the per-layer quantization; and
in response to the difference value being not within the error range, determine that the quantization policy is the per-channel quantization, or perform quantization according to a certain quantization unit.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
extract weight channel features of the deep neural network model; and
take similar weight channels in the deep neural network model as the certain quantization unit by using the weight channel features, wherein the certain quantization unit comprises at least one weight channel.

### Claim 20 (depends on 10)

The non-transitory computer-readable storage medium according to claim 10, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
cluster various weight channels in the deep neural network model to obtain clustering results; and
determine the clustering results as the weight channel features.

### Claim 21 (depends on 20)

The non-transitory computer-readable storage medium according to claim 20, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
obtain weight value ranges of the various weight channels in the deep neural network model, and a statistical distribution chart of weight parameter values of the various weight channels; and
cluster the various weight channels based on the weight value ranges and the statistical distribution chart of the weight parameter values, to obtain the clustering results.
