# US 2022/0036192 A1 · Image processing method, electronic device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036192 A1
- Document ID: US20220036192A1
- Status: Published application
- Kind: A1
- Application number: 17/386,326
- Filed: 2021-07-27
- Published: 2022-02-03
- Earliest claimed: 2020-07-28
- Assignees: Hon Hai Precision Industry Co., Ltd.
- Applicants: Hon Hai Precision Industry Co., Ltd.
- Inventors: I-HUA CHEN; TUNG-TSO TSAI; GUO-CHIN SUN; TZU-CHEN LIN; WAN-JHEN LEE; CHIN-PIN KUO
- Web page: https://patentgrep.com/patent/US20220036192A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06V 10/82: using neural networks
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/063: using electronic means
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An image processing method and an electronic device are disclosed, the method acquires training data in response to receiving an image processing instruction and trains a deep learning model with the training data by using a preset deep learning framework to obtain an initial model. A data type of the initial model is converted to increase the data processing speed of the model. A correction layer is added to the converted initial model and the initial model is optimized by training weights of the correction layer to obtain an image processing model to further increase the data processing speed. After acquiring an image from the image processing instruction, the image processing model can be used to process the image and an image processing result can be outputted. The image is processed based on the optimized image processing model, and the data processing speed and accuracy are guaranteed.

## Claims

### Claim 1

An image processing method, comprising:
acquiring training data in response to receiving an image processing instruction;
training a deep learning model with the training data by using a preset deep learning framework to obtain an initial model;
converting a data type of the initial model;
adding a correction layer to the converted initial model;
optimizing the initial model by training weights of the correction layer to obtain an image processing model;
acquiring an image from the image processing instruction; and
using the image processing model to process the image and output an image processing result.

### Claim 2 (depends on 1)

The image processing method according to claim 1, wherein converting a data type of the initial model comprises:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data; and
converting a data type of the initial model into the target data type.

### Claim 3 (depends on 2)

The image processing method of claim 2, wherein determining the target data type according to the demand data comprises:
determining a required speed of data processing and a required accuracy according to the demand data;
determining that the target data type is int8, required speed of data processing is greater than or equal to a preset speed; and
determining that the target data type is a half-precision, if the required accuracy is greater than or equal to a preset accuracy.

### Claim 4 (depends on 3)

The image processing method of claim 3, wherein a speed of data processing of the model with the int8 data type is higher than that of the model with the half-precision data type, and an accuracy of the model with the int8 data type is lower than that of the model with the half-precision data type.

### Claim 5 (depends on 1)

The image processing method of claim 1, wherein optimizing the initial model by training weights of the correction layer to obtain an image processing model comprises:
training weights of the correction layer by fixing weights of each layer in the initial model; and
if an accuracy of the initial model is greater than or equal to a preset threshold, stopping the training to obtain the image processing model.

### Claim 6 (depends on 1)

The image processing method according to claim 1, wherein optimizing the initial model by training weights of the correction layer to obtain an image processing model comprises:
training the weights of the correction layer with full accuracy data type to obtain the image processing model.

### Claim 7 (depends on 1)

The image processing method according to claim 1, further comprising:
constructing a validation data set; and
verifying the image processing model with the validation data set.

### Claim 8

A electronic device comprising a memory and a processor, the memory stores at least one computer-readable instruction, and the processor executes the at least one computer-readable instruction to:
acquire training data in response to receiving an image processing instruction;
train a deep learning model with the training data by using a preset deep learning framework to obtain an initial model;
convert a data type of the initial model;
add a correction layer to the converted initial model;
optimize the initial model by training weights of the correction layer to obtain an image processing model;
acquire an image from the image processing instruction; and
use the image processing model to process the image and output an image processing result.

### Claim 9 (depends on 8)

The electronic device according to claim 8, wherein the processor converts a data type of the initial model by:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data; and
converting a data type of the initial model into the target data type.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the processor determines the target data type according to the demand data by:
determining a required speed of data processing and a required accuracy according to the demand data;
determining that the target data type is int8, required speed of data processing is greater than or equal to a preset speed; and
determining that the target data type is a half-precision, if the required accuracy is greater than or equal to a preset accuracy.

### Claim 11 (depends on 10)

The electronic device according to claim 10, wherein a speed of data processing of the model with the int8 data type is higher than that of the model with the half-precision data type, and an accuracy of the model with the int8 data type is lower than that of the model with the half-precision data type.

### Claim 12 (depends on 8)

The electronic device according to claim 8, wherein the processor optimizing the initial model by training weights of the correction layer to obtain an image processing model by:
training weights of the correction layer by fixing weights of each layer in the initial model; and
in response that an accuracy of the initial model is greater than or equal to a preset threshold, stopping the training to obtain the image processing model.

### Claim 13 (depends on 8)

The electronic device according to claim 8, wherein the processor optimizing the initial model by training weights of the correction layer to obtain an image processing model further by:
training the weights of the correction layer with full accuracy data type to obtain the image processing model.

### Claim 14 (depends on 8)

The electronic device according to claim 8, wherein the processor is further caused to:
construct a validation data set; and
verify the image processing model with the validation data set.

### Claim 15

A non-transitory storage medium having stored thereon at least one computer-readable instructions that, when the at least one computer-readable instructions are executed by a processor to implement the following steps:
acquiring training data in response to receiving an image processing instruction;
training a deep learning model with the training data by using a preset deep learning framework to obtain an initial model;
converting a data type of the initial model;
adding a correction layer to the converted initial model;
optimizing the initial model by training weights of the correction layer to obtain an image processing model;
acquiring an image from the image processing instruction; and
using the image processing model to process the image and output an image processing result.

### Claim 16 (depends on 15)

The non-transitory storage medium according to claim 15, wherein converting a data type of the initial model comprises:
acquiring demand data from the image processing instruction;
determining a target data type according to the demand data; and
converting a data type of the initial model into the target data type.

### Claim 17 (depends on 16)

The non-transitory storage medium according to claim 16, wherein determining the target data type according to the demand data comprises:
determining a required speed of data processing and a required accuracy according to the demand data;
determining that the target data type is int8, required speed of data processing is greater than or equal to a preset speed; and
determining that the target data type is a half-precision, if the required accuracy is greater than or equal to a preset accuracy.

### Claim 18 (depends on 17)

The non-transitory storage medium according to claim 17, wherein a speed of data processing of the model with the int8 data type is higher than that of the model with the half-precision data type, and an accuracy of the model with the int8 data type is lower than that of the model with the half-precision data type.

### Claim 19 (depends on 15)

The non-transitory storage medium according to claim 15, wherein optimizing the initial model by training weights of the correction layer to obtain an image processing model comprises:
training weights of the correction layer by fixing weights of each layer in the initial model;
when an accuracy of the initial model is greater than or equal to a preset threshold, stopping the training to obtain the image processing model.

### Claim 20 (depends on 15)

The non-transitory storage medium according to claim 15, wherein optimizing the initial model by training weights of the correction layer to obtain an image processing model comprises:
training the weights of the correction layer with full accuracy data type to obtain the image processing model.
