# US 2024/0202893 A1 · Method and device for detecting defect, storage medium and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202893 A1
- Document ID: US20240202893A1
- Status: Published application
- Kind: A1
- Application number: 17/772,562
- Filed: 2021-05-21
- Published: 2024-06-20
- Earliest claimed: 2021-05-21
- Assignees: Beijing Zhongxiangying Technology Co., Ltd.; Boe Technology Group Co., Ltd.
- Applicants: Beijing Zhongxiangying Technology Co., Ltd.; BOE Technology Group Co., Ltd.
- Inventors: Yaoping WANG; Meijuan ZHANG; Wangqiang HE; Dong CHAI; Hong WANG
- Web page: https://patentgrep.com/patent/US20240202893A1

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/90: Determination of colour characteristics
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/776: Validation; Performance evaluation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20092: Interactive image processing based on input by user
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided is a method and device for detecting defect, a computer readable storage medium and an electronic device, the method including: acquiring (S310) a detection task, and acquiring various types of images corresponding to the detection task; acquiring (S320) defect detection models trained by a same initial model corresponding to the types of the images respectively; and obtaining (S330) defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images.

## Claims

### Claim 1

A method for detecting defect, comprising:
acquiring a detection task, and acquiring various types of images corresponding to the detection task;
acquiring defect detection models trained by a same initial model corresponding to the types of the images respectively; and
obtaining defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the acquiring various types of images corresponding to the detection task comprises:
obtaining product information corresponding to the detection task; and
obtaining the various types of images according to the product information.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the obtaining the various types of images according to the product information comprises:
obtaining the various types of images corresponding to a same product corresponding to the product information,
wherein the various types of images are obtained by taking pictures of the product using cameras with different configuration parameters, and
wherein the configuration parameters comprise one or more of resolution, color and zoom factor.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the method further comprises:
determining a maximum number of channels in each of the various types of images; and
preprocessing the various types of images and setting a number of channels of each of various types of images to the maximum number of channels.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the various types of images comprise one or more of an AOI color image, a TDI image, and a DM image of the product.

### Claim 6 (depends on 3)

The method according to claim 3, wherein the obtaining defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images comprises:
setting defect influence weights on the various types of images according to the configuration parameters;
obtaining reference defect detection results corresponding respectively to the various types of images by performing defect detection on the various types of images using the defect detection model corresponding to each type of the images; and
determining the defect detection results according to the defect influence weights and the reference defect detection results.

### Claim 7 (depends on 2)

The method according to claim 2, wherein the obtaining product information corresponding to the detection task comprises:
extracting a product information field in the detection task to obtain the product information.

### Claim 8 (depends on 4)

The method according to claim 4, wherein the preprocessing the various types of images further comprises:
determining a resolution threshold according to the defect detection model; and
adjusting resolutions of the various types of images according to the resolution threshold.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising:
providing a preprocessing parameter modification interface for the preprocessing, to allow a user to adjust parameter information of the preprocessing by the preprocessing parameter modification interface.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the defect detection results comprise image normal information or image defect information.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the defect detection model comprises a feature extraction network and a defect identification network, and the obtaining defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images comprises:
obtaining feature images by performing feature extraction on the various types of images using the feature extraction network, and standardizing the feature images;
obtaining a defect category of each feature image by performing defect classifying according to standardized feature images using the defect identification network, and determining coordinates of each feature image; and
obtaining the defect detection result according to the defect category and the coordinates.

### Claim 12 (depends on 11)

The method according to claim 11, further comprising:
providing a standardizing parameter modification interface, to allow a user to adjust parameter information of the standardizing by the standardizing parameter modification interface.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the obtaining the defect detection result according to the defect category and the coordinates comprises:
screening the feature images to obtain a target feature image, and determining the defect category and the coordinates of the target candidate feature image; and
obtaining the defect detection result according to a preset screening strategy and the defect category and the coordinates of the target candidate feature image.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the screening the feature images to obtain the target feature image comprises:
calculating confidence level of each of the feature images; and
screening the feature images by using an NMS algorithm according to the confidence level to obtain the target feature image.

### Claim 15

A device for detecting defect, comprising:
an image acquiring module, configured to acquire a detection task, and acquire various types of images corresponding to the detection task;
a module acquiring module, configured to acquire defect detection models trained by a same initial model corresponding to the types of the images respectively; and
a defect detection module configured to obtain defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images.

### Claim 16 (depends on 1)

A non-transitory computer readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method according to claim 1 is implemented.

### Claim 17

An electrical device comprising:
a processor; and
a memory, configured to store one or more programs, and when the one or more programs are executed by a processor, causes the processor to:
acquire a detection task, and acquiring various types of images corresponding to the detection task;
acquire defect detection models trained by a same initial model corresponding to the types of the images respectively; and
obtain defect detection results by performing defect detection on respective type of images using the defect detection model corresponding to the type of the images.

### Claim 18 (depends on 17)

The electrical device according to claim 17, wherein the processor is further caused to:
obtain product information corresponding to the detection task; and
obtain the various types of images according to the product information.

### Claim 19 (depends on 18)

The electrical device according to claim 18, wherein the processor is further caused to:
obtain the various types of images corresponding to a same product corresponding to the product information,
wherein the various types of images are obtained by taking pictures of the product using cameras with different configuration parameters, and
wherein the configuration parameters comprise one or more of resolution, color and zoom factor.

### Claim 20 (depends on 19)

The electrical device according to claim 19, wherein the processor is further caused to:
determine a maximum number of channels in each of the various types of images; and
preprocess the various types of images and setting a number of channels of each of various types of images to the maximum number of channels.
