# US 2022/0036533 A1 · Image defect detection method and apparatus, electronic device, storage medium and product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036533 A1
- Document ID: US20220036533A1
- Status: Published application
- Kind: A1
- Application number: 17/444,883
- Filed: 2021-08-11
- Published: 2022-02-03
- Earliest claimed: 2020-12-25
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science and Technology Co., Ltd.
- Inventors: Huihui XIAO; Lei NIE; Jianfa ZOU; Feng HUANG
- Web page: https://patentgrep.com/patent/US20220036533A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06N 3/045: Combinations of networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 20/00: Machine learning
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/001: {using an image reference approach}
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided are an image defect detection method and apparatus, an electronic device, a storage medium and a product. The method includes acquiring a to-be-detected image; obtaining a restored image corresponding to the to-be-detected image based on the to-be-detected image, at least one mask image group and a plurality of defect-free positive sample images, where each mask image group includes at least two binary images having a complementary relationship, and different mask image groups have different image sizes; and locating a defect of the to-be-detected image based on the to-be-detected image and each restored image. The solution solves the problem in which a related defect detection method requires numerous manual operations and has a low detection accuracy due to subjective factors of a worker.

## Claims

### Claim 1

An image defect detection method, comprising:
acquiring a to-be-detected image;
obtaining at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, at least one mask image group and a plurality of defect-free positive sample images, wherein each mask image group of the at least one mask image group comprises at least two binary images having a complementary relationship, and different mask image groups of the at least one mask image group have different image sizes; and
locating a defect of the to-be-detected image based on the to-be-detected image and each of the at least one restored image.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining the at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, the at least one mask image group and the plurality of defect-free positive sample images comprises:
acquiring a currently processed target mask image group and combining the to-be-detected image with a plurality of binary images in the target mask image group separately to obtain a plurality of restoration data pairs;
inputting the plurality of restoration data pairs to a pretrained image completion model to form a plurality of restored pictures corresponding to the target mask image group, wherein the image completion model is obtained by training the plurality of positive sample images; and
merging the plurality of restored pictures to obtain a restored image corresponding to the target mask image group.

### Claim 3 (depends on 2)

The method of claim 2, wherein merging the plurality of restored pictures to obtain the restored image corresponding to the target mask image group comprises:
multiplying each restored picture of the plurality of restored pictures by a binary image comprised in the target mask image group and corresponding to the each restored picture to obtain a plurality of reference images; and
adding up the plurality of reference images to obtain the restored image corresponding to the target mask image group.

### Claim 4 (depends on 2)

The method of claim 2, before inputting the plurality of restoration data pairs to the pretrained image completion model, the method further comprising:
randomly generating a plurality of binary images; and
unsupervisedly training a set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model.

### Claim 5 (depends on 4)

The method of claim 4, wherein unsupervisedly training the set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model comprises:
selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images and multiplying each pixel of the one positive sample image by a corresponding pixel of the one binary image to obtain one defective image;
merging the one defective image with the selected one binary image to obtain one four-channel target image;
inputting the one target image to the machine learning model to train the machine learning model; and
repeatedly performing the step of selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images to the step of inputting the one target image to the machine learning model to train the machine learning model until a training termination condition is satisfied, so as to obtain the image completion model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the at least one restored image comprises a plurality of restored images, and wherein locating the defect of the to-be-detected image based on the to-be-detected image and the each restored image comprises:
comparing the to-be-detected image with the plurality of restored images separately to obtain a plurality of difference thermodynamic diagrams;
using a maximum value of each thermodynamic point in the plurality of difference thermodynamic diagrams to construct a target thermodynamic diagram; and
performing binary segmentation of the target thermodynamic diagram to obtain a defect area in the to-be-detected image.

### Claim 7 (depends on 6)

The method of claim 6, wherein comparing the to-be-detected image with the plurality of restored images separately to obtain the plurality of difference thermodynamic diagrams comprises:
determining a gradient magnitude similarity deviation between the to-be-detected image and the plurality of restored images separately and obtaining the plurality of difference thermodynamic diagrams based on a determination result of the similarity deviation.

### Claim 8

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the following steps:
acquiring a to-be-detected image;
obtaining at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, at least one mask image group and a plurality of defect-free positive sample images, wherein each mask image group of the at least one mask image group comprises at least two binary images having a complementary relationship, and different mask image groups of the at least one mask image group have different image sizes; and
locating a defect of the to-be-detected image based on the to-be-detected image and each of the at least one restored image.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the instructions are configured to enable the at least one processor to perform the obtaining the at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, the at least one mask image group and the plurality of defect-free positive sample images by:
acquiring a currently processed target mask image group and combining the to-be-detected image with a plurality of binary images in the target mask image group separately to obtain a plurality of restoration data pairs;
inputting the plurality of restoration data pairs to a pretrained image completion model to form a plurality of restored pictures corresponding to the target mask image group, wherein the image completion model is obtained by training the plurality of positive sample images; and
merging the plurality of restored pictures to obtain a restored image corresponding to the target mask image group.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the instructions are configured to enable the at least one processor to perform the merging the plurality of restored pictures to obtain the restored image corresponding to the target mask image group by:
multiplying each restored picture of the plurality of restored pictures by a binary image comprised in the target mask image group and corresponding to the each restored picture to obtain a plurality of reference images; and
adding up the plurality of reference images to obtain the restored image corresponding to the target mask image group.

### Claim 11 (depends on 9)

The electronic device of claim 9, wherein the instructions are configured to enable the at least one processor to further perform, before inputting the plurality of restoration data pairs to the pretrained image completion model, the following steps:
randomly generating a plurality of binary images; and
unsupervisedly training a set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the instructions are configured to enable the at least one processor to perform the unsupervisedly training the set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model by:
selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images and multiplying each pixel of the one positive sample image by a corresponding pixel of the one binary image to obtain one defective image;
merging the one defective image with the selected one binary image to obtain one four-channel target image;
inputting the one target image to the machine learning model to train the machine learning model; and
repeatedly performing the step of selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images to the step of inputting the one target image to the machine learning model to train the machine learning model until a training termination condition is satisfied, so as to obtain the image completion model.

### Claim 13 (depends on 8)

The electronic device of claim 8, wherein the at least one restored image comprises a plurality of restored images, and wherein the instructions are configured to enable the at least one processor to perform the locating the defect of the to-be-detected image based on the to-be-detected image and the each restored image by:
comparing the to-be-detected image with the plurality of restored images separately to obtain a plurality of difference thermodynamic diagrams;
using a maximum value of each thermodynamic point in the plurality of difference thermodynamic diagrams to construct a target thermodynamic diagram; and
performing binary segmentation of the target thermodynamic diagram to obtain a defect area in the to-be-detected image.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein the instructions are configured to enable the at least one processor to perform comparing the to-be-detected image with the plurality of restored images separately to obtain the plurality of difference thermodynamic diagrams by:
determining a gradient magnitude similarity deviation between the to-be-detected image and the plurality of restored images separately and obtaining the plurality of difference thermodynamic diagrams based on a determination result of the similarity deviation.

### Claim 15

A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the following steps:
acquiring a to-be-detected image;
obtaining at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, at least one mask image group and a plurality of defect-free positive sample images, wherein each mask image group of the at least one mask image group comprises at least two binary images having a complementary relationship, and different mask image groups of the at least one mask image group have different image sizes; and
locating a defect of the to-be-detected image based on the to-be-detected image and each of the at least one restored image.

### Claim 16 (depends on 15)

The storage medium of claim 15, wherein the computer instructions are configured to cause a computer to perform the obtaining the at least one restored image corresponding to the to-be-detected image based on the to-be-detected image, the at least one mask image group and the plurality of defect-free positive sample images by:
acquiring a currently processed target mask image group and combining the to-be-detected image with a plurality of binary images in the target mask image group separately to obtain a plurality of restoration data pairs;
inputting the plurality of restoration data pairs to a pretrained image completion model to form a plurality of restored pictures corresponding to the target mask image group, wherein the image completion model is obtained by training the plurality of positive sample images; and
merging the plurality of restored pictures to obtain a restored image corresponding to the target mask image group.

### Claim 17 (depends on 16)

The storage medium of claim 16, wherein the computer instructions are configured to cause a computer to perform the merging the plurality of restored pictures to obtain the restored image corresponding to the target mask image group by:
multiplying each restored picture of the plurality of restored pictures by a binary image comprised in the target mask image group and corresponding to the each restored picture to obtain a plurality of reference images; and
adding up the plurality of reference images to obtain the restored image corresponding to the target mask image group.

### Claim 18 (depends on 16)

The storage medium of claim 16, wherein the computer instructions are configured to cause a computer to further perform, before inputting the plurality of restoration data pairs to the pretrained image completion model, the following steps:
randomly generating a plurality of binary images; and
unsupervisedly training a set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model.

### Claim 19 (depends on 18)

The storage medium of claim 18, wherein the computer instructions are configured to cause a computer to perform the unsupervisedly training the set machine learning model based on the plurality of positive sample images and the plurality of binary images to generate the image completion model by:
selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images and multiplying each pixel of the one positive sample image by a corresponding pixel of the one binary image to obtain one defective image;
merging the one defective image with the selected one binary image to obtain one four-channel target image;
inputting the one target image to the machine learning model to train the machine learning model; and
repeatedly performing the step of selecting one positive sample image of the plurality of positive sample images and one binary image of the plurality of binary images to the step of inputting the one target image to the machine learning model to train the machine learning model until a training termination condition is satisfied, so as to obtain the image completion model.

### Claim 20 (depends on 15)

The storage medium of claim 15, wherein the at least one restored image comprises a plurality of restored images, and wherein the computer instructions are configured to cause a computer to perform the locating the defect of the to-be-detected image based on the to-be-detected image and the each restored image by:
comparing the to-be-detected image with the plurality of restored images separately to obtain a plurality of difference thermodynamic diagrams;
using a maximum value of each thermodynamic point in the plurality of difference thermodynamic diagrams to construct a target thermodynamic diagram; and
performing binary segmentation of the target thermodynamic diagram to obtain a defect area in the to-be-detected image.
