# US 12,002,197 B2 · Error reduction in reconstructed images in defect detection method, electronic device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,197 B2
- Document ID: US12002197B2
- Status: Granted
- Kind: B2
- Application number: 17/526,314
- Filed: 2021-11-15
- Published: 2022-06-23
- Granted: 2024-06-04
- Earliest claimed: 2020-12-22
- Assignees: Hon Hai Precision Industry Co., Ltd.
- Applicants: Hon Hai Precision Industry Co., Ltd.
- Inventors: Chin-Pin Kuo; Tung-Tso Tsai; Shih-Chao Chien
- Web page: https://patentgrep.com/patent/US12002197B2

## CPC

- G06T 7/001: {using an image reference approach}
- G06T 1/0007: {Image acquisition}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/30148: Semiconductor; IC; Wafer
- Y02P 90/30: Computing systems specially adapted for manufacturing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS

## Abstract

A defect detection method based on an image of products and an electronic device can accurately determine the error threshold by determining the reconstruction error generated during image reconstruction and by determining the estimated probability generated by the Gaussian mixture model. The test error can then be compared with the error, since the test error and the error threshold are compared numerically, the existence of subtle defects are revealed in the product image, thereby improving the accuracy of defect detection.

## Claims

### Claim 1

A defect detection method, the method comprising:
in response to receiving a defect detection request, acquiring a product image and a plurality of positive sample images according to the defect detection request;
encoding the product image to obtain a target vector corresponding to the product image by using an encoder, encoding the plurality of positive sample images to obtain a plurality of latent vectors corresponding to the plurality of positive sample images by using the encoder;
decoding the target vector to obtain a target image corresponding to the product image by using a decoder corresponding to the encoder, decoding the plurality of latent vectors to obtain a plurality of reconstructed images corresponding to the plurality of positive sample images by using the decoder;
comparing the target image with the product image to obtain a target error, comparing each of the plurality of reconstructed images with the corresponding positive sample image to obtain a reconstruction error;
inputting the target vector into a Gaussian mixture model to obtain a test probability of the product image, inputting the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image;
determining a test error of the product image according to the target error and the test probability, and determining a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability;
selecting an error threshold from a plurality of the sample errors and obtaining a detection result of defect detection in the product image according to the test error and the error threshold.

### Claim 2 (depends on 1)

The defect detection method of claim 1, wherein encoding the product image to obtain a target vector corresponding to the product image by using an encoder comprises:
performing a vectorization process on the product image to obtain a feature vector;
acquiring a first hidden layer in the encoder and calculating the feature vector through the acquired first hidden layer to obtain the target vector.

### Claim 3 (depends on 1)

The defect detection method of claim 1, wherein comparing the target image with the product image to obtain a target error comprises:
acquiring all pixels of the product image as a plurality of examinable pixels, and acquiring all pixels of the target image as a plurality of target pixels;
comparing each target pixel with the corresponding examinable pixel and when a comparison result is that a target pixel is different from the corresponding examinable pixel;
calculating a number of differences between the target pixel as being a first number and the corresponding examinable pixel;
calculating a total number of the plurality of target pixels as being a second number;
dividing the first number by the second number to obtain the target error.

### Claim 4 (depends on 1)

The defect detection method of claim 1, wherein inputting the target vector into a Gaussian mixture model to obtain a test probability of the product image and inputting the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image comprises:
inputting the plurality of latent vectors into the Gaussian mixture model to obtain a feature distribution of the plurality of positive sample images;
determining an average value and a covariance of the plurality of latent vectors according to the feature distribution;
acquiring a mixing coefficient of the Gaussian mixture model;
determining a test probability of the product image according to the target vector, the average value, the covariance and the mixing coefficient;
determining the test probability of each positive sample image according to each latent vector, the average value, and the covariance and the mixing coefficient.

### Claim 5 (depends on 1)

The defect detection method of claim 1, wherein determining a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability comprises:
calculating a logarithmic value of each estimated probability;
calculating a weighted sum of an opposite value of the logarithmic value and the corresponding reconstruction error to obtain the sample error of the corresponding positive sample image.

### Claim 6 (depends on 1)

The defect detection method of claim 1, wherein selecting an error threshold from a plurality of the sample errors comprises:
sorting the sample errors in ascending order to obtain a list of errors and a sequence number of each sample error in the list of errors;
calculating a number of the sample errors as being a third number;
multiplying the third number by a configuration value to obtain a target value;
selecting a sample error which has sequence number equal to the target value from the error list and determining the sequence number of the selected sample error as the error threshold.

### Claim 7 (depends on 1)

The defect detection method of claim 1, wherein obtaining a detection result of defect detection in the product image according to the test error and the error threshold comprises:
determining that a detection result is that there are one or more defects in the product image when the test error is less than the error threshold;
determining that a detection result is that there are no defects in the product image when the test error is greater than or equal to the error threshold.

### Claim 8

An electronic device comprising a memory and a processor, the memory stores at least one computer-readable instruction, which when executed by the processor causes the processor to:
in response to receiving a defect detection request, acquire a product image and a plurality of positive sample images according to the defect detection request;
encode the product image to obtain a target vector corresponding to the product image by using an encoder, encode the plurality of positive sample images to obtain a plurality of latent vectors corresponding to the plurality of positive sample images by using the encoder;
decode the target vector to obtain a target image corresponding to the product image by using a decoder corresponding to the encoder, decode the plurality of latent vectors to obtain a plurality of reconstructed images corresponding to the plurality of positive sample images by using the decoder;
compare the target image with the product image to obtain a target error, compare each of the plurality of reconstructed images with the corresponding positive sample image to obtain a reconstruction error;
input the target vector into a Gaussian mixture model to obtain a test probability of the product image, input the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image;
determine a test error of the product image according to the target error and the test probability, and determine a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability;
select an error threshold from a plurality of the sample errors and obtain a detection result of defect detection in the product image according to the test error and the error threshold.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the processor encodes the product image to obtain a target vector corresponding to the product image by using an encoder by:
performing a vectorization process on the product image to obtain a feature vector;
acquiring a first hidden layer in the encoder and calculating the feature vector through the acquired first hidden layer to obtain the target vector.

### Claim 10 (depends on 8)

The electronic device of claim 8, wherein the processor compares the target image with the product image to obtain a target error by:
acquiring all pixels of the product image as a plurality of examinable pixels, and acquiring all pixels of the target image as a plurality of target pixels;
comparing each target pixel with the corresponding examinable pixel and when a comparison result is that a target pixel is different from the corresponding examinable pixel;
calculating a number of differences between the target pixel as being a first number and the corresponding examinable pixel;
calculating a total number of the plurality of target pixels as being a second number;
dividing the first number by the second number to obtain the target error.

### Claim 11 (depends on 8)

The electronic device of claim 8, wherein the processor inputs the target vector into a Gaussian mixture model to obtain a test probability of the product image and inputs the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image by:
inputting the plurality of latent vectors into the Gaussian mixture model to obtain a feature distribution of the plurality of positive sample images;
determining an average value and a covariance of the plurality of latent vectors according to the feature distribution;
acquiring a mixing coefficient of the Gaussian mixture model;
determining a test probability of the product image according to the target vector, the average value, the covariance and the mixing coefficient;
determining the test probability of each positive sample image according to each latent vector, the average value, and the covariance and the mixing coefficient.

### Claim 12 (depends on 8)

The electronic device of claim 8, wherein the processor determines a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability by:
calculating a logarithmic value of each estimated probability;
calculating a weighted sum of an opposite value of the logarithmic value and the corresponding reconstruction error to obtain the sample error of the corresponding positive sample image.

### Claim 13 (depends on 8)

The electronic device of claim 8, wherein the processor selects an error threshold from a plurality of the sample errors by:
sorting the sample errors in ascending order to obtain a list of errors and a sequence number of each sample error in the list of errors;
calculating a number of the sample errors as being a third number;
multiplying the third number by a configuration value to obtain a target value;
selecting a sample error which has sequence number equal to the target value from the error list and determining the sequence number of the selected sample error as the error threshold.

### Claim 14 (depends on 8)

The electronic device of claim 8, wherein the processor obtains a detection result of defect detection in the product image according to the test error and the error threshold by:
determining that a detection result is that there are one or more defects in the product image when the test error is less than the error threshold;
determining that a detection result is that there are no defects in the product image when the test error is greater than or equal to the error threshold.

### Claim 15

A non-transitory storage medium having stored thereon at least one computer-readable instructions that, when the at least one computer-readable instructions are executed by a processor to implement a defect detection method, which comprises:
in response to receiving a defect detection request, acquiring a product image and a plurality of positive sample images according to the defect detection request;
encoding the product image to obtain a target vector corresponding to the product image by using an encoder, encoding the plurality of positive sample images to obtain a plurality of latent vectors corresponding to the plurality of positive sample images by using the encoder;
decoding the target vector to obtain a target image corresponding to the product image by using a decoder corresponding to the encoder, decoding the plurality of latent vectors to obtain a plurality of reconstructed images corresponding to the plurality of positive sample images by using the decoder;
comparing the target image with the product image to obtain a target error, comparing each of the plurality of reconstructed images with the corresponding positive sample image to obtain a reconstruction error;
inputting the target vector into a Gaussian mixture model to obtain a test probability of the product image, inputting the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image;
determining a test error of the product image according to the target error and the test probability, and determining a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability;
selecting an error threshold from a plurality of the sample errors and obtaining a detection result of defect detection in the product image according to the test error and the error threshold.

### Claim 16 (depends on 15)

The non-transitory storage medium of claim 15, wherein encoding the product image to obtain a target vector corresponding to the product image by using an encoder comprises:
performing a vectorization process on the product image to obtain a feature vector;
acquiring a first hidden layer in the encoder and calculating the feature vector through the acquired first hidden layer to obtain the target vector.

### Claim 17 (depends on 15)

The non-transitory storage medium of claim 15, wherein comparing the target image with the product image to obtain a target error comprises:
acquiring all pixels of the product image as a plurality of examinable pixels, and acquiring all pixels of the target image as a plurality of target pixels;
comparing each target pixel with the corresponding examinable pixel and when a comparison result is that a target pixel is different from the corresponding examinable pixel;
calculating a number of differences between the target pixel as being a first number and the corresponding examinable pixel;
calculating a total number of the plurality of target pixels as being a second number;
dividing the first number by the second number to obtain the target error.

### Claim 18 (depends on 15)

The non-transitory storage medium of claim 15, wherein inputting the target vector into a Gaussian mixture model to obtain a test probability of the product image and inputting the plurality of latent vectors into the Gaussian mixture model to obtain an estimated probability of each corresponding positive sample image comprises:
inputting the plurality of latent vectors into the Gaussian mixture model to obtain a feature distribution of the plurality of positive sample images;
determining an average value and a covariance of the plurality of latent vectors according to the feature distribution;
acquiring a mixing coefficient of the Gaussian mixture model;
determining a test probability of the product image according to the target vector, the average value, the covariance and the mixing coefficient;
determining the test probability of each positive sample image according to each latent vector, the average value, and the covariance and the mixing coefficient.

### Claim 19 (depends on 15)

The non-transitory storage medium of claim 15, wherein determining a sample error of each positive sample image according to the corresponding reconstruction error and the corresponding estimated probability comprises:
calculating a logarithmic value of each estimated probability;
calculating a weighted sum of an opposite value of the logarithmic value and the corresponding reconstruction error to obtain the sample error of the corresponding positive sample image.

### Claim 20 (depends on 15)

The non-transitory storage medium of claim 15, wherein selecting an error threshold from a plurality of the sample errors comprises:
sorting the sample errors in ascending order to obtain a list of errors and a sequence number of each sample error in the list of errors;
calculating a number of the sample errors as being a third number;
multiplying the third number by a configuration value to obtain a target value;
selecting a sample error which has sequence number equal to the target value from the error list and determining the sequence number of the selected sample error as the error threshold.