# US 2021/0334608 A1 · Abnormal Wafer Image Classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334608 A1
- Document ID: US20210334608A1
- Status: Published application
- Kind: A1
- Application number: 17/237,516
- Filed: 2021-04-22
- Published: 2021-10-28
- Earliest claimed: 2020-04-22
- Assignees: PDF Solutions, Inc.
- Applicants: PDF Solutions, Inc.
- Inventors: Tomonori Honda; Richard Burch; Qing Zhu; Jeffrey Drue David
- Web page: https://patentgrep.com/patent/US20210334608A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F 18/2431: Multiple classes
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30148: Semiconductor; IC; Wafer
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A semiconductor image classifier. Convolution functions are applied to modify the wafer images in order to extract key information about the image. The modified images are condensed then processed through a series of pairwise classifiers, each classifier configured to determine that the image is more like one of the pair than the other. Probabilities from each classifier are collected to form a prediction for each image.

## Claims

### Claim 1

A method comprising:
obtaining a first plurality of sample images of semiconductor wafers formed on a semiconductor substrate, the first plurality of sample images including a plurality of different image types;
applying at least a first convolution function to each of the first plurality of images to generate a plurality of modified images each corresponding to a respective one of the plurality of sample images;
applying a pooling function to each of the modified images to form a plurality of corresponding vectors each defining a plurality of features for each respective modified image;
performing variable selection on each corresponding vector in order to select a smaller set of features from the plurality of features that are more important in determining image type for the vector;
processing each corresponding vector in a plurality of machine-learning-based pairwise classifier models, each of the pairwise classifier models configured to determine a corresponding probability that the vector is more like one of unique plurality of pairs of the plurality of different image types;
inputting the determined probabilities from the processing step into a final machine-learning-based classifier model configured to assign one of the plurality of image types to each corresponding vector on the basis of the determined probabilities from the processing step; and
running the final machine-learning-based classifier model.

### Claim 2 (depends on 1)

The method of claim 1, the step of applying at least a first convolution function further comprising:
identifying a kernel having a matrix of values; and
multiplying the first plurality of images by the kernel to generate the plurality of modified images.

### Claim 3 (depends on 2)

The method of claim 2, further wherein the matrix of values has a size that is proportional to feature size associated with one of the plurality of different image types.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
identifying the kernel by trying a plurality of different kernels each having a different matrix of values.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
performing the first convolution function multiple times before generating the plurality of modified images.

### Claim 6 (depends on 1)

The method of claim 1, the step of applying at least a first convolution function further comprising:
applying a plurality of different convolution functions to each of the first plurality of images to generate a plurality of sets of modified images, each set associated with one of the plurality of different convolution functions, each modified image corresponding to a respective one of the plurality of sample images; and
determining that at least one of the plurality of convolution functions is effective at showing a feature associated with at least one of the plurality of image types.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
applying a combination of the plurality of different convolution functions to each of the first plurality of images to generate the plurality of sets of modified images.

### Claim 8 (depends on 1)

The method of claim 1, the step of applying a pooling function further comprising:
applying a statistical measure to a plurality of points in each of the first plurality of images to generate each of the plurality of corresponding vectors.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
taking a first statistical measure for a first plurality of points in a first direction across each of the first plurality of images to form a first plurality of corresponding vectors; and
taking a second statistical measure for a second plurality of points in a second direction across each of the first plurality of images to form a second plurality of corresponding vectors.

### Claim 10 (depends on 1)

The method of claim 1, the step of performing variable selection further comprising:
building a plurality of predictive models each corresponding to one of the plurality of pairwise classifier models;
for each predictive model:
ordering by rank the plurality of features associated with corresponding vectors;
run the predictive model a plurality of times for forward variable selection, each time adding one of the plurality of features into the predictive model by rank order, and incorporating the added one feature into the predictive model when it improves accuracy of the predictive model; and
run the predictive model a plurality of times for backward variable selection, each time removing one of the plurality of features by reverse rank order, and reincorporating the removed one feature into the predictive model when accuracy of the predictive model was decreased by its removal.

### Claim 11

A method comprising:
obtaining a first plurality of sample images of semiconductor wafers formed on a semiconductor substrate, the first plurality of sample images including a plurality of different classes of defects appearing on the sample images;
applying a plurality of mathematical operators, individually or in various combinations, to each of the first plurality of images thereby generating a plurality of modified images, each modified image corresponding to a respective one of the plurality of sample images;
condensing the plurality of modified images to a plurality of single-dimensional representations, each single-dimensional representation defining a plurality of features for each corresponding modified image;
performing variable selection on each single-dimensional representation in order to select a smaller set of features from the plurality of features that are more important in determining the class of defect for the single-dimensional representation;
processing each corresponding single-dimensional representation in a plurality of machine-learning-based pairwise classifier models, each of the pairwise classifier models configured to determine a corresponding probability that the single-dimensional representation is more indicative of one of a unique plurality of pairs of the plurality of different classes of defects; and
running a final machine-learning-based classifier model configured to assign one of the plurality of classes of defects to each corresponding single-dimensional representation on the basis of the determined probabilities from the processing step.

### Claim 12 (depends on 11)

The method of claim 11, the step of performing variable selection further comprising:
building a plurality of predictive models each corresponding to one of the plurality of pairwise classifier models;
for each predictive model:
ordering by rank the plurality of features associated with corresponding single-dimensional representations;
run the predictive model a plurality of times for forward variable selection, each time adding one of the plurality of features into the predictive model by rank order, and incorporating the added one feature into the predictive model when it improves accuracy of the predictive model; and
run the predictive model a plurality of times for backward variable selection, each time removing one of the plurality of features by reverse rank order, and reincorporating the removed one feature into the predictive model when accuracy of the predictive model was decreased by its removal.

### Claim 13 (depends on 11)

The method of claim 11, wherein the mathematical operators are convolution functions.

### Claim 14 (depends on 11)

The method of claim 11, the condensing step further comprising:
applying pooling functions to the plurality of single-dimensional representations.

### Claim 15

A method comprising:
obtaining a sample image;
processing the sample image through a series of pairwise classifiers, each pairwise classifier configured to determine a probability that the sample image is more like one than the other of a unique pair of a plurality of classes of defects; and
processing the determined probabilities to generate a final prediction for assigning one of the plurality of classes of defects to the sample image.

### Claim 16 (depends on 15)

The method of claim 15, the step of processing the sample image further comprising:
applying convolution and pooling functions to the sample image to reduce the sample image to at least one single-dimensional representation defining a plurality of features prior to the pairwise classifiers, such that the pairwise classifiers determine probabilities for the corresponding single-dimensional representation.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
run each pairwise classifier a plurality of times for forward variable selection, each time adding one of the plurality of features into the pairwise classifier, and incorporating the added one feature into the pairwise classifier when it improves accuracy of the pairwise classifier; and
run each pairwise classifier a plurality of times for backward variable selection, each time removing one of the plurality of features, and reincorporating the removed one feature into the pairwise classifier when accuracy of the pairwise classifier was decreased by its removal.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
running a final classifier model configured to assign one of the plurality of classes of defects to the single-dimensional representation on the basis of results from each of the plurality of pairwise classifiers.

### Claim 19 (depends on 19)

The method of claim 19, further comprising:
repeating the method steps for a plurality of sample images.

### Claim 20

A method comprising:
obtaining a first plurality of sample images of semiconductor wafers formed on a semiconductor substrate, the first plurality of sample images including a plurality of different image types;
computing a plurality of single-dimension metrics each corresponding to a respective one of the plurality of sample images, each metric each defining a plurality of features for the respective sample image;
performing variable selection on each metric in order to select a smaller set of features from the plurality of features that are more important in determining image type for the metric;
processing each corresponding metric in a plurality of machine-learning-based pairwise classifier models, each of the pairwise classifier models configured to determine a corresponding probability that the metric is more indicative of one of a unique plurality of pairs of the plurality of different image types; and
running a final machine-learning-based classifier model configured to assign one of the plurality of image types to each corresponding vector on the basis of the determined probabilities from the processing step.