# US 11,029,359 B2 · Failure detection and classsification using sensor data and/or measurement data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,029,359 B2
- Document ID: US11029359B2
- Status: Granted
- Kind: B2
- Application number: 16/297,403
- Filed: 2019-03-08
- Published: 2019-09-12
- Granted: 2021-06-08
- Earliest claimed: 2018-03-09
- Assignees: PDF Solutions, Inc.
- Applicants: PDF Solutions, Inc.
- Inventors: Tomonori Honda; Lin Lee Cheong; Lakshmikar Kuravi
- Web page: https://patentgrep.com/patent/US11029359B2

## CPC

- G01R 31/2894: {Aspects of quality control [QC] (G01R31/31718 takes precedence; program control for QC G05B19/41875)}
- G01R 31/31835: {Analysis of test coverage or failure detectability}
- G06F 30/30: Circuit design
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A model is generated for predicting failures at the wafer production level. Input data from sensors is stored as an initial dataset, then data exhibiting excursions or useless impact is removed from the dataset. The dataset is converted into target features, where the target features are useful in predicting whether a wafer will be normal or not. A trade-off between positive and negative results is selected, and a plurality of predictive models are created. The final model is selected based on the trade-off criteria, and deployed.

## Claims

### Claim 1

A method for generating a robust predictive model for failure detection and classification in a semiconductor process, comprising:
collecting a first set of input data from a plurality of data sources during a plurality of production runs for making semiconductor devices according to a semiconductor process;
organizing the first set of input data into a plurality of training datasets;
converting selected portions of the plurality of training datasets into a plurality of target features for the semiconductor manufacturing process;
modifying the plurality of converted training datasets by removing one or more of the plurality of target features that exhibit negligible impact on classifying the semiconductor devices as normal or abnormal;
analyzing a second set of input data using a plurality of machine learning models, each machine learning model trained using at least one of the plurality of modified training datasets and configured to classify the semiconductor devices as either normal or abnormal; and
selecting one of the plurality of machine learning models for deployment in the semiconductor process.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
storing the input data as an initial dataset configured to generate a predictive model for a selected feature of the semiconductor process;
identifying, in the collected input data, one or more of the plurality of data sources for which a data excursion has been detected;
identifying, in the collected input data, one or more of the plurality of data sources that have a negligible effect on the predictive model;
removing unwanted data from the initial dataset to form a training dataset, the unwanted data includes data collected from the one or more of the plurality of data sources for which a data excursion has been detected and data collected from the one or more of the plurality of data sources that have a negligible effect on the predictive model;
selecting a trade-off criteria for positive and negative results;
building a plurality of predictive models for predicting whether a made device will be normal or abnormal;
selecting one of the predictive models based on the selected trade-off criteria, and
deploying the selected model in the semiconductor process.

### Claim 3 (depends on 1)

The method of claim 1, the identifying step further comprising:
building at least one model using a plurality of target features associated with a combination of a single sensor and a single manufacturing step;
when data received from the target features exhibits high accuracy, a key feature that is contributing to the high accuracy is identified and retained in the training datasets; and
when data received from the target features exhibits low accuracy, all target features associated with the combination of a single sensor and a single manufacturing step are removed from the training datasets.

### Claim 4 (depends on 1)

The method of claim 1, the identifying step further comprising:
building a plurality of models, each model using data from a plurality of sensors for a plurality of target features associated with high accuracy manufacturing steps;
when one or more of the models exhibits high accuracy, key features that are contributing to the high accuracy are identified and retained in the training datasets; and
when one or more of the models exhibits low accuracy, the target features are removed from the training datasets.

### Claim 5 (depends on 1)

The method of claim 1, the identifying step further comprising:
building a plurality of models, each model using data from a plurality of sensors for a plurality of target features associated with high accuracy manufacturing steps;
when one or more of the models exhibits high accuracy, key features that are contributing to the high accuracy are identified and retained in the training datasets; and
when one or more of the models exhibits low accuracy, the target features are removed from the training datasets.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining whether a specific sensor has a statistically different distribution between normal and abnormal devices.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
determining whether a specific manufacturing step has a statistically different distribution between normal and abnormal devices.

### Claim 8 (depends on 1)

The method of claim 1, the converting step further comprising:
using one or more of statistics-based methods, control-based methods, frequency-based methods, wavelet-based features, and regression-based methods to convert time-series data into the plurality of target features.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
prior to the converting step, cleansing the plurality of training datasets.

### Claim 10 (depends on 9)

The method of claim 9, wherein the step of cleansing the plurality of training datasets is performed using an anomaly detection method.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
for each of the plurality of machine learning models, determining an acceptable trade-off between false positive results and false negative results exhibited by the model.

### Claim 12

A method, comprising:
obtaining a plurality of training datasets, each training dataset having input data from a plurality of data sources associated with a semiconductor manufacturing process;
converting selected portions of the plurality of training datasets into a plurality of target features for the semiconductor manufacturing process;
modifying the plurality of converted training datasets by removing one or more of the plurality target features that exhibit negligible impact on classifying devices made from the semiconductor manufacturing process as normal or abnormal;
creating a plurality of models for the plurality of target features, and training the plurality of models using the plurality of modified training datasets; and
selecting one of the plurality of models to deploy for the semiconductor manufacturing process.

### Claim 13 (depends on 12)

The method according to claim 12, further comprising:
building at least one model using a first set of the plurality of target features associated with a combination of a single sensor and a single manufacturing step;
when data received from the first set of target features exhibits high accuracy, a key feature that is contributing to the high accuracy is identified and retained in the training datasets; and
when data received from the first set of target features exhibits low accuracy, all target features associated with the combination of a single sensor and a single manufacturing step are removed from the training datasets.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
determining whether a specific sensor has a statistically different distribution between normal and abnormal devices.

### Claim 15 (depends on 13)

The method of claim 13, further comprising:
determining whether a specific manufacturing step has a statistically different distribution between normal and abnormal devices.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
building a plurality of models, each model using data from a plurality of sensors for a plurality of target features associated with high accuracy manufacturing steps;
when one or more of the models exhibits high accuracy, key features that are contributing to the high accuracy are identified and retained in the training datasets; and
when one or more of the models exhibits low accuracy, the target features are removed from the training datasets.

### Claim 17 (depends on 12)

The method of claim 12, further comprising:
building a plurality of models, each model using data from a plurality of sensors for a plurality of target features associated with high accuracy manufacturing steps;
when one or more of the models exhibits high accuracy, key features that are contributing to the high accuracy are identified and retained in the training datasets; and
when one or more of the models exhibits low accuracy, the target features are removed from the training datasets.

### Claim 18 (depends on 12)

The method of claim 12, further comprising:
prior to the converting step, cleansing the plurality of training datasets.

### Claim 19

A computing system having a processor and non-transitory storage for program instruction, the program instructions configured for causing the processor to:
obtain a plurality of training datasets, each training dataset having input data from a plurality of data sources associated with a semiconductor manufacturing process;
convert selected portions of the plurality of training datasets into a plurality of target features for the semiconductor manufacturing process;
modify the plurality of converted training datasets to remove one or more of the plurality target features that exhibit negligible impact on classifying devices made from the semiconductor manufacturing process as normal or abnormal;
create a plurality of models for the plurality of target features, and training the plurality of models using the plurality of modified training datasets; and
select one of the plurality of models to deploy for the semiconductor manufacturing process.