# US 2021/0232872 A1 · Characterization System and Method With Guided Defect Discovery

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232872 A1
- Document ID: US20210232872A1
- Status: Published application
- Kind: A1
- Application number: 16/847,367
- Filed: 2020-04-13
- Published: 2021-07-29
- Earliest claimed: 2020-01-27
- Assignees: KLA Corporation
- Applicants: KLA Corporation
- Inventors: Bradley Ries; Tommaso Torelli; Muthukrishnan Sankar; Vineethanand Hariharan
- Web page: https://patentgrep.com/patent/US20210232872A1

## CPC

- G06T 7/001: {using an image reference approach}
- G01N 21/8851: {Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002)}
- G06V 10/82: using neural networks
- G06V 30/19173: {Classification techniques}
- G01N 21/956: Inspecting patterns on the surface of objects {(contactless testing of electronic circuits G01R31/308; testing currency G07D; manufacturing processes per se of semiconductor devices implementing a measuring step H10P74/20)}
- G01N 2021/8883: {involving the calculation of gauges, generating models}
- G01N 2021/8887: {based on image processing techniques}
- G06T 2207/10061: from scanning electron microscope
- G06T 2207/20081: Training; Learning
- G06T 2207/30148: Semiconductor; IC; Wafer
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system is disclosed, in accordance with one or more embodiment of the present disclosure. The system may include a controller including one or more processors configured to execute a set of program instructions. The set of program instructions may be configured to cause the processors to: receive images of a sample from a characterization sub-system; identify target clips from patch clips; prepare processed clips based on the target clips; generate encoded images by transforming the processed clips; sort the encoded images into a set of clusters; display sorted images from the set of clusters; receive labels for the displayed sorted images; determine whether the received labels are sufficient to train a deep learning classifier; and upon determining the received labels are sufficient to train the deep learning classifier, train the deep learning classifier via the displayed sorted images and the received labels.

## Claims

### Claim 1

A system comprising:
a controller including one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to:
receive one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identify one or more target clips from the one or more patch clips;
prepare one or more processed clips based on the one or more target clips;
generate one or more encoded images by transforming the one or more processed clips via an autoencoder;
sort the one or more encoded images into a set of clusters via a clustering algorithm;
display one or more sorted images from one or more of the set of clusters to a user via a user interface;
receive one or more labels for the one or more displayed sorted images from the user via the user interface;
determine whether the received one or more labels are sufficient to train a deep learning classifier; and
upon determining the received one or more labels are sufficient to train the deep learning classifier, train the deep learning classifier via the one or more displayed sorted images and the received one or more labels.

### Claim 2 (depends on 1)

The system of claim 1, wherein the preparing the one or more processed clips based on the one or more target clips comprises:
generating at least one of one or more median clips or one or more difference clips.

### Claim 3 (depends on 1)

The system of claim 1, wherein the set of program instructions are further configured to cause the one or more processors to:
apply the deep learning classifier to one or more additional target images to automatically classify the one or more additional target images.

### Claim 4 (depends on 1)

The system of claim 1, wherein the set of program instructions are further configured to cause the one or more processors to:
prior to determining the received one or more labels are sufficient to train the deep learning classifier, determine that the received one or more labels are insufficient to train the deep learning classifier;
display additional one or more sorted images to the user via the user interface;
receive one or more additional labels for the one or more additional displayed sorted images from the user via the user interface; and
determine whether the received one or more additional labels are sufficient to train the deep learning classifier.

### Claim 5 (depends on 1)

The system of claim 1, wherein the characterization sub-system comprises at least one of a scanning electron microscopy (SEM) sub-system or an optical inspection sub-system.

### Claim 6 (depends on 5)

The system of claim 5, wherein the optical inspection sub-system comprises at least one of a bright-field inspection sub-system or a dark-field inspection sub-system.

### Claim 7 (depends on 1)

The system of claim 1, wherein the autoencoder is configured to learn a low dimensional representation of the one or more patch clips.

### Claim 8 (depends on 1)

The system of claim 1, wherein the set of clusters are sorted based on one or more similar defect characteristics.

### Claim 9 (depends on 1)

The system of claim 1, wherein the set of clusters further include:
one or more outlier events.

### Claim 10 (depends on 1)

The system of claim 1, wherein the one or more target clips includes one or more defects of interest (DOIs).

### Claim 11

A system comprising:
a controller including one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to:
receive one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identify one or more target clips from the one or more patch clips;
prepare one or more processed clips based on the one or more target clips;
generate one or more encoded images by transforming the one or more processed clips via an autoencoder;
sort the one or more encoded images into a set of clusters via a clustering algorithm;
display one or more sorted images from one or more of the set of clusters to a user via a user interface;
receive one or more labels for the one or more displayed sorted images from the user via the user interface;
determine whether the received one or more labels are sufficient to train a deep learning classifier;
upon determining the received one or more labels are insufficient to train the deep learning classifier, display additional one or more sorted images to the user via the user interface;
receive one or more additional labels for the one or more additional displayed sorted images from the user via the user interface;
determine whether the received additional one or more labels are sufficient to train the deep learning classifier; and
upon determining the received one or more additional labels are sufficient to train the deep learning classifier, train the deep learning classifier via the one of the one or more displayed sorted images, the one or more additional displayed sorted images, the received one or more labels, and the received one or more additional labels.

### Claim 12 (depends on 11)

The system of claim 11, wherein the preparing the one or more processed clips based on the one or more target clips comprises:
generating at least one of one or more median clips or one or more difference clips.

### Claim 13 (depends on 11)

The system of claim 11, wherein the set of program instructions are further configured to cause the one or more processors to:
apply the deep learning classifier to one or more additional target images to automatically classify the one or more additional target images.

### Claim 14 (depends on 11)

The system of claim 11, wherein the characterization sub-system comprises at least one of a scanning electron microscopy (SEM) sub-system or an optical inspection sub-system.

### Claim 15 (depends on 14)

The system of claim 14, wherein the optical inspection sub-system comprises at least one of a bright-field inspection sub-system or a dark-field inspection sub-system.

### Claim 16 (depends on 11)

The system of claim 11, wherein the autoencoder is configured to learn a low dimensional representation of the one or more patch clips.

### Claim 17 (depends on 11)

The system of claim 11, wherein the set of clusters are sorted based on one or more similar defect characteristics.

### Claim 18 (depends on 11)

The system of claim 11, wherein the set of clusters further include:
one or more outlier events.

### Claim 19 (depends on 11)

The system of claim 11, wherein the one or more target clips includes one or more defects of interest (DOIs).

### Claim 20

A method comprising:
receiving one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identifying one or more target clips from the one or more patch clips;
preparing one or more processed clips based on the one or more target clips;
generating one or more encoded images by transforming the one or more processed clips via an autoencoder;
sorting the one or more encoded images into a set of clusters via a clustering algorithm;
displaying one or more sorted images from one or more of the set of clusters to a user via a user interface;
receiving one or more labels for the one or more displayed sorted images from the user via the user interface;
determining whether the received one or more labels are sufficient to train a deep learning classifier; and
upon determining the received one or more labels are sufficient to train the deep learning classifier, training the deep learning classifier via the one or more displayed sorted images and the received one or more labels.

### Claim 21

A method comprising:
receiving one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identifying one or more target clips from the one or more patch clips;
preparing one or more processed clips based on the one or more target clips;
generating one or more encoded images by transforming the one or more processed clips via an autoencoder;
sorting the one or more encoded images into a set of clusters via a clustering algorithm;
displaying one or more sorted images from one or more of the set of clusters to a user via a user interface;
receiving one or more labels for the one or more displayed sorted images from the user via the user interface;
determining whether the received one or more labels are sufficient to train a deep learning classifier;
upon determining the received one or more labels are insufficient to train the deep learning classifier, displaying one or more additional sorted images to the user via the user interface;
receiving one or more additional labels for the one or more additional displayed sorted images from the user via the user interface;
determining whether the received one or more additional labels are sufficient to train the deep learning classifier; and
upon determining the received one or more additional labels are sufficient to train the deep learning classifier, training the deep learning classifier via the one or more displayed sorted images, the one or more additional displayed sorted images, the received one or more labels, and the received one or more additional labels.

### Claim 22

A system comprising:
a controller including one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to:
receive one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identify one or more target clips from the one or more patch clips;
prepare one or more processed clips based on the one or more target clips;
generate one or more encoded images by transforming the one or more processed clips via an autoencoder;
sort the one or more encoded images into a set of clusters via a clustering algorithm;
display one or more sorted images from one or more of the set of clusters to a user via a user interface;
receive one or more labels for the one or more displayed sorted images from the user via the user interface; and
adjust one or more fabrication tools based on the received one or more labels.

### Claim 23 (depends on 22)

The system of claim 22, wherein the preparing the one or more processed clips based on the one or more target clips comprises:
generating at least one of one or more median clips or one or more difference clips.

### Claim 24 (depends on 22)

The system of claim 22, wherein the characterization sub-system comprises at least one of a scanning electron microscopy (SEM) sub-system or an optical inspection sub-system.

### Claim 25 (depends on 24)

The system of claim 24, wherein the optical inspection sub-system comprises at least one of a bright-field inspection sub-system or a dark-field inspection sub-system.

### Claim 26 (depends on 22)

The system of claim 22, wherein the autoencoder is configured to learn a low dimensional representation of the one or more patch clips.

### Claim 27 (depends on 22)

The system of claim 22, wherein the set of clusters of the one or more encoded images are sorted based on one or more similar defect characteristics.

### Claim 28 (depends on 22)

The system of claim 22, wherein the set of clusters further include:
one or more outlier events.

### Claim 29 (depends on 22)

The system of claim 22, wherein the one or more target clips includes one or more defects of interest (DOIs).

### Claim 30

A method comprising:
receiving one or more images of a sample from a characterization sub-system, wherein the one or more images include one or more patch clips;
identifying one or more target clips from the one or more patch clips;
preparing one or more processed clips based on the one or more target clips;
generating one or more encoded images by transforming the one or more processed clips via an autoencoder;
sorting the one or more encoded images into a set of clusters via a clustering algorithm;
displaying one or more sorted images from one or more of the set of clusters to a user via a user interface;
receiving one or more labels for the one or more displayed sorted images from the user via the user interface; and
adjusting one or more fabrication tools based on the received one or more labels.
