# US 2021/0241048 A1 · Machine Learning-Based Root Cause Analysis of Process Cycle Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241048 A1
- Document ID: US20210241048A1
- Status: Published application
- Kind: A1
- Application number: 17/161,595
- Filed: 2021-01-28
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Illumina, Inc.
- Applicants: Illumina, Inc.
- Inventors: Naghmeh REZAEI; Pedro Miguel FILIPE CRUZ
- Web page: https://patentgrep.com/patent/US20210241048A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/2163: {Partitioning the feature space}
- G06F 18/24323: {Tree-organised classifiers}
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/04: Inference or reasoning models
- G06N 20/20: Ensemble learning
- G06T 5/20: using local operators
- G06V 10/56: relating to colour
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06V 20/698: {Matching; Classification}
- G16B 25/00: ICT specially adapted for hybridisation; ICT specially adapted for gene or protein expression
- G16B 40/10: Signal processing, e.g. from mass spectrometry [MS] or from PCR
- G16B 40/20: Supervised data analysis
- G01N 21/6428: {Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" (in vivo A61B5/00; immunoassay G01N33/53)}
- G01N 2021/6439: {with indicators, stains, dyes, tags, labels, marks}
- G06N 3/045: Combinations of networks
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06T 2207/10064: Fluorescence image
- G06T 2207/20028: Bilateral filtering
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T 2207/30072: Microarray; Biochip, DNA array; Well plate
- G06T 2207/30168: Image quality inspection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)

## Abstract

The technology disclosed relates to classification of process cycle images to predict success or failure of process cycles. The technology disclosed includes capturing and processing images of sections arranged on an image generating chip in genotyping process. Image description features of production cycle images are created and given as input to classifiers. A trained classifier separates successful production images from unsuccessful or failed production images. The failed production images are further classified by a trained root cause classifier into various categories of failure.

## Claims

### Claim 1

A method of training a random forest classifier for genotyping process cycle images, including:
accessing labelled training examples for images from process cycles belonging to a successful class and multiple failure classes;
creating image description features for each labelled training example based on a linear combination of Eigen images;
training the random forest classifier using the image description features of the labelled training examples; and
storing parameters of the trained random forest classifier.

### Claim 2 (depends on 1)

The method of claim 1, further including:
accessing a basis of Eigen images;
ordering the basis of Eigen images according to a measure of variability explained; and
selecting the top ordered basis of Eigen images that cumulatively explain variability above a threshold; and
analyzing the process cycle images using the selected basis of Eigen images.

### Claim 3 (depends on 1)

The method of claim 1, further including training the random forest classifier using the image description features for one-vs-the-rest determination of the successful class vs the multiple failure classes.

### Claim 4 (depends on 1)

The method of claim 1, wherein the random forest classifier includes 100 to 400 decision trees.

### Claim 5 (depends on 1)

The method of claim 1, wherein the random forest classifier has a depth of 10 to 40.

### Claim 6 (depends on 2)

The method of claim 2, further including:
accessing a second random forest classifier upon determination that the image description features do not belong to the successful class, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the multiple failure classes; and
applying the second random forest classifier to the image description features, including scoring each of the multiple failure classes vs the rest and using resulting scores to select among the multiple failure classes as a likely root cause of a failed process cycle.

### Claim 7

A non-transitory computer readable storage medium impressed with computer program instructions to train a random forest classifier for genotyping process cycle images, the instructions, when executed on a processor, implement a method comprising:
accessing labelled training examples for images from process cycles belonging to a successful class and multiple failure classes;
creating image description features for each labelled training example based on a linear combination of Eigen images;
training the random forest classifier using the image description features of the labelled training examples; and
storing parameters of the trained random forest classifier.

### Claim 8 (depends on 7)

The non-transitory computer readable storage medium of claim 7, implementing the method further comprising:
training the random forest classifier using the image description features for one-vs-the-rest determination of the successful class vs the multiple failure classes.

### Claim 9 (depends on 7)

The non-transitory computer readable storage medium of claim 7, implementing the method further comprising:
accessing a basis of Eigen images;
ordering the basis of Eigen images according to a measure of variability explained;
selecting the top ordered basis of Eigen images that cumulatively explain variability above a threshold; and
analyzing the process cycle images using the selected basis of Eigen images.

### Claim 10 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the random forest classifier includes 100 to 400 decision trees.

### Claim 11 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the random forest classifier has a depth of 10 to 40.

### Claim 12 (depends on 9)

The non-transitory computer readable storage medium of claim 9, implementing the method further comprising:
accessing a second random forest classifier upon determination that the image description features do not belong to the successful class, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the multiple failure classes; and
applying the second random forest classifier to the image description features, including scoring each of the multiple failure classes vs the rest and using resulting scores to select among the multiple failure classes as a likely root cause of a failed process cycle.

### Claim 13

A non-transitory computer readable storage medium impressed with computer program instructions to classify genotyping process cycle images, the instructions, when executed on a processor, implement a method comprising:
determining and applying a threshold, from intensities of pixels in a grayscale production cycle image, with which to classify image pixels into bright and dark classes and transform the grayscale production cycle image to a binary image;
segmenting the grayscale image and the transformed binary image into eight or more areas that separate systematic expected noise in the image from signal;
calculating average intensity values for the areas in the grayscale image and the transformed binary image;
accessing a random forest classifier trained to classify the images from process cycles, based on the average intensity values for the areas in the grayscale and the binary images, as belonging to a successful class or to a failure class and comparing a confidence score generated by the trained random forest classifier to a threshold and classifying the image as successful or failed.

### Claim 14 (depends on 13)

The non-transitory computer readable storage medium of claim 13, implementing the method further comprising:
applying a bilateral filter to preserve edges in the production cycle image before transforming the production cycle image.

### Claim 15 (depends on 7)

A system including one or more processors coupled to memory, the memory loaded with computer instructions to train a random forest classifier for process cycle images, when executed on the processors implement the instructions of claim 7.

### Claim 16 (depends on 15)

The system of claim 15, further implementing actions comprising:
accessing a basis of Eigen images;
ordering the basis of Eigen images according to a measure of variability explained; and
selecting the top ordered basis of Eigen images that cumulatively explain variability above a threshold; and
analyzing the process cycle images using the selected basis of Eigen images.

### Claim 17 (depends on 15)

The system of claim 15, wherein the random forest classifier includes 100 to 400 decision trees.

### Claim 18 (depends on 15)

The system of claim 15, wherein the random forest classifier has a depth of 10 to 40.

### Claim 19 (depends on 15)

The system of claim 15, further implementing actions comprising:
accessing a second random forest classifier upon determination that the image description features do not belong to the successful class, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the multiple failure classes; and
applying the second random forest classifier to the image description features, including scoring each of the multiple failure classes vs the rest and using resulting scores to select among the multiple failure classes as a likely root cause of the failed process cycle.

### Claim 20 (depends on 13)

A system including one or more processors coupled to memory, the memory loaded with computer instructions to classify process cycle images, when executed on the processors implement the instructions of claim 13.
