# US 2018/0322634 A1 · Training and machine learning classification of mold in digital microscopy images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322634 A1
- Document ID: US20180322634A1
- Status: Published application
- Kind: A1
- Application number: 15/585,039
- Filed: 2017-05-02
- Published: 2018-11-08
- Earliest claimed: 2017-05-02
- Assignees: Techcyte, Inc.
- Applicants: TechCyte, Inc.
- Inventors: S. Russell Zimmerman; Ralph Yarro, III; Benjamin P. Cahoon; Richard Boyd Smith; Hyrum S. Anderson
- Web page: https://patentgrep.com/patent/US20180322634A1

## CPC

- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/24: Classification techniques
- G06T 7/11: Region-based segmentation
- G06V 20/698: {Matching; Classification}
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems, methods, and devices for classifying or detecting mold samples or training computer models (such as neural networks), are disclosed. A method includes obtaining a microscopy image of a mold sample. The method includes determining a classification of the mold sample based on non-image data corresponding to the mold sample. The method further includes training a computer model based on the microscopy image and a label indicating the classification.

## Claims

### Claim 1

A system for training a computer model for mold spore classification, the system comprising:
one or more processors; and
non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:
obtain a first image of a sample or cultured population;
extract portions of the first image that correspond to mold spores, the mold spores having a known classification based on the sample or cultured population;
place the extracted portions of the first image on a background to create a synthetic image; and
train a computer model based on the synthetic image and a label indicating the known classification.

### Claim 2 (depends on 1)

The system of claim 1, wherein the sample or cultured population is generated by culturing a known mold sample to generate the sample or cultured population, wherein the known mold sample has the known classification.

### Claim 3 (depends on 1)

The system of claim 1, wherein the instructions cause the one or more processors to extract portions of the first image that correspond to mold spores by automatically segmenting the portions of the first image that correspond to mold spores from the first image.

### Claim 4 (depends on 1)

The system of claim 1, wherein the background of the synthetic image comprises a different color than a background of the first image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the background of the synthetic comprises an image of a dirty slide, wherein the dirty slide is free or substantially free from any mold spores.

### Claim 6 (depends on 1)

The system of claim 1, wherein the instructions cause the one or more processors to train the computer model by providing subsections of the synthetic image to the computer model, wherein the subsections comprise portions of the synthetic image where one or more mold spores are located

### Claim 7 (depends on 1)

The system of claim 1, wherein the label comprises a label for each subsection of the synthetic image where one or more mold spores are located.

### Claim 8 (depends on 1)

The system of claim 1, wherein the known classification comprises a classification determined based on a cultured population of a sample using one or more of:
a DNA test;
polymerase chain reaction (PCR) test;
a known source location for a mold sample;
a fluorescence test; and
a microscope examination using one or more of a higher magnification than the first image or an oil immersion lens.

### Claim 9 (depends on 1)

The system of claim 1, wherein the sample or cultured population comprises a first sample or cultured population and the known classification comprises a first classification, wherein the instructions further cause the one or more processors to:
obtain a second image of a second sample or cultured population having a second classification;
extract portions of the second image that correspond to mold spores; and
place the extracted portions of the second image on the background of the synthetic image to create the synthetic image, wherein the synthetic image comprises image data for mold spores having the first classification and mold spores having the second classification.

### Claim 10 (depends on 9)

The system of claim 9, wherein the first classification is determined based on non-image data corresponding to the first sample or cultured population, and wherein the second classification is determined based on non-image data corresponding to the second sample or cultured population.

### Claim 11 (depends on 9)

The method of claim 9, wherein the first classification and second classification comprises classifications indistinguishable to humans based on image data alone.

### Claim 12

A method comprising:
obtaining a microscopy image of a mold sample;
determining a classification of the mold sample based on non-image data corresponding to the mold sample; and
training a computer model based on the microscopy image and a label indicating the classification.

### Claim 13 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on a cultured population generated from the mold sample.

### Claim 14 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on DNA test results for the mold sample.

### Claim 15 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on polymerase chain reaction (PCR) test results for the mold sample.

### Claim 16 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on a known source location of the mold sample.

### Claim 17 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on fluorescence test results for the mold sample.

### Claim 18 (depends on 12)

The method of claim 12, wherein determining the classification comprises determining the classification based on a microscope examination using one or more of a higher magnification than the microscopy image or an oil immersion lens.

### Claim 19 (depends on 12)

The method of claim 12, wherein training the computer model comprises providing subsections of the microscopy image to the computer model, wherein the subsections comprise portions of the microscopy image where one or more mold spores are located.

### Claim 20 (depends on 18)

The method of claim 18, wherein training the computer model comprises providing a label for each subsection of the microscopy image where one or more mold spores are located.

### Claim 21 (depends on 19)

The method of claim 19, wherein obtaining the microscopy image of the mold sample comprises obtaining a microscopy image of a cultured population based on the mold sample.

### Claim 22 (depends on 19)

The method of claim 19, wherein the microscopy image comprises subsections where one or more mold spores are located.

### Claim 23 (depends on 19)

The method of claim 19, further comprising obtaining the mold sample.

### Claim 24 (depends on 19)

The method of claim 19, further comprising culturing the mold sample to generate a cultured population of the mold having the same classification as the mold sample.

### Claim 25 (depends on 19)

The method of claim 19, wherein obtaining the microscopy image of the mold sample comprises obtaining a synthetic microscopy image.

### Claim 26 (depends on 24)

The method of claim 24, further comprising generating the synthetic microscopy image.

### Claim 27 (depends on 25)

The method of claim 25, wherein generating the synthetic microscopy image comprises:
obtaining a first image of a sample or cultured population;
extracting portions of the first image that correspond to mold spores; and
placing the extracted portions of the first image on a background to create the synthetic image.

### Claim 28 (depends on 26)

The method of claim 26, wherein extracting portions of the first image that correspond to mold spores comprises automatically segmenting the portions of the first image that correspond to mold spores from the first image.

### Claim 29 (depends on 26)

The method of claim 26, wherein the background comprises a different color background than the background of the first image.

### Claim 30 (depends on 26)

The method of claim 26, wherein the background comprises an image of a dirty slide, wherein the dirty slide is free or substantially free from any mold spores.

### Claim 31 (depends on 26)

The method of claim 26, wherein the sample or cultured population comprises a first sample or cultured population and the classification comprises a first classification, wherein creating the synthetic microscopy image further comprises:
obtaining a second image of a second sample or cultured population having a second classification;
extracting portions of the second image that correspond to mold spores; and
placing the extracted portions of the second image on the background to create the synthetic image, wherein the synthetic image comprises image data for mold spores having the first classification and mold spores having the second classification.

### Claim 32 (depends on 30)

The method of claim 30, wherein the first classification is determined based on non-image data corresponding to the first sample, and wherein the second classification is determined based on non-image data corresponding to the second sample.

### Claim 33 (depends on 30)

The method of claim 30, wherein the first classification and second classification comprises classifications indistinguishable to humans based on image data alone.

### Claim 34

Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:
receive a microscopy image of a mold sample;
processing the microscopy image of the mold sample using a computer model to determine a classification for the mold sample, wherein the computer model comprises a computer model trained using one or more synthetic microscopy images generated by:
obtaining a first image of a first sample or cultured population;
extracting portions of the first image that correspond to mold spores; and
placing the extracted portions of the first image on a background to create the synthetic microscopy image; and
storing an indication for the classification of the mold sample.

### Claim 35 (depends on 33)

The computer readable storage media of claim 33, wherein the one or more synthetic microscopy images were generated by further:
obtaining a second image of a second sample or cultured population having a second classification;
extracting portions of the second image that correspond to mold spores; and
placing the extracted portions of the second image on the background to create the synthetic image, wherein the synthetic microscopy image comprises image data for mold spores having the first classification and mold spores having the second classification, wherein the classification for the mold sample comprises the first classification or the second classification.

### Claim 36

A system for training a computer model for mold spore classification, the system comprising:
means for obtaining a first image of a sample or cultured population;
means for extracting portions of the first image that correspond to mold spores, the mold spores having a known classification based on the sample or cultured population;
means for placing the extracted portions of the first image on a background to create a synthetic image; and
means for training a computer model based on the synthetic image and a label indicating the known classification.

### Claim 37 (depends on 35)

The system of claim 35, wherein the sample or cultured population comprises a first sample or cultured population and the known classification comprises a first classification;
wherein means for obtaining the first image further comprises means for obtaining a second image of a second sample or cultured population having a second classification;
wherein the means for extracting portions of the first image further comprises means for extracting portions of the second image that correspond to mold spores; and
wherein the means for placing the extracted portions of the first image on the background further comprises means for placing extracted portions of the second image on the background of the synthetic image to create the synthetic image, wherein the synthetic image comprises image data for mold spores having the first classification and mold spores having the second classification.
