# US 2022/0284697 A1 · Training data generation with generative modeling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284697 A1
- Document ID: US20220284697A1
- Status: Published application
- Kind: A1
- Application number: 17/679,115
- Filed: 2022-02-24
- Published: 2022-09-08
- Earliest claimed: 2021-02-24
- Assignees: California Institute of Technology
- Applicants: California Institute of Technology
- Inventors: Emily C. Laubscher; Nitzan Razin; David A. Van Valen
- Web page: https://patentgrep.com/patent/US20220284697A1

## CPC

- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 10/776: Validation; Performance evaluation
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed herein include systems, devices, and methods for generating training data for spot detection using generative models with expectation maximization. The training data can be used to train a machine learning model for spot detection.

## Claims

### Claim 1

A method for generating training data for spot detection comprising:
under control of a hardware processor:
receiving a training image;
for each of a plurality of spot detection methods, detecting a plurality of spots on the training image using the spot detection method;
clustering spots of pluralities of spots to generate a plurality of clusters each comprising one or more spots of the pluralities of spots;
iteratively determining a true positive rate (TPR) and a false positive rate (FPR) for each of the plurality of spot detection methods until the TPR of the spot detection method converge and the FPR of the spot detection method converge, comprising, in each iteration:
for each of the plurality of clusters, determining a probability of the cluster being a true positive (TP) and a probability of the cluster being a false positive (FP) using a TPR and a FPR of the spot detection method, wherein the TPR and the FPR are an initial true positive rate (TPR) and an initial false positive rate (FPR), respectively, or a TPR and a FPR determined from a prior iteration, respectively;
determining (i) an expectation value of the cluster being a TP, (ii) an expectation value of the cluster being a false negative (FN), (iii) an expectation value of the cluster being a FP, and (iv) an expectation value of the cluster being a true negative (TN) using the probability of the cluster being a TP and the probability of the cluster being a FP and whether the cluster comprises a spot of the plurality of spots determined using the spot detection method;
determining (i) an expectation value of a total number of TPs, (ii) an expectation value of a total number of FNs, (iii) an expectation value of a total number of FPs, and (iv) an expectation value of a total number of TNs using the expectation values of each of the plurality of clusters being a TP, a FN, a FP, and a TN; and
determining a TPR and a FPR for spot detection method for the iteration using the expectation values of the total numbers of TPs, FNs, FPs, and TNs; and
generating training data comprising the training image and a plurality of training clusters comprising clusters of the plurality of clusters.

### Claim 2

(canceled)

### Claim 3

A method for generating training data for spot detection comprising:
under control of a hardware processor:
receiving a plurality of training images;
for each training image:
for each of a plurality of spot detection methods, detecting a plurality of spots on the training image using the spot detection method;
clustering spots of pluralities of spots to generate a plurality of clusters each comprising one or more spots of the pluralities of spots;
performing generative modeling with expectation maximization to determine a true positive rate (TPR) and a false positive rate (FPR) for each of the plurality of spot detection methods; and
generating training data comprising the plurality of training images and a plurality of training clusters comprising clusters of the plurality of clusters for each of the plurality of images.

### Claim 4 (depends on 2)

The method of claim 2, wherein
performing generative modeling with expectation maximization comprises: iteratively determining a TPR and a FPR for each of the plurality of spot detection methods until the TPR of the spot detection method converge and the FPR of the spot detection method converge.

### Claim at position 5

-7. (canceled)

### Claim 8 (depends on 2)

The method of claim 2, wherein performing generative modeling with expectation maximization comprises:
(a) for each of the plurality of spot detection methods:
for each of the plurality of clusters:
(a1) determining a probability of the cluster being a true positive (TP) and a probability of the cluster being a false positive (FP) using a TPR and a FPR of the spot detection method, wherein the TPR and the FPR are an initial true positive rate (TPR) and an initial false positive rate (FPR), respectively, or a TPR and a FPR determined from a prior iteration, respectively;
(a2) determining (i) an expectation value of the cluster being a TP, (ii) an expectation value of the cluster being a false negative (FN), (iii) an expectation value of the cluster being a FP, and (iv) an expectation value of the cluster being a true negative (TN) using the probability of the cluster being a TP and the probability of the cluster being a FP and whether the cluster comprises a spot of the plurality of spots determined using the spot detection method;
(a3) determining (i) an expectation value of a total number of TPs, (ii) an expectation value of a total number of FNs, (iii) an expectation value of a total number of FPs, and (iv) an expectation value of a total number of TNs using the expectation values of each of the plurality of clusters being a TP, a FN, a FP, and a TN; and
(a4) determining a TPR and a FPR for the spot detection method for the iteration using the expectation values of the total numbers of TPs, FNs, FPs, and TNs; and
(b) repeating step (a) until the TPR of the spot detection method converge and the FPR of the spot detection method converge.

### Claim 9 (depends on 1)

The method of claim 1, wherein the TPR of the spot detection method converge and the FPR of the spot detection method converge when the TPR of the current iteration and the TPR of the prior iteration differ by less than a threshold and when the FPR of the current iteration and the FPR of the prior iteration differ by less than the threshold, optionally wherein the threshold is at most 0.01.

### Claim 10 (depends on 1)

The method of claim 1,
wherein if the cluster comprises a spot of the plurality of spots detected using the spot detection method, the expectation values of the cluster being (i) a TP, (ii) a FN, (iii) a FP, and (iv) a TN are (i) the probability of the cluster being a TP, (ii) 0, (iii) the probability of the cluster being a FP, (iv) 0, respectively, and
wherein if the cluster does not comprise a spot of the plurality of spots detected using the spot detection method, the expectation values of the cluster being (i) a TP, (ii) a FN, (iii) a FP, and (iv) a TN are (i) 0, (ii) the probability of the cluster being a TP, (iii) 0, and (iv) the probability of the cluster being a FP, respectively.

### Claim 11 (depends on 1)

The method of claim 1, wherein the training image comprises a fluorescent image, and wherein receiving the training image comprises: generating the training image from a labeled sample.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
generating and displaying a graphical user interface (GUI) comprising (i) the training image overlaid with graphical representations of the plurality of spots, and (ii) one or more parameters of the spot detection method used to detect the plurality of spots;
receiving one or more updated parameters;
performing spot detection on the training image using the spot detection method and the one or more updated parameters to generate a plurality of updated spots; and
generating and displaying a GUI comprising (i) the training image overlaid with graphical representations of the plurality of updated spots, and (ii) the one or more updated parameters.

### Claim 13 (depends on 1)

The method of claim 1, wherein the plurality of spot detection methods comprises 3 spot detection methods.

### Claim 14 (depends on 1)

The method of claim 1, wherein the plurality of spot detection methods comprises peak local maximum, Laplacian of Gaussian, difference of Gaussian, Determinant of Hessian, h-dome detection, and/or Crocker-Grier centroid finding.

### Claim 15 (depends on 1)

The method of claim 1, wherein clustering the spots comprises: clustering the spots of pluralities of spots to generate the plurality of clusters each comprising one or more spots of the pluralities of spots based on locations of the spots.

### Claim 16 (depends on 1)

The method of claim 1, wherein each of the plurality clusters comprises at most one spot from each of the pluralities of spots.

### Claim 17 (depends on 1)

The method of claim 1, further comprising: receiving the initial TPR and the initial FPR for one, one or more, or each of the plurality of spot detection methods.

### Claim 18 (depends on 1)

The method of claim 1, further comprising: determining the initial TPR and the initial FPR for one, one or more, or each of the plurality of spot detection methods.

### Claim 19

(canceled)

### Claim 20 (depends on 1)

The method of claim 1, wherein training clusters of the plurality of training clusters have sub-pixel precision.

### Claim 21 (depends on 1)

The method of claim 1, further comprising: generating the plurality of training clusters comprising clusters of the plurality of clusters for the training image.

### Claim at position 20

-24. (canceled)

### Claim 25 (depends on 1)

The method of claim 1, further comprising: training a machine learning model using the training data, optionally wherein the machine learning model comprises a deep learning model.

### Claim at position 22

-30. (canceled)

### Claim 31

A system for spot detection comprising:
non-transitory memory configured to store:
executable instructions, and
a machine learning model, wherein the machine learning model is trained using training data comprising a plurality of training images and a plurality of training clusters on each of the plurality of training images, wherein the plurality of training clusters is generated based on (a) a plurality of clusters each comprising one or more spots of pluralities of spots on a training image, each plurality of spots detected on the training image using one of a plurality of spot detection methods and (b) generative modeling with expectation maximization on the plurality of clusters which determines a true positive rate (TPR) and a false positive rate (FPR) for each of the plurality of spot detection methods for the training image; and
a hardware processor in communication with the non-transitory memory, the hardware processor programmed by the executable instructions to perform:
receiving an input image;
performing spot detection on the input image using the machine learning model.

### Claim 32

(canceled)

### Claim 33 (depends on 31)

The system of claim 31, wherein outputs of the machine learning model comprise a plurality of spots with sub-pixel precision and/or a center of one, one or more, or each of the plurality of spots.

### Claim 34 (depends on 31)

The system of claim 31, wherein outputs of the machine learning model comprise a pixel-wise classification prediction of each pixel in an input image containing a spot and a regression prediction of a distance of each pixel in the input image from the nearest spot in the image.

### Claim at position 27

-55. (canceled)