# US 12,119,091 B1 · Utilizing masked autoencoder generative models to extract microscopy representation autoencoder embeddings

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,119,091 B1
- Document ID: US12119091B1
- Status: Granted
- Kind: B1
- Application number: 18/545,438
- Filed: 2023-12-19
- Granted: 2024-10-15
- Earliest claimed: 2023-12-19
- Assignees: Recursion Pharmaceuticals, Inc.
- Applicants: Recursion Pharmaceuticals, Inc.
- Inventors: Oren Zeev Kraus; Kian Runnels Kenyon-Dean; Mohammadsadegh Saberian; Maryam Fallah; Peter Foster McLean; Jessica Wai Yin Leung; Vasudev Sharma; Ayla Yasmin Khan; Jaichitra Balakrishnan; Safiye Celik; Dominique Beaini; Maciej Sypetkowski; Chi Cheng; Kristen Rose Morse; Maureen Katherine Makes; Benjamin John Mabey; Berton Allen Earnshaw
- Web page: https://patentgrep.com/patent/US12119091B1

## CPC

- G16B 45/00: ICT specially adapted for bioinformatics-related data visualisation, e.g. displaying of maps or networks
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 10/82: using neural networks
- G06V 20/698: {Matching; Classification}
- G16B 20/00: ICT specially adapted for functional genomics or proteomics, e.g. genotype-phenotype associations
- G16B 40/00: ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing generative machine learning models to generate embeddings from phenomic images (or other microscopy representations). For example, the disclosed systems can train a generative machine learning model (e.g., a masked autoencoder generative model) to generate predicted (or reconstructed) phenomic images from masked version of ground truth training phenomic images. In some cases, the disclosed systems utilize a momentum-tracking optimizer while reducing a loss of the generative machine learning model to enable efficient training on large scale training image batches. Furthermore, the disclosed systems can utilize Fourier transformation losses with multi-stage weighting to improve the accuracy of the generative machine learning model on the phenomic images during training. Indeed, the disclosed systems can utilize the trained generative machine learning model to generate phenomic embeddings from input phenomic images (for various phenomic comparisons).

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a plurality of phenomic images corresponding to a plurality of cell perturbations;
generating, utilizing a masked autoencoder generative model, a plurality of phenomic image autoencoder embeddings for the plurality of cell perturbations from the plurality of phenomic images, wherein the masked autoencoder generative model is trained to generate predicted phenomic images from masked training phenomic images corresponding to training phenomic images;
generating perturbation comparisons utilizing the plurality of phenomic image autoencoder embeddings; and
providing, for display within a graphical user interface, the perturbation comparisons for the plurality of phenomic image autoencoder embeddings.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of phenomic images portray cell phenotypes corresponding to the plurality of cell perturbations.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the masked autoencoder generative model is trained utilizing a reconstruction loss between the predicted phenomic images and the training phenomic images.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the masked autoencoder generative model is trained utilizing a momentum-tracking optimizer.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3, wherein the masked autoencoder generative model is trained utilizing Fourier transformation losses between the predicted phenomic images and the training phenomic images.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein the masked autoencoder generative model is trained utilizing a first Fourier transformation loss with a first Fourier weight for a first training phenomic image and a second Fourier transformation loss with a second Fourier weight for a second training phenomic image.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising generating a perturbation similarity heatmap utilizing the perturbation comparisons.

### Claim 8 (depends on 2)

The computer-implemented method of claim 2, further comprising determining a cell count within a phenomic image based on a phenomic perturbation autoencoder embedding.

### Claim 9 (depends on 2)

The computer-implemented method of claim 2, further comprising:
identifying a phenomic image; and
generating, utilizing the masked autoencoder generative model, a corrected phenomic image from the phenomic image.

### Claim 10

A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive a plurality of phenomic images corresponding to a plurality of cell perturbations;
generate, utilizing a masked autoencoder generative model, a plurality of phenomic image autoencoder embeddings for the plurality of cell perturbations from the plurality of phenomic images, wherein the masked autoencoder generative model is trained to generate predicted phenomic images from masked training phenomic images corresponding to training phenomic images;
generate perturbation comparisons utilizing the plurality of phenomic image autoencoder embeddings; and
provide, for display within a graphical user interface, the perturbation comparisons for the plurality of phenomic image autoencoder embeddings.

### Claim 11 (depends on 10)

The system of claim 10, wherein the masked autoencoder generative model is trained utilizing a reconstruction loss between the predicted phenomic images and the training phenomic images.

### Claim 12 (depends on 10)

The system of claim 10, wherein the instructions cause the system to generate a perturbation similarity heatmap utilizing the perturbation comparisons.

### Claim 13 (depends on 11)

The system of claim 11, wherein the instructions cause the system to determine a cell count within a phenomic image based on a phenomic perturbation autoencoder embedding.

### Claim 14 (depends on 11)

The system of claim 11, wherein the masked autoencoder generative model is trained utilizing a momentum-tracking optimizer.

### Claim 15 (depends on 14)

The system of claim 14, wherein the masked autoencoder generative model is trained utilizing Fourier transformation losses between the predicted phenomic images and the training phenomic images.

### Claim 16

A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
receive a plurality of phenomic images corresponding to a plurality of cell perturbations;
generate, utilizing a masked autoencoder generative model, a plurality of phenomic image autoencoder embeddings for the plurality of cell perturbations from the plurality of phenomic images, wherein the masked autoencoder generative model is trained to generate predicted phenomic images from masked training phenomic images corresponding to training phenomic images;
generate perturbation comparisons utilizing the plurality of phenomic image autoencoder embeddings; and
provide, for display within a graphical user interface, the perturbation comparisons for the plurality of phenomic image autoencoder embeddings.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the masked autoencoder generative model is trained utilizing a momentum-tracking optimizer.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions cause the computing device to generate a perturbation similarity heatmap utilizing the perturbation comparisons.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions cause the computing device to:
identify a phenomic image; and
generate, utilizing the masked autoencoder generative model, a corrected phenomic image from the phenomic image.

### Claim 20 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the masked autoencoder generative model is trained to generate predicted phenomic images from masked training phenomic images corresponding to training phenomic images utilizing a first Fourier transformation loss with a first Fourier weight for a first training phenomic image and a second Fourier transformation loss with a second Fourier weight for a second training phenomic image.