# US 12,001,940 B2 · Identifying microorganisms using three-dimensional quantitative phase imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,940 B2
- Document ID: US12001940B2
- Status: Granted
- Kind: B2
- Application number: 17/951,872
- Filed: 2022-09-23
- Published: 2023-01-19
- Granted: 2024-06-04
- Earliest claimed: 2019-03-13
- Assignees: Tomocube, Inc.
- Applicants: Tomocube, Inc.
- Inventors: Kihyun Hong; Hyun-Seok Min; YongKeun Park; Geon Kim; Youngju Jo
- Web page: https://patentgrep.com/patent/US12001940B2

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06V 20/698: {Matching; Classification}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06T 2207/10056: Microscopic image
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying the predicted type of one or more microorganisms. In one aspect, a system comprises a phase-contrast microscope and a microorganism classification system. The phase-contrast microscope is configured to generate a three-dimensional quantitative phase image of one or more microorganisms. The microorganism classification system is configured to process the three-dimensional quantitative phase image using a neural network to generate a neural network output characterizing the microorganisms, and thereafter identify the predicted type of the microorganisms using the neural network output.

## Claims

### Claim 1

A method comprising:
using a phase-contrast microscope to generate one or more three-dimensional (3D) quantitative phase images of one or more microscopic entities,
wherein each of the 3D quantitative phase images comprises a respective 3D representation of the one or more microscopic entities as a 3D refractive index tomogram represented as a 3D matrix of numerical values, where each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; and
processing the one or more 3D quantitative phase images using a neural network, wherein:
the neural network is a convolutional neural network comprising one or more 3D convolutional layers, wherein each of the 3D convolutional layers performs operations comprising:
receiving a layer input that comprises one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and
processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and
the neural network is configured to process the one or more three-dimensional quantitative phase images in accordance with trained parameter values of the neural network to generate a neural network output characterizing the one or more microscopic entities; and
classifying the one or more microscopic entities using the neural network output.

### Claim 2 (depends on 1)

The method of claim 1, wherein one or more of the microscopic entities are microscopic organisms.

### Claim 3 (depends on 2)

The method of claim 2, wherein one or more of the microscopic entities are multi-cellular organisms.

### Claim 4 (depends on 1)

The method of claim 1, wherein each of the one or more 3D quantitative phase images is captured at a different time point.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the neural network output comprises a respective probability value for each of a predetermined number of possible classes; and
the probability value for a given class indicates a likelihood that the one or more microscopic entities are included in the given class.

### Claim 6 (depends on 5)

The method of claim 5, wherein each class represents a respective type of microscopic entity.

### Claim 7 (depends on 6)

The method of claim 6, wherein a type of a microscopic entity characterizes one or more of: a genus of the microscopic entity, a species of the microscopic entity, a strain of the microscopic entity, a gram-stainability of the microscopic entity, a metabolism of the microscopic entity, a morphology of the microscopic entity, or a motility of the microscopic entity.

### Claim 8 (depends on 5)

The method of claim 5, wherein classifying the one or more microscopic entities using the neural network output comprises:
classifying the one or more microscopic entities as being included in a class associated with a highest probability value in the neural network output.

### Claim 9 (depends on 1)

The method of claim 1, wherein the one or more microscopic entities are bacteria, or viruses, or fungi, or parasites, or microalgae.

### Claim 10 (depends on 9)

The method of claim 9, wherein one or more of the microscopic entities are bacteria.

### Claim 11 (depends on 10)

The method of claim 10, wherein the bacteria are present in a blood sample of a patient.

### Claim 12 (depends on 11)

The method of claim 11, further comprising administering an antibiotic therapy to the patient based on the classification of the bacteria.

### Claim 13 (depends on 1)

The method of claim 1, wherein using a phase-contrast microscope to generate one or more three-dimensional quantitative phase images of the one or more microscopic entities comprises:
using the phase-contrast microscope to generate phase and amplitude images of the one or more microscopic entities at each of a plurality of illumination angles; and
reconstructing a three-dimensional refractive index tomogram using the phase and amplitude images.

### Claim 14 (depends on 1)

The method of claim 1, wherein the method takes at most one hour.

### Claim 15

A system comprising:
a phase-contrast microscope that is configured to generate one or more three-dimensional quantitative phase image of one or more microscopic entities,
wherein each of the 3D quantitative phase images comprises a respective 3D representation of the one or more microscopic entities as a 3D refractive index tomogram represented as a 3D matrix of numerical values, where each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; and
a microscopic entity classification system that is configured to perform operations comprising:
processing the one or more 3D quantitative phase images using a neural network, wherein:
the neural network is a convolutional neural network comprising one or more 3D convolutional layers, wherein each of the 3D convolutional layers performs operations comprising:
receiving a layer input that comprises one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and
processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and
the neural network is configured to process the one or more three-dimensional quantitative phase images in accordance with trained parameter values of the neural network to generate a neural network output characterizing the one or more microscopic entities; and
classifying the one or more microscopic entities using the neural network output.

### Claim 16 (depends on 15)

The system of claim 15, wherein one or more of the microscopic entities are microscopic organisms.

### Claim 17 (depends on 16)

The system of claim 16, wherein one or more of the microscopic entities are multi-cellular organisms.

### Claim 18 (depends on 15)

The system of claim 15, wherein each of the one or more 3D quantitative phase images is captured at a different time point.

### Claim 19 (depends on 15)

The system of claim 15, wherein:
the neural network output comprises a respective probability value for each of a predetermined number of possible classes; and
the probability value for a given class indicates a likelihood that the one or more microscopic entities are included in the given class.

### Claim 20 (depends on 19)

The system of claim 19, wherein each class represents a respective type of microscopic entity.