# US 12,601,732 B2 · Cell image analysis device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,601,732 B2
- Document ID: US12601732B2
- Status: Granted
- Kind: B2
- Application number: 17/919,422
- Filed: 2021-02-03
- Published: 2023-05-25
- Granted: 2026-04-14
- Earliest claimed: 2020-04-21
- Assignees: Shimadzu Corporation
- Applicants: Shimadzu Corporation
- Inventors: Shuhei Yamamoto; Ryuji Sawada
- Web page: https://patentgrep.com/patent/US12601732B2

## CPC

- G01N 15/1433: using image recognition
- G01N 33/4833: {of solid biological material, e.g. tissue samples, cell cultures (tissue in vivo A61B5/00; cell suspensions G01N33/48735)}
- C12M 41/36: {of biomass, e.g. colony counters or by turbidity measurements (electrooptical investigation of individual particles G01N15/14, flow cytometers G01N15/1404)}
- G01N 15/1429: Signal processing
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 7/0014: {using an image reference approach}
- G06T 7/0016: {involving temporal comparison}
- G06T 7/155: involving morphological operators
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/695: {Preprocessing, e.g. image segmentation}
- G01N 2015/1006: {for cytology}
- G01N 2015/1486: {Counting the particles}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 20/20: Ensemble learning
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- C12M: APPARATUS FOR ENZYMOLOGY OR MICROBIOLOGY; {APPARATUS FOR CULTURING MICROORGANISMS FOR PRODUCING BIOMASS, FOR GROWING CELLS OR FOR OBTAINING FERMENTATION OR METABOLIC PRODUCTS, i.e. BIOREACTORS OR FERMENTERS}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A cell image analysis device includes: a display unit; a storage unit which stores learning data which is a set of data of a cell image and data of an analyzed image in which a region of interest included in the cell image is specified, and a discriminator which is created using a learning model constructed by machine learning using the learning data; a test data input receiver to receive an input of test data; a discriminator evaluation unit to evaluate the discriminator using the test data; an evaluation result storing unit to store evaluation data including the evaluation result of the discriminator and the test data used for the evaluation in association with the discriminator; and a display processor to display the test data and/or the evaluation result associated with the discriminator on a screen of the display unit in response to an input for selecting the discriminator.

## Claims

### Claim 1

A cell image analysis method comprising:
a preparation step of preparing a discriminator which is created using a learning model constructed by machine learning using learning data, and which outputs data of an analyzed assessment image in which a region of interest included in an assessment cell image is estimated according to an input of data of the assessment cell image, the learning data being at least one set of data of a learning cell image and data of a learning analyzed image in which a region of interest included in the learning cell image is specified;
a test data input reception step of receiving an input of test data which is a set of data of a test cell image and data of a test analyzed image of the test cell image different from the learning data, the test cell image including a cell image in which a type of cell or a type of culture medium is/are same or approximate with those of the assessment a cell image;
a discriminator evaluation step of evaluating the discriminator by comparing data of a test image output from the discriminator with respect to an input of data of the test cell image of the test data with the data of the test analyzed image of the test data, and obtaining an evaluation result;
a selection step of selecting the discriminator based on the evaluation result; and
an analysis step of inputting data of the assessment cell image to the discriminator selected, and obtaining an output related to data of the analyzed assessment image in which the region of interest included in the assessment cell image is estimated.

### Claim 2 (depends on 1)

The cell image analysis method according to claim 1, wherein a plurality of the discriminators are prepared in the preparation step, and the learning data used to create the discriminator is associated with each of the discriminators.

### Claim 3 (depends on 1)

The cell image analysis method according to claim 1, wherein different discriminators are prepared in the preparation step according to the type of cell and/or a the type of culture condition.

### Claim 4 (depends on 1)

The cell image analysis method according to claim 1, wherein the learning data comprises a plurality of sets of data of the learning cell image and data of the learning analyzed image in which the region of interest included in the learning cell image is specified further comprising a cell image input reception step receiving an input of the plurality of sets of data of the learning cell image and data of the learning analyzed image in which the region of interest included in the learning cell image is specified, and
an input of information regarding a respective a culture condition associated with the plurality of sets of data of the learning cell images.

### Claim 5 (depends on 4)

The cell image analysis method according to claim 4, further comprising
a discriminator creation step of creating the discriminator by constructing the learning model by machine learning using the plurality of sets of data of the learning cell image and learning analyzed image received in the cell image input reception step.

### Claim 6 (depends on 5)

The cell image analysis method according to claim 5, wherein further creating learning data and verification data by sorting the plurality of sets of data of the learning cell images received in the cell image input reception step at a predetermined ratio, and
further constructing the learning model in the discriminator creation step by executing the machine learning using the learning data and the verification data sorted in the cell image input reception step.

### Claim 7 (depends on 1)

The cell image analysis method according to claim 1, further comprising: an algorithm preparing step of preparing a preprocessing algorithm for preprocessing the data of the learning cell image and the data of the analyzed learning image; and
a preprocessing step of performing preprocessing of the data of the learning cell image and the data of the learning analyzed image using the preprocessing algorithm prepared in the algorithm preparing step.

### Claim 8 (depends on 7)

The cell image analysis method according to claim 7, wherein at least one of positioning, noise removal, or background removal of the data of the learning cell image and the data of the analyzed learning image, and binarization of luminance of the data of the analyzed learning image is executed in the preprocessing step.

### Claim 9 (depends on 1)

The cell image analysis method according to claim 1, further comprising:
an algorithm preparing step of preparing a postprocessing algorithm for postprocessing the data of the analyzed assessment image output from the discriminator; and
a postprocessing step of performing postprocessing of the data of the analyzed assessment image output from the discriminator using the postprocessing algorithm prepared in the algorithm preparing step.

### Claim 10 (depends on 9)

The cell image analysis method according to claim 9, wherein at least one of calculation of a morphological feature value of the region of interest estimated, graphing of a temporal change of the morphological feature value, graphing of the morphological feature value with respect to the culture condition, or calculation of a ratio between the region of interest and a background region in the data of the analyzed assessment image output from the discriminator is executed in the postprocessing step.

### Claim 11

A cell image analysis device comprising:
a display unit;
a storage unit which stores learning data which is a plurality of sets of data of a learning cell image and data of a learning analyzed image in which a region of interest included in the learning cell image is specified, and a discriminator which is created using a learning model constructed by machine learning using the learning data, and which outputs data of an analyzed assessment image in which the region of interest included in an assessment cell image is estimated according to an input of the data of the assessment cell image;
a test data input receiver configured to receive an input of test data which is a set of data of a test cell image and data of a test analyzed image of the test cell image different from the learning data;
a discriminator evaluation unit configured to evaluate the discriminator by comparing data of a test image output from the discriminator with respect to an input of data of the test cell image of the test data with the data of the test analyzed image of the test data;
an evaluation result storing unit configured to store evaluation data including an evaluation result by the discriminator evaluation unit and the test data used for the evaluation in the storage unit in association with the discriminator;
a display processor configured to display the evaluation result associated with the discriminator on a screen of the display unit in response to an input for selecting the discriminator stored in the storage unit;
a cell image input receiver configured to receive an input of the plurality of sets of data of the learning cell image and data of the learning analyzed image in which the region of interest included in the learning cell image is specified, and an input of information on the culture condition regarding the plurality of sets of data of the learning cell images, and store the input in the storage unit in association with each other; and
a discriminator registration unit configured to receive an input of the discriminator stored in the storage unit, a set of the data of the learning cell image and data of the learning analyzed image of the learning cell image associated with the discriminator, and the culture condition, and store the input in the storage unit.

### Claim 12

A cell image analysis device comprising:
a display unit;
a storage unit which stores learning data which is a plurality of sets of data of a learning cell image and data of a learning analyzed image in which a region of interest included in the learning cell image is specified, and a discriminator which is created using a learning model constructed by machine learning using the learning data, and which outputs data of an analyzed assessment image in which the region of interest included in an assessment cell image is estimated according to an input of the data of the assessment cell image;
a test data input receiver configured to receive an input of test data which is a set of data of a test cell image and data of a test analyzed image of the test cell image different from the learning data;
a discriminator evaluation unit configured to evaluate the discriminator by comparing data of a test image output from the discriminator with respect to an input of data of the test cell image of the test data with the data of the test analyzed image of the test data, and obtaining an evaluation result;
an evaluation result storing unit configured to store evaluation data including the evaluation result by the discriminator evaluation unit and the test data used for the evaluation in the storage unit in association with the discriminator;
a display processor configured to display the evaluation result associated with the discriminator on a screen of the display unit in response to an input for selecting the discriminator stored in the storage unit;
a cell image input receiver configured to receive an input of the plurality of sets of data of the learning cell image and data of the learning analyzed image in which the region of interest included in the learning cell image is specified, and an input of information on the culture condition regarding the plurality of sets of data of the learning cell images, and store the input in the storage unit in association with each other; and
a discriminator output unit configured to output the discriminator stored in the storage unit together with the set of the data of the learning cell image and the data of the analyzed learning image of the learning cell image associated with the discriminator, and the culture condition.

### Claim 13

A cell image analysis device comprising:
a display unit;
a storage unit which stores learning data which is a set of data of a learning cell image and data of a learning analyzed image in which a region of interest included in the learning cell image is specified, and a discriminator which is created using a learning model constructed by machine learning using the learning data, and which outputs data of an analyzed assessment image in which the region of interest included in an assessment cell image is estimated according to an input of the data of the assessment cell image;
a test data input receiver configured to receive an input of test data which is a set of data of a test cell image and data of a test analyzed image of the test cell image different from the learning data, the test cell image including a cell image in which a type of cell or a type of culture medium is/are same or approximate with those of the assessment cell image;
a discriminator evaluation unit configured to evaluate the discriminator by comparing data of a test image output from the discriminator with respect to an input of data of the test cell image of the test data with the data of the test analyzed image of the test data; and obtaining an evaluation result;
an evaluation result storing unit configured to store evaluation data including the evaluation result by the discriminator evaluation unit and the test data used for the evaluation in the storage unit in association with the discriminator; and
a display processor configured to display the evaluation result associated with the discriminator on a screen of the display unit in response to an input for selecting the discriminator stored in the storage unit.