# US 2022/0036971 A1 · Method and system for predicting response to immune anticancer drugs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036971 A1
- Document ID: US20220036971A1
- Status: Published application
- Kind: A1
- Application number: 17/502,339
- Filed: 2021-10-15
- Published: 2022-02-03
- Earliest claimed: 2020-05-08
- Assignees: Lunit Inc.
- Applicants: Lunit Inc.
- Inventors: Donggeun YOO; Chanyoung OCK; Kyunghyun PAENG
- Web page: https://patentgrep.com/patent/US20220036971A1

## CPC

- G16B 20/00: ICT specially adapted for functional genomics or proteomics, e.g. genotype-phenotype associations
- G16H 30/40: for processing medical images, e.g. editing
- G01N 1/30: Staining; Impregnating {; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis}
- G06N 3/02: Neural networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/698: {Matching; Classification}
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure relates to a method, performed by at least one computing device, for predicting a response to an immune checkpoint inhibitor. The method includes receiving a first pathology slide image, detecting one or more target items in the first pathology slide image, determining at least one of an immune phenotype of at least some regions in the first pathology slide image or information associated with the immune phenotype based on the detection result for the one or more target items, and generating a prediction result as to whether or not a patient associated with the first pathology slide image responds to the immune checkpoint inhibitor, based on the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype.

## Claims

### Claim 1

A method, performed by at least one computing device, for predicting a response to an immune checkpoint inhibitor, comprising:
receiving a first pathology slide image;
detecting one or more target items in the first pathology slide image;
determining at least one of an immune phenotype of at least some regions in the first pathology slide image or information associated with the immune phenotype, based on a detection result for the one or more target items; and
generating a prediction result as to whether or not a patient associated with the first pathology slide image responds to the immune checkpoint inhibitor, based on at least one of the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the detecting includes detecting the one or more target items in the first pathology slide image using an artificial neural network model for target item detection, and
the artificial neural network model for target item detection is trained to detect one or more reference target items from a reference pathology slide image.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the at least some regions in the first pathology slide image includes the one or more target items, the one or more target items include items associated with cancer and immune cells, and
the determining includes:
calculating at least one of the number of, a distribution of, or a density of the immune cells in the items related to cancer in the at least some regions in the first pathology slide image; and
determining at least one of an immune phenotype of at least some regions in the first pathology slide image or information associated with the immune phenotype, based on at least one of the calculated number, distribution, or density of the immune cells.

### Claim 4 (depends on 3)

The method according to claim 3, wherein
the items associated with cancer include a cancer area and a cancer stroma,
the calculating includes:
calculating a density of the immune cells in the cancer area in the at least some regions in the first pathology slide image; and
calculating a density of the immune cells in the cancer stroma in the at least some regions in the first pathology slide image, and
the determining includes, based on at least one of the density of the immune cells in the cancer area or the density of the immune cells in the cancer stroma, determining at least one of the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype.

### Claim 5 (depends on 4)

The method according to claim 4, wherein,
if the density of the immune cells in the cancer area is equal to or greater than a first threshold density, the immune phenotype of the at least some regions in the first pathology slide image is determined to be immune inflamed,
if the density of the immune cells in the cancer area is less than the first threshold density and the density of the immune cells in the cancer stroma is equal to or greater than a second threshold density, the immune phenotype of the at least some regions in the first pathology slide image is determined to be immune excluded, and
if the density of the immune cells in the cancer area is less than the first threshold density and the density of the immune cells in the cancer stroma is less than the second threshold density, the immune phenotype of the at least some regions in the first pathology slide image is determined to be immune desert.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the first threshold density is determined based on a distribution of densities of the immune cells in the cancer area in each of a plurality of regions of interest in a plurality of pathology slide images, and
the second threshold density is determined based on a distribution of densities of the immune cells in the cancer stroma in each of the plurality of regions of interest in the plurality of pathology slide images.

### Claim 7 (depends on 3)

The method according to claim 3, wherein the determining further includes, based on the number of the immune cells included in a specific region in a cancer area, determining the immune phenotype of the at least some regions in the first pathology slide image to be one of immune inflamed, immune excluded, or immune desert.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the determining includes determining at least one of the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype by inputting a feature for the at least some regions in the first pathology slide image or the at least some regions in the first pathology slide image to an artificial neural network model for immune phenotype classification, and
the artificial neural network model for immune phenotype classification is trained so as to, upon input of a feature for at least some regions in a reference pathology slide image or the at least some regions in the reference pathology slide image, determine at least one of an immune phenotype of the at least some regions in the reference pathology slide image or the information associated with the immune phenotype.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the feature for the at least some regions in the first pathology slide image includes at least one of: a statistical feature for the one or more target items in the at least some regions in the first pathology slide image; a geometric feature for the one or more target items; and an image feature corresponding to the at least some regions in the first pathology slide image.

### Claim 10 (depends on 1)

The method according to claim 1, wherein
the at least some regions in the first pathology slide image include a plurality of regions of interest,
the immune phenotype of the at least some regions in the first pathology slide image includes an immune phenotype of each of the plurality of regions of interest, and
the generating includes:
based on the immune phenotype of each of the plurality of regions of interest, determining a most common immune phenotype included in a whole region of the first pathology slide image; and
based on the most common immune phenotype included in the whole region of the first pathology slide image, generating a prediction result as to whether or not the patient responds to the immune checkpoint inhibitor.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the generating includes:
generating an immune phenotype map for the at least some regions in the first pathology slide image by using the immune phenotype of the at least some regions in the first pathology slide image; and
inputting the generated immune phenotype map to a response prediction model for immune checkpoint inhibitor to generate a prediction result as to whether or not the patient responds to the immune checkpoint inhibitor, and
the response prediction model for immune checkpoint inhibitor is trained to generate a reference prediction result upon input of a reference immune phenotype map.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the generating includes:
generating an immune phenotype feature map for the at least some regions in the first pathology slide image by using the information associated with the immune phenotype of the at least some regions in the first pathology slide image; and
generating a prediction result as to whether or not the patient responds to the immune checkpoint inhibitor by inputting the generated immune phenotype feature map to a response prediction model for immune checkpoint inhibitor, and
the response prediction model for immune checkpoint inhibitor is trained to generate a reference prediction result upon input of a reference immune phenotype feature map.

### Claim 13 (depends on 1)

The method according to claim 1, further including obtaining information on expression of a biomarker from a second pathology slide image associated with the patient, wherein the generating includes,
based on the information on the expression of a biomarker and at least one of the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype, generating a prediction result as to whether or not the patient responds to the immune checkpoint inhibitor.

### Claim 14 (depends on 13)

The method according to claim 13, wherein
the biomarker is PD-L1, and
the information on the expression of the PD-L1 includes at least one of a tumor proportion score (TPS) value or combined proportion score (CPS) value.

### Claim 15 (depends on 13)

The method according to claim 13, wherein
the biomarker is PD-L1,
the obtaining includes:
receiving the second pathology slide image; and
generating the information on the expression of the PD-L1 by inputting the second pathology slide image to an artificial neural network model for expression information generation, and
the artificial neural network model for expression information generation is trained so as to, upon input of a reference pathology slide image, generate reference information on the expression of the PD-L1.

### Claim 16 (depends on 15)

The method according to claim 15, wherein the generating the information on the expression of the PD-L1 by inputting the second pathology slide image to the artificial neural network model for expression information generation includes,
generating the information on the expression of the PD-L1 by detecting, using the artificial neural network model for expression information generation, at least one of: a location of tumor cells; a location of lymphocytes; a location of macrophages; or whether or not PD-L1 is expressed, in at least some regions in the second pathology slide image.

### Claim 17 (depends on 1)

The method according to claim 1, further including:
outputting at least one of: the detection result for the one or more target items; the immune phenotype of the at least some regions in the first pathology slide image; the information associated with the immune phenotype; the prediction result as to whether or not the patient responds to the immune checkpoint inhibitor; or a density of immune cells in the at least some regions in the first pathology slide image.

### Claim 18 (depends on 1)

The method according to claim 1, further including:
based on the prediction result as to whether or not the patient responds to a plurality of immune checkpoint inhibitors, outputting information on at least one immune checkpoint inhibitor suitable for the patient from among the plurality of immune checkpoint inhibitors.

### Claim 19 (depends on 1)

A non-transitory computer-readable recording medium storing a computer program for executing, on a computer, a method for predicting a response to an immune checkpoint inhibitor according to claim 1.

### Claim 20

An information processing system comprising:
a memory storing one or more instructions; and
a processor configured to execute the stored one or more instructions so as to:
receive a first pathology slide image; detect one or more target items in the first pathology slide image; determine an immune phenotype of at least some regions in the first pathology slide image or information associated with the immune phenotype, based on a detection result for the one or more target items; and generate a prediction result as to whether or not a patient associated with the first pathology slide image responds to an immune checkpoint inhibitor, based on the immune phenotype of the at least some regions in the first pathology slide image or the information associated with the immune phenotype.
