# US 2023/0143860 A1 · Pathology prediction based on spatial feature analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0143860 A1
- Document ID: US20230143860A1
- Status: Published application
- Kind: A1
- Application number: 17/986,737
- Filed: 2022-11-14
- Published: 2023-05-11
- Earliest claimed: 2020-05-18
- Assignees: Genentech, Inc.; Hoffmann-La Roche Inc.
- Applicants: Genentech, Inc.; Hoffmann-La Roche Inc.
- Inventors: Xiao LI; Jian DAI; Fabien GAIRE
- Web page: https://patentgrep.com/patent/US20230143860A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06V 20/698: {Matching; Classification}
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T 2207/10024: Color image
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30096: Tumor; Lesion
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and methods relate to processing digital pathology mages. More specifically, depictions of objects of a first class (e.g., lymphocytes) and depictions of objects of a second class (e.g., tumor cells) are detected. Locations of each biological object depiction are identified, which are used to generate multiple spatial-distribution metrics that characterize where depictions of objects of a first class are located relative to objects of a second class. The spatial-distribution metrics are used to generate a result corresponding to a predicted biological state of or a potential treatment of a subject. For example, the result may predict whether and/or an extent to which lymphocytes have infiltrated a tumor, whether checkpoint blockade therapy would be an effective treatment for the subject, and/or whether a subject is eligible for a clinical trial.

## Claims

### Claim 1

A method comprising:
accessing, by a computing system, a digital pathology image that depicts a section of a biological sample collected from a subject having a given medical condition;
detecting, within the digital pathology image, a set of biological object depictions, the set of biological object depictions comprising a first set of biological object depictions of a first class of biological object and a second set of biological object depictions of a second class of biological object;
generating one or more relational-location representations of the biological object depictions, each of the one or more relational-location representations indicating a location of a first biological object depiction relative to a second biological object depiction;
determining, using the one or more relational-location representations, a spatial-distribution metric characterizing a degree to which at least part of the first set of biological object depictions are depicted as being interspersed with at least part of the second set of biological object depictions;
generating, based on the spatial-distribution metric, a result that corresponds to a prediction regarding a degree to which a given treatment that modulates immunological response will effectively treat the given medical condition of the subject;
determining that the subject is eligible for a clinical trial based on the result; and
generating a display including an indication that the subject is eligible for the clinical trial.

### Claim 2 (depends on 1)

The method of claim 1, wherein the spatial-distribution metric comprises:
a metric defined based on a K-nearest-neighbor analysis;
a metric defined based on Ripley's K-function;
a Morisita-Horn index;
a Moran's index;
a metric defined based on a correlation function;
a metric defined based on a hotspot/coldspot analysis; or
a metric defined based on a Kriging-based analysis.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the spatial-distribution metric is of a first type of metric;
the method further comprises determining, using the one or more relational-location representations, a second spatial-distribution metric characterizing the degree to which at least part of the first set of biological object depictions are depicted as being interspersed with at least part of the second set of biological object depictions, wherein the second spatial-distribution metric is of a second type of metric that is different from the first type of metric; and
the result is generated further based on the second spatial-distribution metric.

### Claim 4 (depends on 3)

The method of claim 3, wherein generating the result comprises processing the first spatial-distribution metric and the section spatial-distribution metric using a trained machine-learning model, the trained machine-learning model having been trained using a set of training elements, each of the set of training elements corresponding to another subject having received the particular treatment associated with the clinical trial, and each of the set of training elements including another set of spatial-distribution metrics and a responsiveness value indicating a degree to which the given treatment activated an immunological response in the other subject.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the result comprises comparing a value of the spatial-distribution metric to a threshold value.

### Claim 6 (depends on 1)

The method of claim 1, wherein the given medical condition is a type of cancer and wherein the given treatment is an immune-checkpoint-blockade treatment.

### Claim 7 (depends on 1)

The method of claim 1, wherein the one or more relational-location representations include, for each biological object depiction of the set of biological object depictions, a set of coordinates that identifies a location of the biological object depiction within the digital pathology image.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating the one or more relational-location representation of the biological object depictions comprises:
identifying, for each biological object depiction of the first set of biological object depictions, a first point location within the digital pathology image corresponding to the biological object depiction;
identifying, for each biological object depiction of the second set of biological object depictions, a second point location within the digital pathology image corresponding to the biological object depiction; and
comparing the first point location and the second point location.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first point location within the digital pathology image is selected by calculating, for the biological object depiction of the first set of biological object depictions, a mean point location, a centroid point location, a median point location, or a weighted point location.

### Claim 10 (depends on 8)

The method of claim 8, wherein determining the spatial-distribution metric comprises calculating, for each of at least some of the first set of biological object depictions and for each of at least some of the second set of biological object depictions, a distance between the first point location corresponding to the biological object depiction of the first set of biological object depictions and the second point location corresponding to the biological object depictions of the second set of biological object depictions.

### Claim 11 (depends on 8)

The method of claim 8, wherein determining the spatial-distribution metric further comprises identifying, for each of the at least some of the first set of biological object depictions, one or more of the second set of biological object depictions associated with a distance between the first point location corresponding to the biological object depiction of the first set of the biological object depictions and the second point location corresponding to the biological object depiction of the second set of biological object depictions.

### Claim 12 (depends on 1)

The method of claim 1, wherein the one or more relational-location representations include, for each of a set of image regions in the digital pathology image, a representation of an absolute or relative quantity of biological object depictions of the first class of biological object identified as being located within the region and an absolute or relative quantity of biological object depictions of the second type of biological object identified as being located within the region.

### Claim 13 (depends on 1)

The method of claim 1, wherein the one or more relational-location representations include a distance-based probability of a biological object depiction of the first set of biological object depictions being depicted as located within a given distance from a biological object depiction of the second set of biological object depictions.

### Claim 14 (depends on 1)

The method of claim 1, further comprising accessing genetic sequencing or radiology imaging data for the subject, wherein the result is generated further based on a characteristic of the genetic sequencing or radiology imaging data.

### Claim 15 (depends on 1)

The method of claim 1, wherein the first class of biological object is a tumor cell and the second class of biological object is an immune cell.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
receiving user input data from a user device comprising an identifier of the subject, wherein the computing system accesses the digital pathology image in response to receiving the identifier;
wherein generating the display including the indication that the subject is eligible for the clinical trial comprises providing the indication that the subject is eligible for the clinical trial to the user device.

### Claim 17 (depends on 16)

The method of claim 16, further comprising receiving an indication that the subject has been enrolled in the clinical trial.

### Claim 18 (depends on 1)

The method of claim 1, wherein generating the display including the indication that the subject is eligible for the clinical trial comprises informing the subject of the determination of eligibility for the clinical trial.

### Claim 19

A system comprising:
one or more data processors; and
a non-transitory computer readable storage medium communicatively coupled to the one or more data processors, and including instructions which, when executed by the one or more data processors, cause the one or more data processors to perform one or more operations comprising:
accessing a digital pathology image that depicts a section of a biological sample collected from a subject having a given medical condition;
detecting, within the digital pathology image, a set of biological object depictions, the set of biological object depictions comprising a first set of biological object depictions of a first class of biological object and a second set of biological object depictions of a second class of biological object;
generating one or more relational-location representations of the biological object depictions, each of the one or more relational-location representations indicating a location of a first biological object depiction relative to a second biological object depiction;
determining, using the one or more relational-location representations, a spatial-distribution metric characterizing a degree to which at least part of the first set of biological object depictions are depicted as being interspersed with at least part of the second set of biological object depictions;
generating, based on the spatial-distribution metric, a result that corresponds to a prediction regarding a degree to which a given treatment that modulates immunological response will effectively treat the given medical condition of the subject;
determining that the subject is eligible for a clinical trial based on the result; and
generating a display including an indication that the subject is eligible for the clinical trial.

### Claim 20

One or more computer-readable non-transitory storage media including instructions that, when executed by one or more data processors, cause the one or more data processors to performed operations comprising:
accessing a digital pathology image that depicts a section of a biological sample collected from a subject having a given medical condition;
detecting, within the digital pathology image, a set of biological object depictions, the set of biological object depictions comprising a first set of biological object depictions of a first class of biological object and a second set of biological object depictions of a second class of biological object;
generating one or more relational-location representations of the biological object depictions, each of the one or more relational-location representations indicating a location of a first biological object depiction relative to a second biological object depiction;
determining, using the one or more relational-location representations, a spatial-distribution metric characterizing a degree to which at least part of the first set of biological object depictions are depicted as being interspersed with at least part of the second set of biological object depictions;
generating, based on the spatial-distribution metric, a result that corresponds to a prediction regarding a degree to which a given treatment that modulates immunological response will effectively treat the given medical condition of the subject;
determining that the subject is eligible for a clinical trial based on the result; and
generating a display including an indication that the subject is eligible for the clinical trial.
