Deep learning models for tumor evaluation

An apparatus comprising:

• a memory storing instructions;
• and
• processing circuitry configured by execution of the instructions stored in the memory to determine a clinical value for an individual based on a tumor in an image by:
• determining a lymphocyte distribution of lymphocytes in a tumor based on an image of the tumor,
• applying a classifier to the lymphocyte distribution to classify the tumor, the classifier configured to classify tumors into a class selected from at least two classes associated with lymphocyte distributions, and
• outputting the clinical value based on prognoses of individuals with tumors in the class into which the classifier classified the tumor, wherein,
• the at least two classes are a low-risk tumor class and a high-risk tumor class,
• determining the lymphocyte distribution further comprises applying a convolutional neural network to the image, the convolutional neural network configured to measure the lymphocyte distribution of lymphocytes for different area of the image,
• the classifier includes a two-way Gaussian mixture model configured to determine, for respective classes, a probability distribution of features for tumors in the class within a feature space, and
• the instructions further cause the processing circuitry to,
• apply a Cox proportional hazards model to clinical features of the tumor to determine a class of the tumor, and
• determine the clinical value for the individual based on the prognoses of the individuals with tumors in the class into which the classifier classified the tumor and the class determined by the Cox proportional hazards model.

A non-transitory computer-readable medium storing instructions that, when executed by processing circuitry, cause the processing circuitry to determine a clinical value for an individual based on a tumor in an image by:

• determining a lymphocyte distribution of lymphocytes in a tumor based on an image of the tumor,
• applying a classifier to the lymphocyte distribution to classify the tumor, the classifier configured to classify tumors into a class selected from at least two classes associated with lymphocyte distributions, and
• outputting the clinical value for the individual based on prognoses of individuals with tumors in the class into which the classifier classified the tumor, wherein
• the at least two classes are a low-risk tumor class and a high-tumor class,
• determining the lymphocyte distribution further comprises applying a convolutional neural network to the image, the convolutional neural network configured to measure the lymphocyte distribution of lymphocytes for different types of the image,
• the classifier includes a two-way Gaussian mixture model configured to determine, for respective classes, a probability distribution of features for tumors in the class within a feature space, and
• the instructions further cause the processing circuitry to,
• apply a Cox proportional hazards model to clinical features of the tumor to determine a class of the tumor, and
• determine the clinical value for the individual based on the prognoses of the individuals with tumors in the class into which the classifier classified the tumor and the class determined by the Cox proportional hazards model.

A method of operating an apparatus including processing circuitry and a convolutional neural network that is trained to determine a lymphocyte distribution of lymphocytes in an area of an image, the method comprising:

• executing, by the processing circuitry, instructions that cause the apparatus to:
• receive an image depicting at least part of a tumor,
• invoke the convolutional neural network to determine a lymphocyte distribution of lymphocytes in respective areas of the tumor based on the image,
• apply a classifier to the lymphocyte distribution to classify the tumor, the classifier trained to classify tumors into a class selected from at least two classes associated with lymphocyte distributions, and
• determine a clinical value for an individual based on a set of prognosis data corresponding to individuals with tumors in the class into which the classifier classified the tumor,
• wherein the convolutional neural network is further trained to classify an area of the image as one or more area types selected from an area type set including a tumor area, a lymphocyte area, and a stroma area,
• wherein determining the lymphocyte distribution of lymphocytes in the tumor includes, for respective stroma areas of the image,
• determining a distance of the stroma area to a tumor area, and
• based on the distance, characterizing the stroma area as one of a tumor-infiltrating stroma area, and a tumor-adjacent stroma area, and
• wherein the classifier is configured to further classify the tumor based on the characterizing of the stroma area.