Methods and related aspects for classifying lesions in medical images

A method of treating a disease in a subject, the method comprising:

• extracting one or more image features from at least one region-of-interest (ROI) that comprises the lesion in at least one axial slice of a positron emission tomography (PET) and/or computed tomography (CT) image of at least a portion of the subject using a convolutional neural network (CNN) to generate CNN-extracted image feature data, wherein the at least one axial slice is segmented on a per-slice basis and wherein the portion of the subject that comprises the lesion and at least one segmentation of the lesion are input into the CNN as a binary mask;
• extracting one or more radiomic features from the PET and/or CT image to generate radiomic feature data, wherein the one or more radiomic features comprise one or more lesion intensity characteristics and/or one or more morphology characteristics, wherein the one or more lesion intensity characteristics are selected from the group consisting of:
• a mean and variance of lesion intensity, a mean and variance of lesion background intensity, a lesion-to-background ratio, and a maximum standardized uptake value (SUV) within the lesion, and wherein the one or more morphology characteristics are selected from the group consisting of:
• a lesion volume, a circularity measure, a solidity measure, and an eccentricity measure;
• combining the CNN-extracted image feature data and the radiomic feature data with anatomical location information about the lesion to generate combined information;
• inputting the combined information into an artificial neural network (ANN) that classifies the lesion in the PET and/or CT image using the combined information to generate a classification;
• and,
• administering, or discontinuing administering, one or more therapies to the subject based at least in part upon the classification, thereby treating the disease in the subject.

A system, comprising at least one controller that comprises, or is capable of accessing, computer readable media comprising non-transitory computer executable instructions which, when executed by at least one electronic processor, perform at least:

• extracting one or more image features from at least one region-of-interest (ROI) that comprises a lesion in at least one axial slice of a positron emission tomography (PET) and/or computed tomography (CT) image of at least a portion of a subject using a convolutional neural network (CNN) to generate CNN-extracted image feature data, wherein the at least one axial slice is segmented on a per-slice basis and wherein the portion of the subject that comprises the lesion and at least one segmentation of the lesion are input into the CNN as a binary mask;
• extracting one or more radiomic features from the PET and/or CT image to generate radiomic feature data, wherein the one or more radiomic features comprise one or more lesion intensity characteristics and/or one or more morphology characteristics, wherein the one or more lesion intensity characteristics are selected from the group consisting of:
• a mean and variance of lesion intensity, a mean and variance of lesion background intensity, a lesion-to-background ratio, and a maximum standardized uptake value (SUV) within the lesion, and wherein the one or more morphology characteristics are selected from the group consisting of:
• a lesion volume, a circularity measure, a solidity measure, and an eccentricity measure;
• combining the CNN-extracted image feature data and the radiomic feature data with anatomical location information about the lesion to generate combined information;
• and,
• inputting the combined information into an artificial neural network (ANN) that classifies the lesion in the PET and/or CT image using the combined information to generate a classification.