Automatically determining a medical recommendation for a patient based on multiple medical images from multiple different medical imaging modalities

One or more non-transitory computer-readable media comprising one or more computer readable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform a method for automatically determining a medical recommendation for a patient based on multiple medical images from multiple different medical imaging modalities, the method comprising:

• receiving a first medical image of a patient from a first medical imaging modality and a second medical image of the patient from a second medical imaging modality, the first medical imaging modality being different from the second medical imaging modality;
• mapping a first region of interest (ROI) on the first medical image to a second ROI on the second medical image, the first ROI and the second ROI both related to a medical condition of the patient;
• generating first annotation data related to the first ROI and second annotation data related to the second ROI;
• generating first medical clinical data related to the first ROI and second medical clinical data related to the second ROI;
• training a machine learning classifier with annotation data and medical clinical data of medical images of one or more other patients;
• inputting, into the machine learning classifier, the first and second annotation data and the first and second medical clinical data;
• in response to the inputting, automatically determining, by the machine learning classifier, a medical recommendation for the patient related to the medical condition of the patient;
• receiving a third medical image of the patient from the first medical imaging modality and a fourth medical image of the patient from the second medical imaging modality;
• mapping a third ROI on the third medical image to a fourth ROI on the fourth medical image, the third ROI and the fourth ROI both related to the medical condition of the patient;
• generating third annotation data related to the third ROI and fourth annotation data related to the fourth ROI;
• generating third medical clinical data related to the third ROI and fourth medical clinical data related to the fourth ROI;
• entering, into the machine learning classifier, the third and fourth annotation data and the third and fourth medical clinical data;
• and
• in response to the entering, automatically determining, by the machine learning classifier, an updated medical recommendation for the patient related to the medical condition of the patient.

A computing device comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media comprising one or more computer readable instructions that, when executed by the one or more processors, cause the computing device to perform a method for automatically determining a medical recommendation for a patient based on multiple medical images from multiple different medical imaging modalities, the method comprising:
• receiving a first medical image of a patient from a first medical imaging modality and a second medical image of the patient from a second medical imaging modality, the first medical imaging modality being different from the second medical imaging modality;
• mapping a first region of interest (ROI) on the first medical image to a second ROI on the second medical image, the first ROI and the second ROI both related to a medical condition of the patient;
• generating first annotation data related to the first ROI and second annotation data related to the second ROI;
• generating first medical clinical data related to the first ROI and second medical clinical data related to the second ROI;
• training a machine learning classifier with annotation data and medical clinical data of medical images of one or more other patients;
• inputting, into the machine learning classifier, the first and second annotation data and the first and second medical clinical data;
• in response to the inputting, automatically determining, by the machine learning classifier, a medical recommendation for the patient related to the medical condition of the patient;
• receiving a third medical image of the patient from the first medical imaging modality and a fourth medical image of the patient from the second medical imaging modality;
• mapping a third ROI on the third medical image to a fourth ROI on the fourth medical image, the third ROI and the fourth ROI both related to the medical condition of the patient;
• generating third annotation data related to the third ROI and fourth annotation data related to the fourth ROI;
• generating third medical clinical data related to the third ROI and fourth medical clinical data related to the fourth ROI;
• entering, into the machine learning classifier, the third and fourth annotation data and the third and fourth medical clinical data;
• and
• in response to the entering, automatically determining, by the machine learning classifier, an updated medical recommendation for the patient related to the medical condition of the patient.