Generating natural language text describing a change between sets of medical imaging data

A computer implemented training method of training a machine learning model for generating natural language text describing a change between first medical imaging data and second medical imaging data, the method comprising:

• providing the machine learning model, the machine learning model comprising:
• (a) a first machine learning model configured to generate, based on an input of given imaging data representative of given first medical imaging data and given second medical imaging data, or of a difference between the given first medical imaging data and the given second medical imaging data, an image feature vector, and
• (b) a second machine learning model configured to generate, based on an input of data representative of the image feature vector, natural language text describing a change between the given first medical imaging data and the given second medical imaging data;
• providing training data comprising a plurality of sets of training imaging data, each set of training imaging data representative of first training medical imaging data and second training medical imaging data, or of a difference between the first training medical imaging data and the second training medical imaging data, the training data further comprising, for each set of training imaging data, ground truth natural language text describing a change between the first training medical imaging data and the second training medical imaging data;
• and
• training the machine learning model based on the training data so as to minimize a loss function between the natural language text generated for the sets of training imaging data by the machine learning model and the corresponding ground truth natural language text for the sets of training imaging data.

A system for generating natural language text describing a change between first medical imaging data and second medical imaging data:

• a non-transitory memory device for storing computer readable program code;
• and
• a processor in communication with the non-transitory memory device, the processor being operative with the computer readable program code to perform steps including
• obtaining imaging data representative of the first medical imaging data and the second medical imaging data, or of a difference between the first medical imaging data and the second medical imaging data,
• inputting the imaging data into a first trained machine learning model to generate an image feature vector representative of the difference between the first medical imaging data and the second medical imaging data, and
• inputting data representative of the image feature vector into a second trained machine learning model to generate the natural language text describing the change between the first medical imaging data and the second medical imaging data.