Processing fundus camera images using machine learning models trained using other modalities

A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• generating training data for training a fundus image processing machine learning model,
• wherein the fundus image processing machine learning model is configured to process one or more fundus images captured by a fundus camera to generate a predicted label,
• wherein each of the one or more fundus images are an image of a fundus of an eye of a patient,
• wherein the predicted label characterizes a health of the patient, and
• wherein generating the training data comprises:
• receiving a plurality of sets of one or more training fundus images captured by a fundus camera, each set corresponding to a respective patient, and each training fundus image in the set being an image of an eye of the patient corresponding to the set;
• receiving, for each of the plurality of sets, a ground truth label assigned to a different image of the eye of the patient corresponding to the set that has been captured using an imaging modality that is different from the fundus camera;
• and
• generating, for each set of one or more training fundus images, a training example that includes (i) the set of one or more training fundus images in association with (ii) the ground truth label assigned to the different image of the eye of the patient corresponding to the set that has been captured using the different imaging modality;
• training the fundus image processing machine learning model on the training examples in the training data to generate, by processing the sets of one or more training fundus images, predicted labels that match the ground truth labels associated with the sets;
• and
• providing data specifying the trained fundus image processing machine learning model for use in generating predicted labels for new fundus images.

One or more non-transitory computer-readable media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• generating training data for training a fundus image processing machine learning model,
• wherein the fundus image processing machine learning model is configured to process one or more fundus images captured by a fundus camera to generate a predicted label,
• wherein each of the one or more fundus images are an image of a fundus of an eye of a patient,
• wherein the predicted label characterizes a health of the patient, and
• wherein generating the training data comprises:
• receiving a plurality of sets of one or more training fundus images captured by a fundus camera, each set corresponding to a respective patient, and each training fundus image in the set being an image of an eye of the patient corresponding to the set;
• receiving, for each of the plurality of sets, a ground truth label assigned to a different image of the eye of the patient corresponding to the set that has been captured using an imaging modality that is different from the fundus camera;
• and
• generating, for each set of one or more training fundus images, a training example that includes (i) the set of one or more training fundus images in association with (ii) the ground truth label assigned to the different image of the eye of the patient corresponding to the set that has been captured using the different imaging modality;
• training the fundus image processing machine learning model on the training examples in the training data to generate, by processing the sets of one or more training fundus images, predicted labels that match the ground truth labels associated with the sets;
• and
• providing data specifying the trained fundus image processing machine learning model for use in generating predicted labels for new fundus images.