Systems and methods for processing of fundus images

A system for predicting a risk of cardiovascular disease (CVD) from one or more fundus images, the system comprising comprising one or more processors and one or more storage devices storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:

• processing one or more fundus images associated with an individual using a Quality Assurance (QA) set of one or more convolutional neural networks (CNNs) to determine whether the one or more fundus images are of sufficient quality for further processing;
• processing the one or more fundus images determined to be of sufficient quality for further processing using an eye-identification set of one or more CNNs (eye-ID CNN), to identify the one or more fundus images belonging to a single eye;
• processing the one or more fundus images using a plurality of risk contributing factor sets of one or more CNNs (RCF CNN), wherein each RCF CNN is configured to output an indicator of probability of the presence of a different risk contributing factor in each of the one or more fundus images,
• wherein at least one of the RCF CNNs is configured in a jury system model comprising a plurality of jury member CNNs, wherein each jury member CNN is configured to output a probability of a different feature in the one or more fundus images, and the outputs of the plurality of jury member CNNs are processed to determine the indicator of probability of the presence of the risk contributing factor output by the RCF CNN;
• producing an individual feature vector based on meta-information for the individual, and the outputs of the plurality of RCF sets of one or more CNNs;
• processing the individual feature vector using a CVD risk prediction neural network model to output a prediction of overall CVD risk for the individual, wherein the CVD risk prediction neural network model is configured to determine a relative contribution of each of the risk contributing factors to the prediction of overall CVD risk;
• and
• reporting the overall CVD risk, comprising reporting the relative contribution of the risk contributing factors to the overall CVD risk.

A computer program product for predicting a risk of cardiovascular disease (CVD) from one or more fundus images, the computer program product comprising a non-transitory computer-readable storage medium containing computer program code for:

• processing one or more fundus images associated with an individual using a Quality Assurance (QA) set of one or more convolutional neural networks (CNNs) to determine whether the one or more fundus images are of sufficient quality for further processing;
• processing the one or more fundus images determined to be of sufficient quality for further processing using an eye-identification set of one or more CNNs (eye-ID CNN), to identify the one or more fundus images belonging to a single eye;
• processing the one or more fundus images using a plurality of risk contributing factor sets of one or more CNNs (RCF CNN), wherein each RCF CNN is configured to output an indicator of probability of the presence of a different risk contributing factor in each of the one or more fundus images,
• wherein at least one of the RCF CNNs is configured in a jury system model comprising a plurality of jury member CNNs, wherein each jury member CNN is configured to output a probability of a different feature in the one or more fundus images, and the outputs of the plurality of jury member CNNs are processed to determine the indicator of probability of the presence of the risk contributing factor output by the RCF CNN;
• producing an individual feature vector based on meta-information for the individual, and the outputs of the plurality of RCF sets of one or more CNNs;
• processing the individual feature vector using a CVD risk prediction neural network model to output a prediction of overall CVD risk for the individual, wherein the CVD risk prediction neural network model is configured to determine a relative contribution of each of the risk contributing factors to the prediction of overall CVD risk;
• and
• reporting the overall CVD risk, comprising reporting the relative contribution of the risk contributing factors to the overall CVD risk.