Deep learning model to predict data from an image

A computer system for predicting data from an image, the computer system comprising:

• one or more computer-readable non-transitory storage media configured to store computer program code;
• and
• one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:
• dividing code configured to cause the one or more computer processors to divide a hematoxylin and eosin (H&E) image into one or more patch images;
• compressing code of a first model configured to cause the one or more computer processors to compress spatial features corresponding to the one or more patch images;
• predicting code of a second model configured to cause the one or more computer processors to predict output molecular data corresponding to the compressed spatial features;
• and
• adjust parameters of the first model based on minimizing a first loss function corresponding to the compressed spatial features and adjust parameters of the second model based on minimizing a second loss function corresponding to the compressed spatial features,
• wherein the first loss function comprises a regression loss function corresponding to compressed feature expression images output by the first model based on the one or more patch images, and
• wherein the second loss function comprises a feature compression loss function corresponding to second output data, output by the second model, indicated by the H&E image based on the compressed feature expression images.

A non-transitory computer readable medium having stored thereon a computer program for predicting data from an image, the computer program configured to cause one or more computer processors to:

• divide a hematoxylin and eosin (H&E) image into one or more patch images;
• compress, using a first model, spatial features corresponding to the one or more patch images;
• predict, using a second model, output molecular data corresponding to the compressed spatial features;
• and
• adjust parameters of the first model based on minimizing a first loss function corresponding to the compressed spatial features and adjusting parameters of the second model based on minimizing a second loss function corresponding to the compressed spatial features,
• wherein the first loss function comprises a regression loss function corresponding to compressed feature expression images output by the first model based on the one or more patch images, and
• wherein the second loss function comprises a feature compression loss function corresponding to second output data, output by the second model, indicated by the H&E image based on the compressed feature expression images.