Determining biomarkers from histopathology slide images

A computing device configured to identifying biomarkers in digital image of a Hematoxylin and eosin (H&E) stained slide of target tissue, the computing device comprising:

• one or more memories;
• and
• one or more processors configured to,
• receive the digital image to an image-based biomarker prediction system having one or more processors;
• separate the digital image into a plurality of tile images, where each of the plurality of tile images contains a different portion of the digital image;
• apply the plurality of tile images to a deep learning framework comprising one or more trained biomarker classification models, each trained biomarker classification model being trained to classify a different biomarker, wherein the deep learning framework comprises a multiscale deep learning framework;
• predict a biomarker classification for each of the plurality of tile images using the one or more trained biomarker classification models;
• from the predicted biomarker classifications of each of the tile images, determine a predicted presence of one or more biomarkers in the target tissue;
• and
• generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers,
• wherein to apply the plurality of tile images to the deep learning framework and to predict the biomarker classification for each of the plurality of tile images, the one or more processors are further configured to,
• apply each of the tile images to one or more trained deep learning multiscale classifier models, each trained deep learning multiscale classifier model being trained to classify a different tissue classification for each tile image and determine a tissue classification for each of the plurality of tile images, using the multiscale deep learning framework,
• identify, using the one or more processors, cells within the digital image using a trained cell segmentation model, and
• from the tissue classification determined for each tile image and from the identified cells within the digital image, predict the biomarker classification for each tile image.

A computing device configured to identifying biomarkers in digital image of a Hematoxylin and eosin (H&E) stained slide of target tissue, the computing device comprising:

• one or more memories;
• and
• one or more processors configured to,
• receive the digital image to an image-based biomarker prediction system having one or more processors;
• separate the digital image into a plurality of tile images, where each of the plurality of tile images contains a different portion of the digital image by performing an image tiling process by applying the digital image to a trained multiple instance learning controller that separates the digital image into the plurality of tile images;
• apply the plurality of tile images to a deep learning framework comprising one or more trained biomarker classification models, each trained biomarker classification model being trained to classify a different biomarker, wherein the deep learning framework comprises a single-scale deep learning framework;
• predict a biomarker classification for each of the plurality of tile images using the one or more trained biomarker classification models;
• from the predicted biomarker classifications of each of the tile images, determine a predicted presence of one or more biomarkers in the target tissue;
• and
• generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers,
• wherein the one or more processors are further configured to,
• provide each tile image to a tile selection process that infers a class status for each tile image in the H&E slide training image, and
• based on inferred class status, selectively discard tile images based on a tile selection criteria before applying the remaining plurality of tile images to the deep learning framework.

A computing device configured to identifying biomarkers in digital image of a Hematoxylin and eosin (H&E) stained slide of target tissue, the computing device comprising:

• one or more memories;
• and
• one or more processors configured to,
• receive the digital image to an image-based biomarker prediction system having one or more processors;
• separate the digital image into a plurality of tile images, where each of the plurality of tile images contains a different portion of the digital image by performing an image tiling process by applying the digital image to a trained multiple instance learning controller that separates the digital image into the plurality of tile images;
• apply the plurality of tile images to a deep learning framework comprising one or more trained biomarker classification models, each trained biomarker classification model being trained to classify a different biomarker, wherein the deep learning framework comprises a single-scale deep learning framework;
• predict a biomarker classification for each of the plurality of tile images using the one or more trained biomarker classification models;
• from the predicted biomarker classifications of each of the tile images, determine a predicted presence of one or more biomarkers in the target tissue;
• and
• generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers,
• wherein the one or more processors are further configured to,
• provide each tile image to a tile selection process that infers a class status for each tile image in the H&E slide training image, and
• based on inferred class status, randomly discard tile images before applying the remaining plurality of tile images to the deep learning framework.

A computing device configured to identifying biomarkers in digital image of a Hematoxylin and eosin (H&E) stained slide of target tissue, the computing device comprising:

• one or more memories;
• and
• one or more processors configured to,
• receive the digital image to an image-based biomarker prediction system having one or more processors;
• separate the digital image into a plurality of tile images, where each of the plurality of tile images contains a different portion of the digital image by performing an image tiling process by applying the digital image to a trained multiple instance learning controller that separates the digital image into the plurality of tile images;
• apply the plurality of tile images to a deep learning framework comprising one or more trained biomarker classification models, each trained biomarker classification model being trained to classify a different biomarker, wherein the deep learning framework comprises a single-scale deep learning framework;
• predict a biomarker classification for each of the plurality of tile images using the one or more trained biomarker classification models;
• from the predicted biomarker classifications of each of the tile images, determine a predicted presence of one or more biomarkers in the target tissue;
• and
• generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers,
• wherein the one or more processors are further configured to,
• receive a molecular training dataset for a plurality of training tissue samples, the molecular training dataset comprising RNA transcriptome counts from sequencing of a substantially similar sample associated with each training tissue sample;
• perform a clustering process on the molecular training dataset to identify one or more molecular data subsets each corresponding to a different biomarker;
• for each of the one or more molecular data subsets, receive a plurality of digital images of H&E stained training slides of training tissue samples corresponding to the respective biomarker to an image-based biomarker prediction system having one or more processors;
• and
• generate for each of the one or more molecular data subsets, one of the trained biomarker classification models, based on the plurality of digital images of the H&E stained training slides.