US 11,893,732 B2 · Computer supported review of tumors in histology images and post operative tumor margin assessment

Abstract

A computer apparatus and method for identifying and visualizing tumors in a histological image and measuring a tumor margin are provided. A CNN is used to classify pixels in the image according to whether they are determined to relate to non-tumorous tissue, or one or more classes for tumorous tissue. Segmentation is carried out based on the CNN results to generate a mask that marks areas occupied by individual tumors. Summary statistics for each tumor are computed and supplied to a filter which edits the segmentation mask by filtering out tumors deemed to be insignificant. Optionally, the tumors that pass the filter may be ranked according to the summary statistics, for example in order of clinical relevance or by a sensible order of review for a pathologist. A visualization application can then display the histological image having regard to the segmentation mask, summary statistics and/or ranking. Tumor masses extracted by resection are painted with an ink to highlight its surface region. The CNN is trained to distinguish ink and no-ink tissue as well as tumor and no-tumor tissue. The CNN is applied to the histological image to generate an output image whose pixels are assigned to the tissue classes. Tumor margin status of the tissue section is determined by the presence or absence of tumor-and-ink classified pixels. Tumor margin involvement and tumor margin distance are determined by computing additional parameters based on classification-specified inter-pixel distance parameters.

Claims21

Dependent claims

A. computer apparatus for identifying tumors in a histological image, comprising:

a memory configured to store computer-executable instructions;
and
a hardware processor in communication with the memory, wherein the computer-executable instructions, when executed by the processor, configure the processor to:
generate, from an output image, a segmentation mask having areas occupied by individual tumors in the output image marked in the segmentation mask, wherein the output image is generated by a convolution neural network based on the histological image, and wherein the output image is mapped to the histological image and has one of a plurality of tissue classes assigned to each pixel of the output image representing classified tissue of the histological image;
compute statistics for each tumor marked in the segmentation mask;
apply a filter to the statistics of the tumors to edit the segmentation mask, wherein application of the filter selects and deselects tumors according to the filter to edit the segmentation mask to remove insignificant tumors;
score each tumor in the edited segmentation mask according to a scoring algorithm to assign a score to each tumor;
and
rank the tumors according to their score.

6 more elements

A non-transitory computer readable medium for identifying tumors in a histological image, the computer readable medium having program instructions for causing a hardware processor to perform a method of:

generating, from an output image, a segmentation mask having areas occupied by individual tumors in the output image marked in the segmentation mask, wherein the output image is generated by a convolution neural network based on the histological image, and wherein the output image is mapped to the histological image and has one of a plurality of tissue classes assigned to each pixel of the output image representing classified tissue of the histological image;
computing statistics for each tumor marked in the segmentation mask;
applying a filter to the statistics of the tumors to edit the segmentation mask, wherein application of the filter selects and deselects tumors according to the filter to edit the segmentation mask to remove insignificant tumors;
scoring each tumor in the edited segmentation mask according to a scoring algorithm to assign a score to each tumor;
and
ranking the tumors according to their score.

3 more elements

A method of identifying tumors in a histological image, the method comprising:

generating, from an output image, a segmentation mask having areas occupied by individual tumors in the output image marked in the segmentation mask, wherein the output image is generated by a convolution neural network based on the histological image, and wherein the output image is mapped to the histological image and has one of a plurality of tissue classes assigned to each pixel of the output image representing classified tissue of the histological image;
computing statistics for each tumor marked in the segmentation mask;
applying a filter to the statistics of the tumors to edit the segmentation mask, wherein application of the filter selects and deselects tumors according to the filter to edit the segmentation mask to remove insignificant tumors;
scoring each tumor in the edited segmentation mask according to a scoring algorithm to assign a score to each tumor;
and
ranking the tumors according to their score.

3 more elements

Dependencies are inferred from the claim text.

Computer supported review of tumors in histology images and post operative tumor margin assessment

Life of this patent

Computer supported review of tumors in histology images and post operative tumor margin assessment

Life of this patent

Abstract

Claims21